The German took charge of United on an interim basis for the last six months of the season following the sacking of Ole Gunnar Solskjaer last November and oversaw a sixth-placed Premier League finish.
Rangnick was expected to remain at Old Trafford for the next two years in a consultancy position but he was recently appointed the new Austria boss and it seems holding the dual roles is an unviable option.
"We would like to thank Ralf Rangnick for his efforts as interim manager over the past six months," a statement on United's website read.
"By mutual agreement, Ralf will now focus solely on his new role as manager of the Austria national team and will not therefore be taking up a consultancy role at Old Trafford.
"We would like to wish Ralf the best of luck in this next chapter of his career."
Erik Ten Hag has been tasked with reviving United's fortunes after they missed out on Champions League football next season.
A haul of 58 points was their lowest in the Premier League era.