The 48-year-old former All Blacks boss joined the Aviva Premiership club at the end of November, having previously worked for them on a consultancy basis, as the Sharks battle to avoid relegation from the Aviva Premiership.
Mitchell guided the club to just their second league win of the season against Worcester on Friday night, but Sale have confirmed he has left his role, leaving Steve Diamond, Bryan Redpath and Pete Anglesea to lead the Sharks' battle against the drop.
Owner Brian Kennedy said in a statement: "John was brought in on a consultancy basis with a view to the possibility of being employed in a longer-term position within Sale Sharks.
"He has had to return to South Africa for personal reasons which we respect. We are grateful for his valuable and insightful contributions albeit for a shorter term than envisaged.
"Steve Diamond, Bryan Redpath and Pete Anglesea will continue to coach the side, implementing many of John's ideas, and building on last night's excellent performance."