Premier League - RVP hat-trick breaks Southampton's hearts
United trailed the match with just three minutes to go before a Van Persie double made up for an earlier penalty miss from the Dutchman when he was caught out for trying a cheeky chip.
His winner in stoppage time though was his 100th Premier League goal and made sure that Sir Alex Ferguson's 1,000th league game in charge of Manchester United would be a memorable one.
It was tough luck on Southampton though who played superbly for most of the contest and were twice in front but they remain without a point in this their first Premier League season for seven years.
United started the match the brighter side and dominated possession in the early exchanges but Southampton then started to come into it after a nervy opening 10 minutes and in the 16th minute they took the lead.
Shinji Kagawa, who struggled with the pace of the game at times, was caught in possession in midfield. The ball was eventually played out to Jason Puncheon on the left and he stood up a fine cross to the back post. Waiting there was Rickie Lambert who peeled onto the mismatched Rafael before guiding his header into the corner of the net.
United levelled just seven minutes later courtesy of a trademark finish from Van Persie. The Dutch striker took Antonio Valencia's cross from the right on his chest at the back post and then drilled the ball low into the opposite corner. He had all the time in the world to do so too after Nathaniel Clyne had slipped.
It wouldn't be the only time Clyne would lose balance in such a way either. He was lucky later in the half when Patrice Evra was able to get a header on target after Clyne again lost his footing with Kelvin Davis this time getting him out of jail with a decent save.
It was end to end stuff at times in the first half with both sides looking capable of adding more goals.
Adam Lallana was popping up everywhere for Southampton and was at the hub of most of their attacks while on a couple of occasions United nearly caught out Southampton with long balls over the top aimed at Van Persie.
The second half was similarly breathless and St Mary's was rocking again in the 55th minute when Southampton retook the lead.
Again it came from a header; this time Lambert was the provider as he curled a right-footed cross to the back post, Evra then slipped in similar fashion to Clyne earlier, allowing Morgan Schneiderlin to run in unchallenged and head into the corner of the net.
Southampton gifted United the chance to level in the 68th minute when, for the second time in the match, Kelvin Davis saw a poor pass out of his goal pounced upon by United. The ball was eventually played into Van Persie in the box and Jos Hooiveld foolishly brought him down from behind for a penalty.
But Van Persie showed he is not perfect with an ill-advised spot-kick. He tried to chip it into the corner in a lethargic, almost Panenka style, but Davis made up for his error by getting a hand on it and pushing it wide.
The game buzzed on with intensity and up until Nigel Adkins decided to take off his three dangerous attackers – Lallana, Lambert and Puncheon – in the final 15 minutes, Southampton looked as likely to score a third as United an equaliser.
However, United got on top of proceedings in the closing stages and Van Persie tapped home the equaliser from close range in the 88th minute after Rio Ferdinand had headed a Rafael cross off the post.
That seemed harsh enough on Southampton but the true dagger was to come in the second minute of stoppage time when Van Persie peeled away from the front post to angle a header superbly into the opposite corner from Nani's fine corner.
It was only Van Persie's second start in a United shirt but he has already netted four goals to take his total Premier League tally up to 100.
Fitness permitting there are sure to be many more goals to come from the Dutchman in a United shirt, but few are likely to be as dramatic as his late winner here.