The Chiefs started slowly and trailed 15-0 at the break, but came storming home to snatch a two-point lead when big prop Ben Tameifuna barged over in the 73rd minute.
Waratahs flyhalf Bernard Foley laid off a delightful short pass to Ulugia, however, who thundered 10 metres to cross near the posts in the 77th minute and break the Chiefs' hearts on a gusty night at Allianz Stadium.
The victory was the Waratahs' fourth from eight matches, keeping their playoff hopes alive at the halfway mark of the season.
"In that second half we just slackened off again ... but we're really happy with the result," NSW captain Dave Dennis said in a pitchside interview.
"We'll take some great momentum from this win tonight."
The Waratahs were switched on from the outset, opening the scoring with a penalty goal in the third minute and dominating possession and territory.
Wallabies back Adam Ashley-Cooper made the Chiefs pay for slack defending when he burst through a gaping hole before passing outside to winger Cam Crawford who cantered over for an easy try in the 15th minute.
The Chiefs were rocked by a foot injury to fullback Andrew Horrell who had to be taken off on a stretcher, but held firm until a sublime passage of play resulted in the Waratahs' second try two minutes before halftime.
Flyhalf Brendan McKibbin found space in front of the posts but chipped a kick toward the left corner where soaring fullback Israel Folau caught the ball between two defenders and glided five metres over the line without breaking stride.
Replacement back Gareth Anscombe blew the Chiefs' lone scoring attempt for the half when he shanked a penalty kick wide after the siren to leave the visitors trailing 15-0 at the break.
The Waratahs took the pressure off a few minutes after the re-start when lock Sitaleki Timani was sent off for a high tackle on substitute back Tim Nanai-Williams.
The Chiefs lifted their intensity and were rewarded in the 48th minute when the dangerous Aaron Cruden slipped past Ashley-Cooper with a feinted pass and fed Anscombe on the left wing.
Anscombe bounced off one tackler and bolted over the line for the Chiefs' first score, and after slotting another two penalty goals, brought the visitors within five points by the 62nd minute.
Attacking in waves, the Chiefs drove towards the line in furious bursts, and when Tameifuna spun around to plant the ball by the posts, appeared set to close out victory.
But Ulugia emerged an unlikely hero to snatch back the lead and the Waratahs defended grimly to seal the win ahead of a tough tour of South Africa.
The Waratahs' win was soured slightly by a hamstring injury to winger Drew Mitchell, who left the ground after only seven minutes and sat out the rest of the match.