Burton were then awarded a spot-kick of their own in first-half stoppage-time when Russell Penn was adjudged to have handled Billy Kee's shot and the striker stepped up to restore hope for the Brewers.
Rory Delap levelled in the second half as it finished 2-2 at the Abbey Business Stadium, with Yates left seething.
"I spent two hours at a managers' meeting at Walsall recently, with half an hour spent on handball decisions and how the referee has given that one against Russell Penn I will never know," Yates said.
"We should have had one too, but he booked Terry Gornell for diving and he got them back in the game.
"It was a clear penalty because there was a noise and a contact. Maybe the referee was ring rusty, but Burton got away with it and credit to them for fighting back.
"Their first goal deflated us for some reason and the second goal from a set piece was pretty awful.
"It was two decent teams and we are disappointed the referee has influenced the game."
Burton boss Gary Rowett admitted he was relieved to hit back for a point, but he felt his team could have gone on to take all three after a much-improved second-half showing.
"I am relieved we managed to come back from 2-0 down at a place like Cheltenham on the first day of the season, but I am disappointed in a way in the end that we didn't win the game," he said.
"We had some half chances in the box and one or two off Rory's throws if we'd anticipated them a bit better. A draw was probably fair in the end and I was pleased with the response.
"The key moment was the penalty because that gave us a chance to galvanise after not pressing or looking after the ball in the first half, when I thought we were a little bit tepid."
Cheltenham lost new signing Cureton to a shoulder injury sustained as he slotted the ball into the net when young goalkeeper Jordan Pickford, on loan from Sunderland, flapped at a long clearance from his opposite number Scott Brown.
His replacement Harrison doubled Cheltenham's advantage in the 32nd minute, firing low into the net after Jermaine McGlashan's shot hit Marcus Holness.
Cheltenham were on top for most of the first half, but Burton's penalty provided the spark they needed.
They pulled level 11 minutes into the second half when the excellent Delap marked his debut with a goal from Alex MacDonald's corner.
Sam Deering hit the bar for Cheltenham, while Pickford denied Harrison a second with a fine stop as honours between last season's beaten play-off semi-finalists finished even.