After Billy Kee's opener from the spot had been cancelled out by Ricky Holmes just before half-time, Barnet won a penalty of their own eight minutes into the second half when the ball seemingly struck the hand of Jack Dyer in the area.
But Rowett was adamant it should not have been awarded, with Holmes going on to convert and then complete his hat-trick prior to Jacques Maghoma's late consolation, again from the spot.
"I think I would be better off not commenting too much on the officiating but it's clearly had a huge, huge influence on the game," Rowett said.
"Firstly the corner that was given in the lead up to the goal, it's not hit any of our players and they gave it as a corner.
"And then he's given it as a penalty when Jack (Dyer) had both hands down by his side, and the ref signalled that he had them both up in the air so you can understand our frustrations.
"You can read into that what you will but Jack is an honest player and he says his arms weren't up in the air and I believe him.
"We had lots of chances after that but I felt any chance to build any sort of momentum was killed by that penalty. Perhaps we should have responded better in some areas but it was a sucker punch and we didn't recover from it."
After Rowett had voiced his dissatisfaction to the referee in the wake of the penalty decision he turned his anger towards the Barnet bench, leading to joint-Bees head coach Edgar Davids refusing his offer of a handshake after the final whistle.
"If you are a manager you have to be above the team and you can't disrespect someone on the sideline," the former Holland international said. "I don't pretend. If you don't show respect I am not going to show you any. He crossed the line during the game."