Nineteen-year-old striker Caldwell struck a stoppage-time winner at Tynecastle to hand Pat Fenlon's side a 2-1 win - their first Clydesdale Bank Premier League victory at the home of Hearts in four years.
The Gorgie victory was the second Hibs have enjoyed over their old foes this term after they knocked them out of the William Hill Scottish Cup in January.
This season's three other SPL meetings ended in draws, leaving Hearts unable to muster a league victory in the derby for the first time since the 2000/01 season.
And Caldwell says it is those facts which show the Leith side can now rightly claim to be top dogs in the city.
He said: "It was an unbelievable feeling to score the winner in a derby. Three quarters of the stadium went silent and our end has just erupted.
"It was amazing seeing all the passion and emotion on the fans faces.
"That's us ended the season unbeaten against Hearts now and it's a great achievement. The tide is turning.
"The balance of power is beginning to shift. It hasn't shifted completely. Just because we have gone one season undefeated does not make it permanent but we are getting there."
Caldwell hopes his strike, his second for the club, will boost his chances of a starting slot.
"I've got two games left and I'm trying to do my best," he said. "If that gives the manager a headache then that's great as obviously every player wants to take part in a cup final."