The 19-year-old struck a stoppage-time winner at Tynecastle to hand Pat Fenlon's side a 2-1 win - their first Clydesdale Bank Premier League victory at the home of Hearts in four years.
The Gorgie victory was the second Hibs have enjoyed over their old foes this term after they knocked them out of the William Hill Scottish Cup in January. This season's three other SPL meetings ended in draws, leaving Hearts unable to muster a league victory in the derby for the first time since the 2000/01 season.
And Caldwell says it is those facts which show the Leith side can now rightly claim to be top dogs in the city. He said: "It was an unbelievable feeling to score the winner in a derby. Three quarters of the stadium went silent and our end has just erupted. It was amazing seeing all the passion and emotion on the fans faces.
"That's us ended the season unbeaten against Hearts now and it's a great achievement. The tide is turning. The balance of power is beginning to shift. It hasn't shifted completely. Just because we have gone one season undefeated does not make it permanent but we are getting there."
But Hearts defender Darren Barr rubbished those claims and believes it will take "years" of success before Hibs can claim to be kings of the capital.
He said: "No I don't think the balance is shifting. That's just talk. If you watched that game you will have seen there was nothing in it. How many chances did they create really? It would take them years of finishing above us before they can say they are the dominant side."
Hibs were better in the first half but fell behind as Barr rammed home a loose ball in the box a minute before half-time. Visiting keeper Ben Williams failed to read Kevin McHattie's swerving corner and after Michael Ngoo saw his header blocked on the line by Tom Taiwo, Barr pounced for the opener.
However, that lead lasted just two minutes into the second half when Leigh Griffiths netted his 28th goal of the season with a stunning free-kick that skimmed the Hearts wall before flying into the top corner of Jamie MacDonald's net. With a fourth SPL draw between the sides looking increasingly likely, Caldwell swept home to secure a dramatic win.
Hibs now have their rearranged trip to Kilmarnock on Wednesday night and round off their SPL campaign at home to Dundee before they can focus on facing champions Celtic in the Scottish Cup final on May 26.