Rodgers' men went in at the break 2-0 up thanks to goals from Raheem Sterling and Daniel Sturridge, and might have been much further ahead, such was their dominance, with Steven Gerrard and Daniel Agger both hitting the post.
But Sky Bet League One County stunned their Barclays Premier League hosts in the second half as Yoann Arquin scored a header and then, after Kolo Toure had hit the post at the other end, substitute Adam Coombes converted to force the additional period.
An injury to Toure saw Liverpool reduced to 10 men in extra time and the visitors then had penalty appeals rejected when Enda Stevens went down in the box under the attentions of Sterling.
Moments later, Sturridge made it 3-2 and substitute Jordan Henderson then wrapped up the victory for the hosts.
Asked after the final whistle what his feelings were about the contest, Rodgers said: "A bit of everything really.
"I have to give credit to the players.
"A year ago we might have lost that game, but I think we showed tremendous character, especially having gone down to 10 men.
"We controlled the game at that stage - we showed we wanted to win the game and get the goals, and we did that.
"To be 2-0 up and then give away the goals we did is obviously disappointing.
"But I can only commend the two groups really - my players for the brilliant character they showed to keep going and fighting, and also you have to give big credit to Notts County because they were 2-0 down, we had hit the post and should have scored more, but they stayed in it.
"You have seen it - no matter what level you are at, when you get a goal back to make it 2-1, you are always in with a chance of getting something and they forced extra-time.
"Chris (Kiwomya, the Notts County manager) can be very proud of his team. I'm sure it was a very entertaining game to watch."