Norwich debutant Gary Hooper stole the headlines as he scored twice, once in the 95th minute and the other the extra-time winner, but Watford let slip a two-goal lead after Javier Acuna and Davide Faraoni had struck either side of half-time.
The Sky Bet Championship side looked to be sailing in to the fourth round at that point but Norwich hit back.
Firstly, academy graduate Josh Murphy fired in off the crossbar before Hooper's glancing header in stoppage time forced an extra 30 minutes with the summer signing from Celtic ending his full debut by slotting home the decisive goal for a 3-2 win.
Zola was downbeat at full-time but believes his side will have taken a great deal of experience from the result.
"It is a crushing feeling of course," he said.
"We were winning a game and deserving to against Premier League opposition. But this game is about the attention you put in at crucial moments and that is the learning side for us.
"The first goal was a cracking one so we couldn't do anything about it. The second one we should not have let the delivery get into the box. That's the only thing I can complain about.
"The first half they started very well, then after the goal it changed, we found more courage. Second half was excellent I have to say, I thought we were controlling the game."
Canaries manager Chris Hughton was pleased with the character shown by his side as they bounced back from a disappointing defeat at home to Aston Villa on Saturday.
"I knew once we got the first goal that we would have great momentum but of course when it is that late on you don't manage to do it on most occasions," he said.
"We are all conscious that the result on Saturday wasn't the result we wanted. There was some good offensive play that we wanted to take in to today and on the back of that disappointment it was even better."
Hooper had to wait to make his first start for Norwich after the striker suffered a shin injury during a pre-season friendly but Hughton was pleased with his impact and with the fact he managed to get through the whole 120 minutes unscathed.
"He is a finisher," he said.
"I'm probably as delighted that he has got 120 minutes under his belt than I am with the goals. He is used to scoring but with all strikers it is about the service they get and showing a little bit of quality when they can.
"Even in that second half of extra time I was looking at how he was running and he was fine."