Dunne's side earned their first away win since the opening day of the season and ended Oldham's nine-match unbeaten home run.
They made a flying start thanks to hapless Oldham defender Ellis Plummer, who slid the ball into his own net after eight minutes when trying to clear Craig Eastmond's left-wing cross.
Plummer, on loan from Manchester City, was again at fault for Colchester's second goal after 32 minutes as he allowed Dominic Vose to set up Jabo Ibehre for a simple tap-in.
Colchester's win followed a 4-0 home drubbing by Notts County last week and Dunne said: "We were outstanding, fantastic.
"The players had to do it for themselves and for the fans and that was just the response I wanted from them.
"It's a wonderful win, fully deserved, and we could have scored more if we had taken our chances.
"Tactically and technically we were very good, but it's more about the players believing in themselves and responding in the way they did.
"We've been picking up resilient points but not enough wins, so we've said 'let's stop talking about it - let's just do it'.
"We controlled the tempo from start to finish and this gives us a big lift going into the festive period.
"We showed what we are capable of and I believe there will be more of that.
"It might not happen consistently and we need the same mentality at home, but it will happen."