Walker, on loan from Chelsea, saved former U's striker Kevin Lisbie's spot-kick with 12 minutes to go, to help Colchester move six points clear of the relegation zone.
Gavin Massey headed Colchester in front after 20 minutes, after substitute Bradley Garmston had brilliantly picked him out in the area.
The U's doubled their lead four minutes later when Drey Wright finished another good attacking move, after Clinton Morrison had helped Brian Wilson's low cross into his path.
But Lisbie broke the offside trap to pull a goal back for the O's, eight minutes before the break.
Colchester boss Dunne said: "It was a wonderful save by Sam and he's been immense for us since he came here.
"We watched every penalty that Kevin Lisbie has taken in the last few years. We did our preparation and we looked at every penalty he had taken.
"He put most of them in the same place, so we said if Lisbie had a penalty to go that way. It was a very good save by Sam and he's been immense for us since he came here.
"It was a big victory for us and a big three points. It wasn't pretty and we looked very nervous. We were young and nervous and we made mistakes, but we were resilient.
"We're all a bit drained, but we will take the three points. The game-plan worked really well early on and we looked like we were going to score every time we attacked."