The Welshman netted a first-half double, including a 25-yard stunner, as the Glovers returned to winning ways at Huish Park after two consecutive draws.
Oldham's miserable day was compounded in the second half when substitute Robbie Simpson was sent off, before James Hayter and Paddy Madden added late goals to make the final score 4-1.
Yeovil boss Gary Johnson was delighted with his side's performance and described Williams' second goal as a goal of the season contender.
"To score four goals at home after being a goal behind so early is really pleasing," said the Londoner. "It was a very important victory. There were some quality bits of play and one quality goal - which must be one of the goals of the season.
"We were disappointed that they scored with their first attack but the good thing about that is we had 87 minutes to get the game back.
"Gavin [Williams] was playing well even before his second goal, but he went and beat a couple of players and it all opened up like the Red Sea.
"He hit it with real venom and it went right into the top corner - it was a fantastic goal and something like that lifts players, management, supporters and directors. We all went up as one."
Johnson added: "We probably could have scored one or two more at the end, but it was a great performance and a great game for our supporters.
"I have to credit our new groundsman for getting the game on, which was a credit to the whole football club."