The Gills were two-up within 13 minutes as former AFC Wimbledon striker Danny Kedwell scored a quick-fire brace, both from the penalty spot.
The first penalty came after 11 minutes after defender Tom Flanagan fouled Cody McDonald, and the same players were involved in the second penalty decision 120 seconds later - following which Flanagan received a straight red card.
Bamford headed in from Danny Green's free-kick after 26 minutes, before he coolly converted from close range on the half hour mark to level things up.
Bamford nearly scored an impressive first-half hat-trick, but agonisingly for the striker he was millimetres away from diverting Izale McLeod's teasing cross past Gillingham goalkeeper Stuart Nelson.
Gills' winner came after 59 minutes as Charlton loanee Hollands latched onto Michael Harriman's looping cross to head the hosts to their first home victory since April 6.
Gillingham manager Martin Allen said: "It was that strange situation: 2-0 up, playing against 10 men.
"Everybody who has watched football over the years has seen that the 10 men seem to play better. I don't know why, but people drop off. MK Dons play good football, and it was always going to be a difficult fixture.
"You've got to give the lads a lot of credit. After it went to 2-2, at half time I made a conscious decision to chat to the players. I asked them: 'what are you going to do? Buckle and fold, or use the training and preparation we've done and make something happen to win it?'
"They went up another gear, we had numerous chances to have won by more and I think the players were immense.
"It's just two wins, still. We've got to make this team powerful, strong and disciplined. I have total faith in the players, they can all do it.
"I was upset after the Bradford game [on September 21]. The buck stops with me; it's my responsibility to make things better. You win a championship and you're a god, the next you're a dog. I thrive on that."