Lee Brown's penalty three minutes before the break set Rovers on the way to victory before Rotherham debutant Claude Davis received his marching orders for his second yellow card in first-half added time.
And worse quickly followed for the promotion favourites, who have now dropped to ninth in the table. Substitute Ellis Harrison doubled Rovers' advantage in the 53rd minute following a great solo run, and eight minutes later Joe Anyinsah broke down the right and made it three.
By the time substitute David Noble gave Rotherham back a little pride after 73 minutes, many of the Millers fans, who had spent the morning clearing snow off the pitch, had already left the stadium.
"If you are going to make individual errors like we made, you are going to lose football matches," said Evans. "I don't think there is any football manager in the world that can legislate for the errors that gave Bristol Rovers three goals.
"We didn't have the spark and the passion but we still dominated on the ball. I am not going to fault the team for their effort in the second half because we played for over half of the game with only ten men.
"We felt there was a lack of effort in the first half, but that changed in the second half because for the first 15 minutes we dominated with ten men, but if you make sloppy mistakes, if you make huge errors, you are going to get punished."
A delighted Rovers manager John Ward said: "It's good to be off the bottom but we want to be better than that so it doesn't stop there. That's the first stage. We are trying to move a bit further if we can do that.
"We've had a nice little run, we've had five games and lost only one of them and we are still at the bottom because other teams have done well but it's nice now that there is a bit of movement to get above one or two of them."