Westwood was booked during the closing stages of last weekend's 2-1 home defeat against Liverpool for a challenge on Jordan Henderson.
However, replays showed the former Crewe player may have been fortunate to escape a red card as he was showing his studs when connecting with the Reds midfielder.
Villa boss Lambert, whose side visit Stoke on Saturday, said: "He's got the ball. I know what people are saying because of the way he's gone in but he has actually got the ball.
"If he had got the lad, then it's a different matter. You are not sitting telling me that Westwood is a dirty player.
"He's nowhere near that. He doesn't need to be dirty because he doesn't give the ball away."