Harding grabbed the crucial goal in the 88th minute as the Cobblers made it seven wins on the bounce against an improving Rovers side.
Ward said: "We're disappointed. We deserved a point and I think even the most biased supporters would agree with that.
"For a team near the bottom I think we deserved to get something out of the game but that's the way it goes.
"We defended very well all night with their threat at set-pieces so it was very disappointing to concede so late in the game. But I've told the players that if you're going to get beaten that's the way to get beaten, going right to the end of the game.
"They are annoyed that they let it slip at the end. If you can't win the first ball you hope you can win the second one but we didn't. We dealt it with it for most of the game but not when it mattered."