Greg Halford headed the home side in front, but Scott Wootton levelled things up before Elliott Ward - one of two loan signings this week - powered home from a corner in the last 10 minutes to hand Forest victory.
Coupled with results elsewhere, it leaves Peterborough in the relegation zone and Ferguson was unhappy with his side's defending.
He said: "To lose two goals and the game to two set-pieces is very disappointing because I thought a draw would have been a fair result.
"I am repeating myself a lot this season but I told the players afterwards that it was another game we should not have lost but it was frustrating to concede the way we did.
"Forest don't have too much height in the team for a Championship side but their two tallest players have got the goals from headers.
"Things went against us at times. I have seen the first goal and I'm still not convinced that it was over the line but the referee's assistant decided from 40 yards away that it was.
"We should have also had what I thought was a blatant penalty but there's no point in talking to officials about things like that."