Palace opened the scoring through Glenn Murray, who bundled home after eight minutes but Forest levelled on the stroke of half-time thanks to Chris Cohen's superb finish from 25 yards.
Murray restored the advantage with nine minutes remaining with a precision, looping header, but Billy Sharp had the final say in added time, tapping in after his shot struck a post.
"I thought for 75% of the game we were a very good side and the lads are gutted because, like me, they feel we could have won the game," Holloway said.
"The lads are getting used to the way I want them to play and if we keep playing like we did for long periods of this game we will do very well.
"We got ourselves into a situation to win the match and we've ended up with one point instead of three - and that's frustrating.
"I was very proud of them, their work ethic was excellent and they played some exceptional football - as did Forest - in atrocious conditions.
"But at times, particularly at the end of the first half, we dropped deep and it cost us the first equaliser. Having said that I have to give credit to the strike from Andy Reid. He's got himself fit and he's a good player."