Danny Gabbidon's deflection gave Palace an early lead before Steven Fletcher came off the bench to equalise on his first appearance for five months.
But the hosts hit back through a Dwight Gayle penalty, with Sunderland skipper John O'Shea sent off for bringing down Jason Puncheon, and a superb injury-time goal from Stuart O'Keefe.
There had been suggestions Holloway was close to losing his job after defeats against Tottenham and Stoke but he was delighted to see his side pick up what could prove to be a vital win.
"That is not the word I would like to use," he said when asked if he was relieved to break Palace's points duck.
"All you can do is work on your performances. They scored and we didn't wobble - we had a little spell when they put Fletcher on but that was a commanding performance. I was so pleased, it is not relieved.
"I was very, very pleased with the determination, the work rate and the performance, some of that passing was nice to watch."
Holloway has been open in his quest to add more players to his squad before the transfer window shuts on Monday evening and he confirmed at least two are now set to move to Selhurst Park: Huddersfield's Jack Hunt and Reading's Jimmy Kebe.
He said: "We've got Jack Hunt and Jimmy Kebe [coming in] - we are trying to sign another four, maybe even five, but I can't tell you an update on it."