After securing just five points in 2013, the Gunners were set for a second successive home draw until 12 minutes from time.
Substitute Lukas Podolski got the goal, although it was Geoff Cameron who did the damage by deflecting the German's free-kick past his own goalkeeper.
Arsenal remain sixth in the standings after the 1-0 win but cut the gap on fourth-place Tottenham to a single point - much to Wenger's relief.
"I said already before the West Ham and Liverpool games, that we are in a position where we cannot drop points," he said. "We dropped two points against Liverpool but we couldn't do it again.
"What is more important is the quality of performances is there. The spirit is there. The target to be in the top four and to focus on the two cups we are still in is very important now as we have strengthened our belief and our confidence."
The win was only Arsenal's second in the league since the turn of the year but one Wenger believes his side deserved.
"They made it difficult for us," he said. "They were organised, defended very deep.
"We didn't find the space but we kept going and, while we are a bit lucky with the goal, we deserved this win.
"We had chances, their 'keeper made a lot of saves. We played in their half a lot and overall I think we deserved to win the game."