Santi Cazorla fired 10-man Arsenal to victory as Arsene Wenger saw his side survive a late onslaught.
The Spaniard struck 10 minutes before the break to give the Gunners a deserved lead as Black Cats keeper Simon Mignolet turned in another fine display to otherwise keep them at bay.
However, the 62nd-minute dismissal of defender Carl Jenkinson, who was only included after Laurent Koscielny injured himself during the warm-up, for a second bookable offence left the Londoners hanging on in front of a crowd of 46,402.
Steven Fletcher had earlier passed up a glorious opportunity to level but, even after Jenkinson's departure, Arsenal had chances and Theo Walcott was unfortunate to see a 75th-minute effort come back off the post.
However, goalkeeper Wojciech Szczesny spared Per Mertesacker's blushes three minutes later when he blocked Fletcher's shot after he had charged down the defender's attempted clearance and the visitors managed to see the game out amid a frenzied conclusion.
O'Neill said: "It was a great game from start to finish. The tempo was very high early on and it remained so. Obviously Arsenal, a top-quality team, caused us a lot of problems in the first half but I also thought we looked dangerous as well.
"But in the second half, we just came roaring at them, and that was even before the man had been sent off.
"Seriously, looking back at a couple of the chances, I just don't know how we didn't score."
O'Neill's disappointment was compounded by the loss of full-back Danny Rose with a recurrence of his hamstring problem, while skipper Lee Cattermole lasted only 45 minutes on his return from a knee injury, although his premature departure was due in part to an early yellow card.