The Northern Irishman thought he had clawed his way back to trail 9-7 after holing a tricky final black in the 16th frame at full stretch only to see the white ball gather enough pace to land in a baulk bag.
With Cao a couple of points ahead, it was enough to condemn Allen to a shock defeat.
"The white was nowhere near the pocket until it took a big bounce, " said Allen. "It summed up the way the match went, but I didn't perform and he deserved to win.
"I maybe underestimated him a bit early on. And to give someone like him a three-frame lead is not really good enough. This turns a decent season into a mediocre one. I never got anything going out there. Full credit to Cao.
"If ever there is a ranking position that is false, that would have to be it. "
World number 81 Cao had never reached the last 64 of a tournament, but is now in the last 16 of the sport's blue-chip event with £16,000 guaranteed if he loses in the next round. In his first season on Tour, Cao, 21, has accumulated only £8,908 in earnings.
The highlight of Cao's afternoon came with a 108 break in frame 11 and a run of 113 in the 14th frame while Allen made 101 in the penultimate frame of the match.
Defeat will come as a bitter disappointment to Allen - a losing finalist to Judd Trump in December's UK Championship. He was rated among the favourites to win the event having clinched his first ranking tournament at the Haikou World Open in March.
Allen's best run at the World Championship remains a losing semi-final to John Higgins in 2009.
In the afternoon's other match, 2005 world champion
Playing carefree snooker and aided by some uncharacteristic Allen mistakes, Cao took the opening three frames, a break of 69 following a 7-0 to pile the pressure on the Ulsterman. A break of 86 from Allen reduced the arrears. A very cagey fifth saw Cao edge it with a decisive green along the rail but Allen hit back to reduce the arrears before an ambitious red let Cao in to clear to go 5-2 ahead.
Allen hit back to take a scrappy ninth and found his form in the decider to knock in an 88 and trail by just one overnight.