Patrick Bamford squandered a golden opportunity for the Dons in the opening three minutes.
And it proved a key moment as Colchester took a 20th-minute lead.
Magnus Okuonghae missed a good chance for Colchester early in the second half.
And the Dons reduced the deficit 10 minutes after half-time when Spence nodded in from a corner, seconds after Sam Walker had brilliantly denied substitute Daniel Powell.
But Colchester sub Macauley Bonne wrapped up the win with 12 minutes to go with a fine left-foot strike which flew inside the near post.