Striker Eliot Richards, who joined on loan from Bristol Rovers last week, came off the bench to bag an 87th-minute winner.
Exeter initially went in front after 10 minutes when Arron Davies' header from Daniel Bentley's poor clearance sent Jimmy Keohane through on goal and the striker made no mistake with an assured 12-yard finish.
However, two minutes later, the Shrimpers were back level when Mark Phillips headed home a right wing free-kick from Ben Coker.
Phillips, who had gone 25 games without a goal, popped up again in the 22nd minute to nod in a corner from Kevan Hurst.
However, with 82 minutes on the clock, the Grecians drew back level in stunning circumstances when Liam Sercombe sent a well-struck 25-yard shot flying in to the top right-hand corner
And Exeter claimed the spoils as Richards cut in off the left flank, beat a couple of men and netted with a low shot.
"Of course I'm disappointed and, after being 2-1 up we should have seen out the game, but Exeter deserved to win," said Brown.
"They played with the verve and tempo we couldn't ever seem to get in our play and the whole place just seemed to be a little bit flat.
"We've gone off the boil a bit since losing to Hull and we need to get back out there now, roll up our sleeves and get stuck in rather than lying back and thinking of England."