The Shrimpers seemed to be heading for a sixth successive win and the top of the table when Kevan Hurst put them ahead just before the break.
But Richards' spot-kick six minutes into time added on handed the Robins a dramatic share of the spoils.
The Shrimpers made the most of those let-offs four minutes before half-time when Hurst fired home at the second attempt after being picked out by Barry Corr's downward header from John White's right-wing cross.
But the Robins replied right at the death, with Richards sending his late penalty into the left-hand side of the goal after Southend winger Anthony Straker had handled Jermaine McGlashan's header at the far post.
- Sports & Recreation
- Kevan Hurst
- the Robins