But substitute Armand Gnanduillet headed home a Roberts corner in the fifth minute of added time to end Southend's 100% start in Sky Bet League Two and send the Spireites joint-top of the table.
Shrimpers boss Phil Brown felt his were the better side despite defeat at the Proact Stadium.
He also bemoaned the amount of injury-time at the end of the game, which culminated in the Spireites snatching victory. But Richardson felt the amount of injury-time added on at the end was about right.
"There were a few aspects of our game that were not great but the result is fantastic," said Richardson. "It was probably the best time to score and the team we played against are very strong so it's a great win.
"To be honest I thought there would be around about that (much time added on) with the two goals and the six subs being made - and Southend slowed the game down a touch which the officials were aware of so when the six minutes went up were were happy with that because we are at home and wanted to win the game."
Richardson added: "I said at half time not to get frustrated because we were getting frustrated for no reason. Phil Brown's teams are very organised and it's going to take a bit of brilliance to open the game up and it certainly did.
"It was a very, very poor goal to concede from our part but we will reflect on that and the lads will learn from it and learn quickly. "