James Gray had put Stanley into a 58th-minute lead before Shrimpers defender Ryan Cresswell saw red for tangling with the Stanley goalscorer near the dugouts as they jumped for a header.
But struggling Accrington could not hold on for a much-needed win as Hurst lashed home in the 82nd minute for high-flying Southend.
"They didn't really cause us problems but then you have to hold your hands up with goals like that," said Richardson.
"You would rather they scored like that than through a mistake by us. But it shows how good a performance it was by us that Southend will be rubbing their hands that they got a point.
"I am so proud of the players and, if we continue to put in performances like that, we will win a lot of games and be okay. You could see that, with the couple of additions we made in January, we are stronger and things are beginning to come together. We are a good team and we showed Southend that."