Argyle fell behind early on but then took command with two quick goals from Rommy Boco and Lewis Alessandra.
Jason Taylor's deflected shot made it 3-3 and Cheltenham pressed for a winner, but Plymouth held out and emerged 5-4 winners in the shootout.
"When you are 3-1 up we shouldn't have made it so hard for ourselves," Sheridan said. "We should have been in control and dominated at that stage but we went a bit sloppy.
"I won't be too hard on them because it was a good effort and we created chances, but full credit to Cheltenham, who tried to play as well and they worked hard to get something out of the game.
"I think we deserved to win it, but at 3-1 we should have been home and dry and we switched off."
Sheridan said he had no complaints about the sending off and admitted Cheltenham were unlucky not to be awarded a spot-kick in the first half when Guy Branston upended Byron Harrison.
- Sports & Recreation
- Curtis Nelson
- Hamza Bencherif