Jason Taylor's deflected shot made it 3-3 and Cheltenham pressed for a winner, but Plymouth held out and emerged 5-4 winners in the shootout.
"We responded well and maybe our season has finally started," Yates said. "We played some decent stuff, but conceded really poor goals again.
"We have to defend better, we kept 20 clean sheets last year through hard work and we are not doing enough at the minute.
"We can't keep shooting ourselves in the foot, but we responded in the right manner."