The club brought in two players on Thursday, highly-rated England goalkeeper Jack Butland from Birmingham and USA international winger Brek Shea from FC Dallas. However, defenders Matthew Upson and Rory Delap left the club for Brighton and Barnsley respectively, leaving the Potters short in defence, especially on the left-hand side.
Kemp said: "We've brought in two excellent young players but they are both for the future. To be honest, the squad is unbalanced. We've been aware of that since the summer. We tried to correct it in the last window and didn't manage it, and the plan was to get through to this window and address it and unfortunately we haven't."
Kemp continued: "So there is a bit of concern there. And we also let two senior players go out on the preface we were bringing players in, so in some respects you could argue we're a bit weaker than it might appear from the outside.
"A left-back is one - we are short of left-sided players in the club. We let Matty Upson go so there's another one we were hoping to replace.
"In each window, I think the way football's gone now you have to keep freshening the squad up. We didn't do it this time last year, and I felt we suffered for that a bit, we did in the summer window but we've missed the boat again a little bit."
Butland's signing has inevitably led to speculation that Asmir Begovic will be sold in the summer, with the Bosnia international keeper reportedly attracting the attentions of the Premier League's biggest clubs. But Kemp stressed the opportunity to sign one of England's brightest prospects was simply too good to turn down.
He said: "Jack's obviously one of the biggest up-and-coming goalkeepers in the country. It's a bit of a coup for us to get him to come here and we're delighted with that. We have a core of young players here and I think players look at it and see Stoke's a good place to go and improve their career."
The Potters did hold on to strikers Peter Crouch and Kenwyne Jones, who were heavily linked with QPR and Swansea respectively, and the same squad that drew 2-2 with Wigan on Tuesday is likely to travel to Arsenal for Saturday's Barclays Premier League clash.