The club brought in two players on deadline day, highly-rated England goalkeeper Jack Butland from Birmingham and USA international winger Brek Shea from FC Dallas.
Butland's capture on a four-and-a-half-year deal is a coup, with the 19-year-old opting for Stoke over Chelsea, but he will not be an immediate asset to the Potters as he has headed straight back to Birmingham on loan.
Shea, 22, is also seen as one for the future, partly because he is still recovering from a foot operation, and Kemp admitted Stoke missed out on other targets.
Defenders Matthew Upson and Rory Delap left the club for Brighton and Barnsley respectively, leaving the Potters short in defence, especially on the left-hand side.
Kemp said: "We've brought in two excellent young players but they are both for the future.
"To be honest, the squad is unbalanced. We've been aware of that since the summer. We tried to correct it in the last window and didn't manage it, and the plan was to get through to this window and address it and unfortunately we haven't.
"So there is a bit of concern there. And we also let two senior players go out on the preface we were bringing players in, so in some respects you could argue we're a bit weaker than it might appear from the outside.
"A left-back is one - we are short of left-sided players in the club. We let Matty Upson go so there's another one we were hoping to replace.
"In each window, I think the way football's gone now you have to keep freshening the squad up. We didn't do it this time last year, and I felt we suffered for that a bit, we did in the summer window but we've missed the boat again a little bit."