With a new £7 billion TV contract due to start next season, top flight clubs will have even more cash to scour the planet and bring in the best global talent. Charlton's own Manchester United are one of the obvious beneficiaries, but he acknowledges England are paying the price.
"The number of players we have to choose from is ridiculous," said Charlton. "Didn't Fabio Capello say there were only 94 footballers he could select out of the whole of the league? You can't expect them all to be world-class players."
He added: "I would like to think that one day we will be favourites going into a World Cup, but we are not anymore. I am not happy about it, but I don't know how you can solve it either."
So bad has the situation become, Charlton has been forced to admit previous bullish views about how England would fare at major tournaments were made out of 'obligation'.
Speaking at the Soccerex Conference in Manchester, Charlton said: "When I am asked to give my opinion, I always say yes (we can win the World Cup) but it is a fool's errand.
"People wonder whether we will ever win it again. I worry about that as well. You need good players. But if all the spaces at English clubs are taken by foreign players, you have no chance."
It is the job of new FA director of elite development Dan Ashworth to help arrest that decline. However, after taking up his post last month after leaving West Brom, Ashworth accepts the improvement needed will not come quickly.
"Make no bones about it there are no quick fixes in youth," said Ashworth. "There are no shortcuts. It's all about building an education to get them through and that is a long process.
"The amount of revenue generated by the Premier League is not something we can control. Our job at the Football Association, the Football League and the Premier League is to make sure is that our own home grown players are at a world class level to get into a world class league."