Smith, who is in his first season as Surrey skipper, underwent scans on his ankle problem on Wednesday and it was decided that he will need to have an operation.
The 32-year-old batsman will now return to his native South Africa and, according to Surrey "will miss a few months of the English county season".
That must also put a question mark over his involvement in next month's Champions Trophy.
Smith, South Africa's Test captain, has been named in the Proteas' 15-man squad for the tournament, which takes place in England and Wales from June 6.
After a slow start to his Surrey career, making scores of just two and three in his first two championship innings against Somerset and Sussex, Smith has started to find his feet with two half-centuries and an unbeaten 48 in four innings.
A statement on Surrey's website, www.kiaoval.com, read: "Despite issues with his ankle caused by a hard winter's cricket for the Proteas, Graeme was determined to start his Surrey career and has already made a big impact at the club.
"However, after scans this afternoon it has been determined that surgery is required and he will miss a few months of the English county season for Surrey."
Smith said: "To have to head home now, so early in my time at Surrey is very frustrating but I leave the team in a good place and hope to be back with them as soon as possible."
- Sports & Recreation