The 21-year-old centre-back joined the Bluebirds from Tottenham in an £8million deal on Wednesday, putting pen to paper on a four-year contract at Cardiff City Stadium.
The deal was a real coup for Cardiff boss Malky Mackay, with Caulker regarded as one of the best young defenders in Britain.
Caulker, who has one England cap to his name, made 28 appearances in total for Spurs last season but felt he needed to be first-choice at a Premier League club to fulfil his ambition of being part of Roy Hodgson's plans in Brazil next summer.
He said: "If you ask any English player, they have got one eye on the World Cup.
"First and foremost I am going to have to work hard on and off the pitch to earn that sort of achievement, but it is something I am definitely working towards.
"It wasn't that I wanted to leave Tottenham but I am at a stage in my career where I want to play regular football.
"Spurs have three fantastic centre-halves and it may have been a case of being fourth-choice this season and not getting as many games as I would have liked.
"I didn't want to put my career on hold, I wanted to get out and get as much experience as possible and these opportunities don't come around very often.
"I want to help Cardiff progress as a team and progress myself."
Caulker has previous experience of playing in south Wales, having enjoyed an excellent loan spell with Swansea during the 2011-12 season.
His move to Cardiff is likely to add some spice to the derby fixtures between the two clubs, but he revealed he has received messages of support from his former team-mates at the Liberty Stadium.
"I enjoyed my time at Swansea and the people in south Wales are great so it was a no-brainer to move here," he said.
"I thank Swansea for that opportunity, it progressed my career, but now I am a Cardiff player and I hope to go above and beyond for them.
"I spoke to a few Swansea players before their game (against Malmo) last night, I wished them luck and they wished me luck. There are no hard feelings between us as players.
"It will be fantastic to be involved in the derby, I played in a London derby a few years ago and I am definitely looking forward to this one."