The 29-year-old completed the 100 and 200 metres breaststroke double at the Athens and Beijing Olympics but finished outside the medals in both events in London.
His silver in the men's 4x100m medley relay came as scant consolation after recovering from injury and pushing himself to get ready for his fourth Olympics earlier this month.
"On an individual level it's a question of where to take my goals," Kitajima told the Nikkan Sports newspaper.
"I want to take my time to think about it," added Kitajima, who finished fifth in the 100 and fourth in the 200, team mate Ryo Tateishi taking bronze ahead of him, in London.
Kitajima flirted with retirement after retaining his breaststroke titles at the 2008 Beijing Olympics before opting to come back and set the Japanese qualifiers alight.
A huge celebrity in Japan, Kitajima blamed his punishing training schedule for his failure to perform at the 2012 Games.
"I just kept wanting to push myself one more level higher," he said. "It resulted in overworking it I guess. I was so bummed when I touched (at the end of the races)."
Kitajima, who turns 30 next month, has been coaching school children in Tokyo and tipped Japan's new sensation Akihiro Yamaguchi as one to watch at the 2016 Olympics in Rio.
Yamaguchi, who narrowly missed out on an Olympic place, clocked faster times in winning the just-finished Japanese high school championships than Kitajima managed in London.