The former middleweight champion was scheduled to face super-middleweight champion Andre Ward on January 26 in Los Angeles, but the fight was ultimately cancelled after Ward injured his right shoulder.
"When you stay in the sport too long you have health problems. That's a big, big thing for me," American Pavlik told ESPN.
"I'm not talking about now. I'm talking about in the future. I'm talking about when I'm 55 or 60. What's gonna happen to me then? Why take any more chances, especially in that sport. It's a brutal sport and you never know what can happen.
"I won the world title, I defended my title, I was champ for three years and I made good money. Why take the chance of medical problems? That's a big part of it. I also don't think the drive is there anymore. I'm moving on to a new chapter in my life.
"I've been a pro for 13 years and doing this since I was nine. I go away for two or three months at a time (to train) and I'm tired of leaving my family. It comes to a point where you just don't want to do that anymore.
"I put my money away and then with the Ward fight being cancelled, well, health and time with my family is more important at this stage, especially with no guaranteed big fight or date."
Pavlik ends with a 40(34)-2 record. He lost the middleweight title he won from Jermain Taylor in 2007 in a war with Sergio Martinez which went to the cards.
His other loss came against Bernard Hopkins.
Pavlik has also had problems away from the ring, including two stints in rehab for alcoholism.
"I had a pretty good career," Pavlik said. "I was 40-2 and I only lost to two of the best guys, Martinez and Hopkins. I'm content. It's not like I got knocked out. It's not like I got knocked out the way Manny Pacquiao got knocked out [by Juan Manuel Marquez before Christmas].
"I'm fine physically. But I feel like why keep pushing the envelope? I made it this long and this far and I have had fights that were tough fights. I got dropped by Jermain Taylor, got dropped earlier in my career. But you constantly take punches and it all adds up. When you're done, you might have brain problems, you might be punchy, you might develop Parkinson's. Why push it? I don't need it and my heart's not in it anymore. So why take the chances? I've been very fortunate up to this point."
Pavlik is married to Samantha. They have two children aged six and three together.
"For two or three years, she was telling me she wants me to be done. She said when I got to the age of 30, she wanted me to retire, so she backed it," Pavlik said.
"Her biggest fear was long-term (physical) affects of boxing. My parents are behind it. They always said if I am not going into the ring 100 per cent mentally ready, they didn't want me doing it, so they've been supportive of it."
