"I quit. I took the decision a long time ago," the 26-year-old Manaudou, who added she is expecting a second child, told the French television channel Canal Plus.
"I am three months pregnant. I have stopped swimming for two months and I'm fine with it. I want to meet new people. I will not come back this time."
Manaudou was the Olympic 400m champion in Athens, as well as picking up a silver and bronze in the same Games.
She had retired for the first time in 2009, gave birth to a daughter in 2010, but returned in time to qualify for London 2012.
She competed in the 100m backstroke, 200m backstroke and 4x100m medley relay in 2012, but did not advance from the heats on either occasion
Manaudou's brother Florent was crowned Olympic 50m freestyle champion last summer.