Didier Deschamps's side toiled in a dull encounter at the Stade de France and lost to Japan for the first time after Yasuyuki Konno poked the ball home at the end of a lightning counter-attack two minutes from time.
France were shaky at the back, toothless up front and failed to stamp their authority on the midfield in the absence of the injured Abou Diaby and Rio Mavuba.
The home team could have taken the lead a few minutes before Konno's goal but Olivier Giroud's fine volley was superbly parried by keeper Eiji Kawashima.
"It's a bad result. Japan had more opportunities after the break but we also had ours and you can't win when you don't score from your chances," Deschamps told the TF1 television station.
Yasuhito Endo also tied the record of 122 full international appearances held by former Japan defender Masami Ihara.
Spain warmed up for Tuesday's game against France with a comprehensive 4-0 World Cup win over Belarus in Minsk on Friday.
France and Spain both have six points from two games in Group I, with the world and European champions top on goal difference.