Although Australia have already qualified for next year's World Cup finals, the defeat at the Parc des Princes followed another 6-0 humiliation at the hands of Brazil in Brasilia last month.
The FFA were quick to point out the sacking was the result not just those two results but also of a lacklustre World Cup qualifying campaign during which Australia managed just three wins from eight matches.
"The decision was taken after an internal review of the 2014 FIFA World Cup qualification campaign and the international matches played since, including the 6-0 defeat to Brazil on 6 September and the 6-0 defeat to France earlier today," read an FFA statement.
Osieck, an assistant coach with the Germany team that won the 1990 World Cup, was brought in after Australia bowed out at the group stage of the 2010 finals to oversee the transition from an ageing 'golden generation' to a younger group.
The 65-year-old German started his reign encouragingly by taking Australia to the final of the 2011 Asian Cup, where they were unfortunate to lose a close match to Japan.
But the fact the team for Friday's match was captained by 35-year-old defender Lucas Neill with 33-year-old Tim Cahill leading the forward line indicated that he had only been partially successful in overhauling his squad.
"The decision is based on the longer term issues of the rejuvenation of the Socceroos team and the preparations for the World Cup and the Asian Cup," FFA chairman Frank Lowy said.
"FFA has set a strategic objective of having a highly competitive team in Brazil and then handing over a team capable of winning the Asian Cup on home soil in January 2015.
"We have come to the conclusion that change is necessary to meet those objectives. I thank Holger for his contribution to Australian football and wish him well in his future endeavours."
Osieck's assistant Aurelio Vidmar will act as caretaker coach for a friendly against Canada in London on Tuesday but is unlikely to be handed the job full time.
Dutchman Guus Hiddink, who led Australia to the last 16 at the 2006 World Cup after taking charge less than a year earlier, would be an obvious choice, while former Chile coach Marcelo Bielsa and Frenchman Gerard Houllier might also be sounded out.