Eurosport - Fri, 18 Jun 14:25:00 2010
The game came alive after the 36th minute dismissal of Klose for his second bookable offence and within two minutes Milan Jovanovic had given Serbia the lead.
Podolski had a second-half penalty saved by Vladimir Stojkovic and Germany were unable to force an equaliser, leaving the group wide open with both sides and Ghana - who play Australia on Saturday - level on three points.
It was Germany's first group-phase loss since a 2-0 defeat to Denmark in 1986.
Referee Alberto Undiano of Spain dished out four yellow cards in the opening quarter of the match so it did not come as a shock when one of the sides lost their full allocation.
Klose, booked for an early trip on Branislav Ivanovic, was given his marching orders after tracking back and clipping the heels of Dejan Stankovic.
It was a tame dismissal and Serbia immediately made the 10 men pay as Milos Krasic got to the by-line and stood up a cross at the back post for giant striker Nikola Zigic to head across to Jovanovic, who lashed home from four yards.
Serbia, under pressure after their opening 1-0 defeat by Ghana, celebrated wildly but were nearly pegged back on the stroke of half-time when Stojkovic punched a corner to Sami Khedira who smashed his shot against the bar from 16 yards.
After the break, Bastian Schweinsteiger saw a shot fumbled by Stojkovic while Podolski missed a pair of decent chances, dragging a shot wide of the far post after getting behind Ivanovic before slashing wildly into the side netting.
It was t he Cologne man who spurned the 60th-minute chance from the penalty spot, gifted to them when - for the second successive match - a flailing Serb hand made contact with a seemingly innocuous cross.
This time Manchester United's Nemanja Vidic was the culprit but Podolski's penalty lacked power and Stojkovic made the save, diving away to his left.
As Germany threw men forward, gaps appeared at the other end with Jovanovic hitting the post with a curler after a surging run from Krasic, the winger then crossing to Zigic who soared above Per Mertesacker only to see his header clip the top of the crossbar.
Germany could feel further aggrieved as a late Stankovic kick on sub Mario Gomez in the penalty area was missed by Mr Undiano but Serbia held on to end a run of five consecutive World Cup defeats to kick-start their campaign.
Germany v Serbia
1st Half Goals 0-1
Shots on Target 2-1
Shots off Target 9-9
Blocked Shots 5-0
Yellow Cards 3-4
Red Cards 1-0
Passing Success 84.3%-83%
Tackles Success 94.1%-61.1%
Territorial Advantage 55.2%-44.8%