Jeff Schlupp's superb left-foot volley 11 minutes from time sealed a first league win in 10 matches for Nigel Pearson's men and saw them replace Wanderers in the play-off positions.
Leicester made the worst possible start falling behind in the third minute to David Ngog's penalty.
The Foxes conceded a last-minute penalty to draw with Birmingham on Friday night and they conceded another here after just 75 seconds when captain Wes Morgan tripped Lee Chung-yong.
As he did with Chris Burke's equaliser four days ago, Kasper Schmeichel dived the right way but Ngog's spot-kick was right in the bottom left-hand corner and found the back of the net via the post.
However, two goals in as many minutes just before half-time turned the game on its head.
Chris Wood's 39th-minute penalty after Lloyd Dyer's quick feet left Danny Butterfield in a tangle brought the Foxes level before Dyer put them ahead in emphatic fashion after 41 minutes.
Wood did brilliantly to draw the attention of three defenders before finding Dyer with a reverse pass.
Dyer's first touch took him into the penalty area and with his second he crashed a left-foot past Adam Bogdan and into the net off the underside of the crossbar.
Darren Pratley levelled things up again midway through the second half, sweeping home substitute Sam Ricketts' low cross from the right, but Schlupp had the final word, despite some intense late pressure from the visitors.