Lloyd Dyer claimed Leicester's other goal in a one-sided contest at the King Power Stadium.
Danny Ings' reply came too late for the visitors, who also ended the match with 10 men following the last-minute dismissal of Michael Duff.
Charlie Austin had given Rangers the lead but they were unable to hold on and Quinn's effort a minute from time snatched the points for Rovers.
Blackpool's 2-0 victory over Sheffield Wednesday, courtesy of goals in each half from Ricardo Fuller and Dan Gosling, keeps them fourth in the table while Wednesday suffered their seventh defeat of the season and sit second from bottom.
Barnsley now prop up the league after their 3-0 defeat to Birmingham.
Right-back Paul Caddis scored his first goal for Birmingham, then followed it up with his second before Nikola Zigic competed the comprehensive win.
Tommy Spurr's header was enough to hand Blackburn a 1-0 victory over Leeds.
Ashley Barnes' equaliser claimed Brighton a point in their lunchtime clash at Bournemouth. Matt Richie scored first for the Cherries, with the game ending 1-1.
Tommy Smith headed the winner for Ipswich in a 1-0 success at Charlton, while Middlesbrough saw off Bolton by the same scoreline thanks to Grant Leadbitter's penalty.