The striker's impressive streak in front of goal continued as he grabbed the only goal after netting his first hat-trick at the weekend.
The win allowed Rangers to take their revenge on the Bairns, who dumped Ally McCoist's men out of the Scottish Communities League Cup last season - with the two sides set to meet again in that competition at Ibrox next week.
The breakthrough came on the stroke of half-time when Little collected from Francisco Sandaza before driving low past Michael McGovern from a tight angle and into the bottom corner.
Falkirk set about trying to haul themselves back into the tie after the break and Stephen Kingsley picked out Kieran Duffie at the back post but he fired straight into the hands of Neil Alexander.
There was a brief stop in play when assistant referee John Gilmour appeared to suffer a knock but he was able to continue and the match action resumed.
The first booking of the match came with just over 20 minutes to go when Craig Sibbald was cautioned for a challenge on Black.
Rangers then made a double switch when Kevin Kyle and Kal Naismith were thrown into the action for Barrie McKay and Sandaza.
The visitors tried to end any hopes of a late Falkirk fightback and a shot from Lewis McLeod was hooked off the line by Blair Alston, after another penalty claim by Kirk Broadfoot.
Rangers held on to claim a narrow victory and ensure their names will be in the draw for the next round.