Sean Burns grabbed the winner for the Irn-Bru Third Division side after a Jamie Longworth penalty had cancelled out Steven Milne's early opener, as Neil McGregor saw red for the visitors.
Kilmarnock, Hibernian and Ross County were all dumped by lower league opposition on Tuesday.
An Andrew Shinnie double ensured Inverness progressed to the third round though with a 2-0 victory over Second Division Arbroath.
Caley Thistle had exited this competition at the same stage last year with a 1-0 defeat to Ayr - but Shinnie eased the nerves with a seventh-minute goal, before putting the result out of Arbroath's reach 19 minutes from time.
St Mirren cruised through with a comfortable 5-1 win over Ayr United.
Lewis Guy raced on to Paul McGowan's slide-rule pass to open the scoring after 16 minutes and Steven Thompson doubled the hosts' lead on the half-hour mark.
A wonderful individual effort from McGowan stretched the Paisley side's advantage to three before Kenny McLean added a fourth from the penalty spot.
The visitors pulled one back through Michael Moffat six minutes before the break but the Clydesdale Bank Premier League side added a fifth through Gary Teale in the second half.