Berwick Rangers needed extra-time to see off Livingston in the second round.
The hosts thought they had done enough to win it in regular time thanks to Ross Gray’s first-half header, but the First Division visitors levelled in injury time when Marc McNulty headed home to make amends for a missed penalty.
Berwick regained the lead nine minutes into extra-time through Josh Morris’s superb strike from the edge of the box.
Livingston had just been reduced to 10 men when Jason Talbot was dismissed for abusive language, and Berwick continued to press, extending their lead to 3-1 thanks to a goal for debutant Owen Ronald in injury time at the end of the first period.
Livingston did pull one back seconds later when Simon Mensing converted a penalty following a Dean Hoskins foul on McNulty, but it was just a consolation as Berwick held on for victory.
Annan Athletic beat Morton 1-0 in their second-round clash, with debutant striker Peter Weatherson netting a superb 71st-minute free-kick against his old club.
Third Division Peterhead also beat a higher-division side, ousting Second Division Brechin 2-1 in their first round North East match.
Scottish League Cup results:
First round North East
Alloa (D1) 0-1 Dundee FC (D1)
Cowdenbeath (D1) 1-3 Dunfermline (D2)
Elgin (D3) 2-0 Montrose (D3)
Forfar (D2) 2-1 East Fife (D2)
Formartine Utd (HFL) 2-0 East Stirling (D3)
Peterhead (D3) 2-1 Brechin (D2)
Raith Rovers (D1) 2-1 Stirling Albion (D3)
Stenhousemuir* (D2) 4 - 4 Arbroath (D2) ; AET, Stenhousemuir win 3-2 on penalties
Airdrieonians (D1) 2-1 Hamilton (D1)
Annan Athletic (D3) 1-0 Morton (D1)
Berwick 3-2 (D3) Livingston (D1); AET
Clyde (D3) 1-2 Falkirk (D1)
Queen of the South (D1) 4-0 Spartans (ESFL)
Queens Park (D3) 1-2 Ayr (D2)
Stranraer (D2) 4 - 2 Dumbarton (D1)