Chib Chilaka scored the only goal after 20 minutes to send the Conference North side into the second round.
Non-league Macclesfield pulled off the biggest shock in the first round of the FA Cup when they won 2-0 at League One high-flyers Swindon.
The Silkmen were not the only non-league side to beat Football League teams with Chelmsford and Hereford also moving into the second round at the expense of so-called superior opposition.
Blue Square Bet Premier Macclesfield took the lead in the 63rd minute through a stunning long-range free-kick from from Tony Diagne and Louis Thompson's own goal at the death secured victory.
The Robins also saw Aden Flint sent off for the second time in four games when he was given a straight red card for a challenge on Macclesfield midfielder Sam Wedgbury.
Anthony Cook, Donovan Simmonds and Jamie Slabber were on target as Conference South side Chelmsford beat Colchester - 12th in League One - 3-1, Michael rose netting the consolation.
Hereford, 14th in the Blue Square Bet Premier, provided the first shock of the day when they dumped out League One Shrewsbury 3-1 thanks to two goals by Ryan Bowman and another from Will Evans.