They rode their luck and were indebted to 20-year-old goalkeeper Alastair Worby on more than one occasion but the Saints held on and will now travel to Macclesfield, who are just one step above Brackley in the football pyramid, next month.
Worby was worked for the first time in the seventh minute when he had to tip over a swerving volley from Bradley Dack, whose goal nine days ago in the seventh-minute of injury-time spared Gillingham's blushes and set up Monday night's replay.
Gills controlled the early stages and came close again when Worby did well to keep out Adebayo Akinfenwa's header from Myles Weston's cross.
But completely against the run of play, the Saints went ahead after 21 minutes when Walker's strike from outside the box was deflected into the bottom corner of the net.
The visitors came within a whisker of equalising in the first minute of the second half as Joe Martin's shot from 25 yards came back off a post.
The League One side continued to press without reward and Akinfenwa's powerful shot from Dack's knock-down dipped just over the crossbar.
Worby was then the hero again as he kept out Chris Whelpdale's header before tipping another effort from Cody McDonald on to the post with two minutes left.
- Sports & Recreation
- Adebayo Akinfenwa
- Bradley Dack