Neilson hit a stunning drive which dipped and swerved on its way into the top corner with seven minutes to go on a night of great goals at the Broadfield.
Richie Barker's men now progress into the last 16 of the Southern Section of the competition for League One and Two club.
Reds had fallen behind to a header from Bradley Dack but Joe Walsh scored his first senior goal with a cracking strike from 20 yards just before the break.
Billy Clarke converted an inviting cross from Josh Simpson two minutes into the second half but it wasn't long before Lewis Montrose equalised for Gillingham.
Reds pressed for the winner and Neilson provided it on a night of stunning goals for the home side.
Barker had made six changes to the side which beat Leyton Orient in the league with Walsh and Michael Jones both handed full debuts while Mark Connolly, Dannie Bulman, Shaun Cooper and Jonathan Forte returned to the starting line-up.
The hosts began on the front foot and the football from both sides in the opening quarter pleasing on the eye. Nicky Adams had our first opening but his 11th-minute shot hit the roof of the net.
Gills went ahead from virtually their first attack of note. Reds failed to clear Chris Whelpdale's long throw on 22 minutes and teenager Bradley Dack headed over Paul Jones at the far post to claim his first senior goal.
Adams went close again as Reds sought a response and Mark Connolly nearly equalised when his 35-yard thunderbolt took a deflection and hit the post before two goals in six minutes either side of half-time perked up the home side.
Walsh scored his first senior goal on 41 minutes when the ball dropped to him 20 yards out and he hit a stunning shot with his supposedly weaker right foot into the top corner. Then two minutes after the break Simpson made a dart deep into Gillingham territory and produced a great cross which Clarke converted as he got across his marker.
The lead didn't last long though. On 50 minutes another long throw wasn't dealt with and Montrose turned to shoot past Jones.
Barker brought on Neilson, Scott Davies and John Akinde in the search for a winner and after Forte and Clarke had gone close it came with seven minutes to go when Neilson picked up possession 25 yards out and hit a screamer into the top right-hand corner which Gills' keeper Tommy Forecast had no chance with.
A deserved win for Reds and Barker was pleased with the performance. He said: "I thought we played some excellent stuff at times and thoroughly deserved to go through.
"At times we were perhaps guilty of over-playing it but the goals we scored were first-class. I was delighted for Joe and then Scott Neilson showed what he can do. It was important to get some of the lads game time because we have a hectic schedule coming up and we are going to need everyone over the next few weeks."
Reds: P.Jones, Hunt (capt), Connolly, Cooper, Walsh, M.Jones (Davies 76), Simpson, Bulman, Adams (Neilson 61), Forte (Akinde 79), Clarke. Unused subs: Kuipers, Wassmer.
Attendance: 1249 (179 Gillingham)