It was a game in which Cheltenham dominated for large parts, with Mansfield's only shot on target in the initial 90 minutes being their early goal from Stephen McLaughlin.
However the hosts got the equaliser that their performance merited on 73 minutes, with Alfie May scoring a brilliant individual effort.
That goal forced the game into a further 30 minute period, but the home team avoided penalties thanks to Will Boyle's goal ten minutes from time.
Story of the match
The game began at lightning pace, with the visitors being the fasted out of the blocks, opening the scoring after just three minutes.
McLaughlin played the ball to Jordan Bowery, before getting it back and firing a first time effort at goalkeeper Joshua Griffiths, who only succeeded in diverting the shot into his own bottom corner.
Cheltenham finally got going on ten minutes, as forward George Lloyd managed an effort on target, but it was cleared away by Harry Charsley.
Lloyd was then presented with another chance six minutes later, connecting with a free-kick that he guided onto the crossbar, as the Robins searched for an equaliser.
May fashioned another couple of chances for the host around the half hour mark, first firing a low shot straight at keeper Marek Stech, before hooking an effort inches over after the ball fell to him.
And as the first half drew to close, Lloyd once again had a chance, but was unable to sort his feet out at the back post. Concluding a half in which Cheltenham would have felt they deserved to at least be level.
The second half got off to a much more quiet start than the first, however Cheltenham did continue to assert themselves, piling men forward, and getting into the Mansfield box often.
On 67 minutes, the home side had appeals for a penalty waved away, after Andy Williams was blocked off inside the area by Ryan Sweeney.
But six minutes later, Cheltenham got the goal that their efforts more than deserved. As May collected the ball on the corner of the area, before cutting inside and curling a wonderful shot into the net.
Mansfield nearly managed to respond ten minutes later, but the flag was raised after Charsley had put the ball in the net.
The team in red looked the more likely to prevent the game from going to extra time however, with a couple of dangerous crosses into the box. Particularly one on 89 minutes that just evaded Matty Blair.
The sides could not be separated however, as the game went into extra-time.
Much like the start of normal time, Mansfield began brightly, as Jamie Reid forced Griffiths into an excellent save after four minutes, as they looked to snatch the win.
The yellows continued to press and almost found the net on 100 minutes when George Lapslie's shot took a huge deflection before dropping just wide.
With penalties looming Cheltenham managed to find the breakthrough. A dangerous long throw from Ben Tozer caused chaos, as it was touched in by defender Will Boyle.
There were still ten minutes of extra-time to hang on, but the Robins could have extended their lead right at the death, with Alex Addai seeing a one-on-one chance saved well by Stech.
Cheltenham's dominance ultimately shines through to keep cup run going.
Robins manager Michael Duff will be elated that his side eventually managed to break down a side that they would have been expected to beat on paper. They certainly had their chances to win the game in 90, but a sloppy opening few minutes meant it went the distance.
Encouragement can be taken from the performance, and can now look forward to the fourth round draw, where they could come up against potentially huge opponents.
Duff will now turn his attention back to the league, with his side currently sat in fifth place, where he will be hoping to avoid the play-off heartache of last season.
Clough's focus must now return to keeping up league momentum, and put a forgettable start to the campaign behind them.
When Nigel Clough took over at Mansfield in November, they were sat in 22nd place in League Two. But recent good form, including three straight wins for the first time in two years, means that they have pulled themselves away from danger.
And a cup run like this would have helped to fuel that positivity, but ultimately it wasn't meant to be, falling to defeat in a performance that deserved as much.
The Stags must now turn their attention back to their good form in the league, and will hope that they can get back to winning ways immediately to propel themselves further up the table.
Man of the match
The 27-year-old striker was a constant threat throughout the game, having a couple of first half chances, before getting his goal with just under 20 minutes to go.
You would have initially expected him to cross, but he was able to cut inside and finish with great precision, in what was certainly the goal of the game.
May has eight goal contributions already this season, so his team knew that he will have been eventually able to bring his side level, if the opportunities were presented to him.
It was no more than the team deserved as a whole for their performance, and May will now be hoping to express himself further in the next round.