However, Ollie McBurnie took advantage of a mistake at the back to draw the Blades level and force penalties.
The stage was then set for Forster, who produced two fine saves to allow Oriol Romeu to fire home the decisive spot kick and send Ralph Hasenhuttl's side into the last 16.
Blades strike early on
However, after a stuttering start to life back in the Championship, Sheffield United have found their feet in recent weeks and that confidence was evident as they went in front on the eight minute mark.
Jayden Bogle found room down the right and he pulled the ball across from the byeline to Stevens, who picked his spot and wrong-footed Forster as he shot back across the diving keeper.
The visitors searched for a response, with Diallo somehow steering Nathan Tella's low driven cross wide when well placed before the Frenchman made amends moments later.
Diallo heads Saints level
Both sides made changes at the break, but it was United's substitutes who combined to calve out a chance as Rhian Brewster played in Ben Osborn, but Forster denied him with his legs to keep Saints level.
Salisu's first for Saints
It proved an important stop, with the visitors soon taking the lead. Tella's quickly taken corner caught the Blades napping, as the youngster exchanged passes with James Ward-Prowse before fizzing in a low cross which Salisu met at the near post to turn it in.
Error proves costly
A loose pass from Lyanco conceded possession and Osborn picked out McBurnie at the far post, who slammed the ball past Forster.
Che Adams, with four goals in his last three games against his former club, was summoned with ten minutes to go and he was presented with a great chance to win the tie late on.
A brilliant cross by Perraud from the left picked out the striker who looked destined to score, but Foderingham reached out and somehow kept the substitute at bay to ensure the tie went to penalties.
Shoot-Out Success for Saints
Ward-Prowse stepped up first and buried his spot-kick, as did Adams, but Brewster and Armando Broja were both denied, leaving the scores level after three kicks each.
When Diallo converted his penalty and Forster kept out McBurnie, it was left to Romeu to fire home the winning penalty to put Saints in the hat for the Quarter-Final draw on Wednesday night.