Shar converted a penalty after Bristol City centre-half Alfie Mawson was judged to have handled the ball on the line, in an effort to stop David McGoldrick's goal-bound effort.
Chris Wilder made four changes to the side that was narrowly defeated by Thomas Tuchel's Chelsea on Sunday evening. Ethan Ampadu - who was ineligible against his parent club - Rhian Brewster, Billy Sharp and John Eagan all came back into the side.
Story of the match
On a chilly night in Yorkshire there was certainly no fireworks to warm up those in the vicinity of Bramall Lane. The game mirrored a training ground exercise for Wilder's side, as they dominated the ball for the vast majority of the opening half.
Chances were something of a premium on the night, falling few and far between to anyone who was able to break beyond either of the resolute defences.
The best chances of the opening 45 minutes fell to both of the Blades' wingbacks, Jayden Bogle and Max Lowe. The latter's effort came on the 10 minute mark when he found himself through with Max O'Leary in the Bristol goal, after some excellent combination play from McGoldrick and Bogle opened the space. However, Lowe couldn't keep his effort from eight yards-out down, as it brushed the crossbar before going behind.
Bogle found himself on the end of Ampadu's cross from the left hand-side - on his arrival into the box - although he was unable to direct his header into either corner, making life fairly easy for O'Leary between the posts.
The Robins enjoyed some half chances of their own, a half volley from Thomas Kalas forced Aaron Ramsdale into action early on in the tie.
As the referee signaled an end to the first half, neither side could have had any complaints about heading into the interval all-square.
Wilder's men headed out for the second half and immediately looked sharper, quicker and hungrier than the side that took to the field in the first half.
The Blades soon got their reward for their increased output, In the 65th minute a frantic goal-line scramble resulted in McGoldrick and John Lundstram both having shots cleared off the line. Although after several replays via the VAR monitor, referee Rob Jones, adjudged that McGoldrick's effort had been kept out via the hand of Alfie Mawson.
Mawson saw red for his temporary spell as the Robins' 'keeper. Skipper Billy Sharp stepped up from 12 yards and made no mistake with his sturdy spot-kick, slamming it into the bottom left corner to claim his fourth goal since the turn of the year.
McGoldrick nearly doubled the home sides lead seconds after the resulting City restart, when he was played through into the box but was unable to orchestrate enough bend on the ball to nestle it into the far corner.
United operated in a rather comfortable manner in the closing stages of the contest, as Holden's side broke from their rigid shape in hope of snatching a goal that would've extended their stay in this years competition.
Burke had the chance to wrap the tie up for his side in added time when O'Leary found himself chasing Ampadu's channel ball. The Bristol City 'keeper was beaten to the ball by Burke down by the far corner, but the former Nottingham Forrest man's effort was cleared meters from the goal line by Kalas.
Sheffield United XI: Ramsdale, Bogle, Basham, Egan, Ampadu, Lowe, Lundstram, Fleck, McGoldrick, Brewster, Sharp.
Bristol City XI: O'Leary, Hunt, Kalas, Mawson, Moore, Towler, Vyner, Williams, Paterson, Wells, Diedhiou