Rovers, temporarily under the reins of Andy Parkinson and Ian Dawes, took an early lead after Corey Blackett Taylor's fine solo goal on 13 minutes.
Stanley were back in action after a spate of COVID cases had seen their last four matches postponed and hauled themselves level shortly afterwards through Colby Bishop.
With the closely-fought affair poised to go to extra-time, Clarke's header snatched victory for the side 17th in the fourth tier.
Story of the game
The first opportunity fell to the visitors after Scott Davies fumbled Joe Pritchard's corner and the ball dropped to Bishop, whose close-range effort deflected wide.
But Stanley had committed men forward and that left them vulnerable to the counter-attack. Blackett-Taylor exchanged passes with Otis Khan, the early replacement for an injured Paul Lewis and drove menacingly into the left channel with little resistance.
And from there, he was able to find the far corner with an impressive low finish.
Khan, the provider, then had an opportunity to score himself after Michael Nottingham failed to clear Kieron Moore's cross, but his strike from seven yards was tipped around the post by Chelsea loanee Nathan Baxter.
He was left ruing the miss on 25 minutes when Nottingham was able to deliver after Liam Ridehalgh failed to close him down. The cross was met by Bishop at the back post, and Davies was unable to prevent his header creeping over the line.
Tranmere went closest to regaining the lead before the interval as Morris intercepted a clearance, played a one-two with Liam Feeney and struck the top of the crossbar from the edge of the box.
Stanley began the second period by hitting the woodwork themselves following Nottingham's arrowed shot from distance.
Indeed, the League One side were on top for 20 minutes or so, but then Tranmere began to reassert themselves.
Khan worked Baxter following a tidy link-up with substitute Kaiyne Woolery and Manny Monthe very nearly took advantage of a scramble following a corner.
And with five minutes of normal time remaining, they regained the lead. Clarke, only around thirty seconds after going close with a glancing header, powered home from Feeney's corner.
Rovers, a surprise package in last season's competition, coped well with the late pressure to secure progression.
Man of the Match - Joe Pritchard
He may have been on the losing side, but Pritchard stood out, particularly in the early part of the second half. The left midfielder got back and defended diligently when required and looked like the man capable of inspiring a Stanley victory.
That was until John Coleman opted to substitute him on 60 minutes. Perhaps there was a fitness concern, but it was an eyebrow-raiser at best and a costly error at worst