The hosts entered the tie as firm underdogs, holding the second worse defensive record in the Championship with 66 goals conceded. Such stat looked out of character going off their first-half display, keeping a ruthless Man City side out in the opening 45 minutes of the contest.
The Citizens showcased glimpses of what they are capable of in the second half, however. Mahrez's effort easing any nerves for the visitors before Grealish got on to the scoresheet to put any rumours of a cup upset to bed with 25 minutes to go. A massive moment for the former Villa man, too - a fourth goal of the season for the no.10.
The Premier League leaders will find out their next round opponents later this week while Peterborough turn their attention back to league action. New manager Grant McCann finds his side eight points adrift but after tonight's display, they may believe their fate isn't sealed just yet.
Story of the game
The visitors dominated possession from the get-go, with Gabriel Jesus bearing down on Peterborough keeper Steven Benda in the opening minute of the game, almost deflecting his long kick up field into the net.
Grant McCann's side were not afraid to break away on the counter, however, with Jeando Fuchs getting goal-side of Nathan Aké before forcing Ederson into a quick save at his near post. A strong start from the team bottom of the EFL Championship.
Pep Guardiola's men spent the majority of the opening 45 minutes camped outside the Peterborough box but lacked composure when a chance arose. İlkay Gündoğan's curling effort from distance was the best of the lot as his strike whistled past the post with the Posh's defensive resilience frustrating the Premier League champions.
Ultimately, the Citizens failed to establish authority before the break - which was warmly received by the the Weston Homes Stadium home support.
In a night where chances were few and far between for the hosts, Sammie Szmodic's early second-half chance was as good as it was going to get. A long throw from Joel Ward dropped to the Posh no.7 who snatched at the opportunity, watching on as the ball agonisingly dribbled wide.
Wasted chances soon proved fatal by the hour-mark, with Riyad Mahrez finding himself in plenty of space on the right-hand side, curling the ball into the bottom-left hand corner before a man in blue could close the Algerian down.
A pinpoint through ball by Phil Foden to Jack Grealish minutes later showed the quality rooted into this City squad, with the former Villa man slotting home to double the visitors lead to ensure full control of the match.
Substitute Aymeric Laporte failed to get in on the act in the dying moments of the match, with the ball dropping to his feet just six-yards from goal. An opportunity which looked harder to miss.
Nevertheless, any doubts of City from the first-half were soon eased in the second. While it wasn't a ruthless day in front of goal, partly due to Peterborough's strong defensive performance, the Premier League side got the job done and progress into the next round.
Peterborough: Benda; Knight, Edwards, Kent (c); Ward (Mumba 79), Grant, Fuchs, Coulson (Burrows 79); Poku (Marriott 71), Jade-Jones (Clarke-Harris 71), Szmodics (Brown 71).
Subs: Cornell, Beevers, Corbett, Barker.
Man City: Ederson; Cancelo, Dias (Stones 46), Aké (Laporte 46), Zinchenko (c); Fernandinho, Gundogan, Foden; Grealish, Mahrez, Jesus
Subs (not used): Carson, Slicker, Sterling, Rodrigo, De Bruyne, Silva.