The first period ended on a high after a slow start, with chances for Kelechi Iheanacho, Zlatan Ibrahimovic and Marcus Rashford but to no avail.
United started the second half brilliantly, with Paul Pogba hitting the post before Mata struck home in the 53rd minute. Chances for Ibrahimovic and Rashford followed, but Mata's strike proved the crucial difference.
Off to a slow start
Once again the city of Manchester was buzzing as its two sides went toe-to-toe, but it was a slow start from both sides.
City went close in the opening two minutes, as Jesus Navas played a great ball onto the head of Iheanacho but it flew over from six yards.
There were calls for a penalty in the 11th minute, as Aleix Garcia was put through by David Silva. The young Spaniard was bundled over by Michael Carrick, but Mike Dean waved away the appeals.
United were looking to right the wrongs of Sunday's heavy defeat to Chelsea, and two good chances towards the end of the half.
Rashford played a great ball through to Luke Shaw on the by-line, the full-back cut it back to Zlatan via a touch from Pogba, but the Swede's effort failed to trouble Willy Caballero.
He then had his own opportunity just seconds later, as his touch had Nicolas Otamendi on the deck, but when he finally got his shot away, it was deflected wide.
Excellent start to the half
Jose Mourinho's side came flying out of the traps in the second period as they looked to get one over their bitter rivals.
They almost had it four minutes after the restart, as Ibrahimovic put Pogba clean through, but the Frenchman's effort was pushed onto the post by Caballero.
It wasn't long until United managed to take the lead; Ibrahimovic started the move holding off Otamendi, before playing it across the box. It set up well for Mata, as Ander Herrera held off Fernando to allow his compatriot to slot into the bottom corner.
It was almost two in the 64th minute when Rashford played a scrumptious ball in from the left, it looked so easy for Ibrahimovic to put it home from six yards but it went under his foot.
Pushing it right to the end
Both sides kept pushing as the clocked ticked down, with more than a place in the last eight at stake.
United looked to have a good chance in the 79th minute, Ibrahimovic puts Rashford through again who looks to pick out Pogba, but Pablo Maffeo snuffs out the chance.
The final chance fell to the Reds inside stoppage time, another great through ball looks for Pogba, but Caballero comes flying out to beat him to the ball.