Daniel Sturridge and Sadio Mane had twice put Liverpool ahead, with Defoe's first penalty sandwiched in-between those goals, but it was Mane who conceded the 84th minute spot-kick from which Defoe levelled.
Mannone excellent throughout
Vito Mannone, in for the injured Jordan Pickford, knew he'd be in for a busy day and looked up to the task early on as Liverpool had a flurry of early shots.
Quick play by Roberto Firmino and Sadio Mane gave Daniel Sturridge a chance to shoot from the right hand side of the box, one which Mannone tipped away just minutes before being called into action by Gini Wijnaldum, who watched his effort get pushed over the bar.
Simon Mignolet had shots to save at the other end too, ex-Everton man Jack Rodwell testing him with a long range drive, Mignolet doing well to avoid pushing the ball straight into Defoe's path.
Sturridge opener quickly cancelled out
Liverpool were slowly turning the screw though, and after another saved Sturridge shot had been followed up by Seb Larsson nearly turning an Adam Lallana cross into his own net, the opening goal came.
It was Sturridge who scored, his second goal in as many games. Coming from a corner, the Englishman was the only player to react to Dejan Lovren's wayward volley, moving across the box quickly to flick a fantastic header into the net.
You'd have forgiven Sunderland fans for worrying that the floodgates could open after that 19th minute goal, meaning they were as surprised as delighted when their team wasted no time in equalising.
Didier Ndong found himself sandwiched between Klavan and Wijnaldum as he burst into the area, going down under pressure and leaving the referee with little choice but to point to the spot. Defoe stepped up and delivered, sending the ball into the bottom left hand corner just inches away from Mignolet's outstretched arm.
Goal acts as catalyst for Sunderland pressure
Sunderland should have gone ahead thereafter, with Liverpool losing control of the game as Defoe found himself one-vs-one with Mignolet, the Belgian 'keeper doing brilliantly to get a hand to the ball and not the striker as Defoe looked to round him. The ball did break kindly for Fabio Borini, only for the Italian to shoot wide.
The away side would trouble Mannone's goal again before the break, again through Sturridge, but Klopp decided that a change was needed at half-time, bringing Alberto Moreno on for James Milner in an attempt to add some width to his side.
Sunderland's attempts to ruffle Liverpool's feathers continued into the second half as the Reds were denied any comfortable possessio for long periods, but they did create chances.
Mane couldn't get anything on the end of a fantastic Nathaniel Clyne cross, despite the goal gaping. Mane would later make up for it, but almost did sooner as he set Emre Can up, the German opting to shoot from the edge of the area instead of pass, seeing the ball fly straight at Mannone.
It looked like Moyes was settling for a point, despite Borini and Defoe looking threatening up front, as defender Javier Manquillo replaced midfielder Rodwell. A decision that the Scotsman may have come to regret soon after, with Mane restoring Liverpool's lead.
Sadio goes from hero to zero
Moreno's corner was flicked on by a Sunderland head towards the back post where Mane lay unmarked, easily poking the ball home.
His day wasn't done yet though as, just after Sturridge went off with an injury, Mane gave away the penalty that let Sunderland back into the game with just six minutes left. A controversial free-kick was given just outside the area, but there was nothing controversial about the way Mane handled from Larsson's effort on goal, Defoe sending Mignolet the wrong way the second time around to make it 2-2.
Mannone, who was excellent throughout, stayed strong until the end of the game to defend the Black Cats' point. Claiming a free-kick, saving from Lallana and then Firmino, the Italian kept Liverpool from scoring a third in five minutes of stoppage time, and 2-2 it ended.