The first-half found chances few and far between with Alex Oxlade-Chamberlain and Scott Arfield going close but to no avail, the Reds managed to take the lead after the hour mark through Sadio Mané as the game began to become an end-to-end affair. It did look like Burnley managed to snatch a deserved point with Jóhann Berg Guðmundsson's late equaliser, but it was Klavan's effort four minutes into extra-time that managed to secure the victory.
Testing the waters
Jürgen Klopp made seven changes to his side ahead of the trip to Turf Moor with both Mohamed Salah and Philippe Coutinho absent from the starting XI, Burnley's home record gave the indication that chances would be hard to come by and it showed in the opening 45 minutes.
Oxlade-Chamberlain did well in the 17th minute to drive at goal and get a shot away from 25 yards out, but it looked comfortable for goalkeeper Nick Pope as he did well to dive and push it away from danger.
It was then the turn of the home side to have a crack at skipper Simon Mignolet's goal. Burnley did well to win possession in the Liverpool half as Arfield flicked on Steven Defour's cross following Ashley Barnes' blocked effort but it was straight at Mignolet.
A moment of excellence breaks the deadlock
It was going to take a moment of excellence to try and break the deadlock in Lancashire, and that arrived just after the hour mark. It was all Mané as he did well to control the cross from the right and turn. The Senegalese then smashed the effort from the edge of the area and into the top corner.
Becoming a much more open affair
Mané's effort proved to be the spark as the game burst into life from that point onwards, with both chances having equally good chances to try and add to the scoreline.
Trent Alexander-Arnold hit an excellent effort in the 74th minute as sweet effort from 25 yards out looked destined for the top corner, but Pope went one better to dive and push away the effort.
Oxlade-Chamberlain looked to add a second ten minutes later with a good effort from ten yards out, it initially looked to have hit the post but it was once again good work from Pope to react quickly and stick out a glove.
Sean Dyche's side went down the other end and almost grabbed an equaliser, Sam Vokes did well with the header and looked to be heading for the bottom corner but Mignolet did well to save his effort.
Snatching a late point ?
Both sides continued to throw everything but the kitchen sink at each other's goal as the seconds ticked on by, but The Claret's looked to have snatched a point with their equaliser three minutes from time.
It was a textbook Burnley move as Charlie Taylor's cross in from the left was flicked on to the far post by Vokes, and Guðmundsson was there to steal in unmarked to turn home from close-range.
Taking the win back to Merseyside
It looked like that a stalemate would be the final result as the clash entered extra-time, but The Reds managed to snatch all three points with the final roll of the dice.
The centre-back pairing combined well as Dejan Lovren flicked on the free-kick towards goal, and Klavan was there at the far-post to nod home.