Liverpool achieved a fourth successive victory to start the season emulating the achievements of the 1990/91 side that were defending their title over 27 years ago.
First-half goals from Sadio Mane and Roberto Firmino were enough to make it 12 points from 12 for Klopp's side despite Rachid Ghezzal pulling one back for the hosts following an Alisson Becker error.
However, the victory ensures the Reds made it the perfect start to the domestic campaign and also meant Liverpool now go into the international break with maximum points.
Klopp spoke beforehand about the Reds intent to focus only on the three points rather than an opportunity to make a statement as potential title challengers to Manchester City.
Liverpool must "improve"
His Liverpool players did the job regarding result but the performance of the team was undoubtedly lacking, and that didn't go unnoticed by the Reds German manager.
He said: "Result-wise it could not have been better.
"Performance-wise, we know we can improve and we really know that we could have already done better.
"Again, the start today showed how good we can be but it’s still early in the season – so we are not 100 per cent stable in these situations to do the right things constantly.
It was another match in which Klopps and his troops had to dig deep to secure a vital three points, but if the Reds are going to be title challengers, they need to be more ruthless in front of goal as well as cut out individual mistakes.
Both Firmino and Mohamed Salah missed golden opportunities in the first half that would have given the visitors a superior advantage going in at the break.
"It was a bit difficult; when the pitch got drier and drier, it changed our game", said Klopp.
"We scored a second one before half-time and then wanted to control the game better in the second half.
"We didn’t do that with the ball, but without the ball, we defended a bit deeper and had our counter-attacks. We didn’t finish them for different reasons; one or two times it was a smart foul from Leicester, a tactical foul, so we couldn’t go through.
"In the other situations, we didn’t play the right pass in the right moment. Then we gave the goal away, which obviously keeps the game open and we have to deal with that."
Alisson error "will not happen again"
Another win for the Reds came at a cost as the victory was overshadowed by an individual error from goalkeeper Alisson who was caught in possession by Kelechi Iheanacho who latched on to the Brazilians failed drag-back before setting up Ghezzal who salvaged a goal back for the hosts.
Despite the Reds No.1's first mistake of the season, Klopp is still adamant that the former Roma man will learn from his errors and despite his fault, his manager still takes positives from his keeper's efforts to constantly play out the back.
He said: "It’s only positive that he is that good a footballer.
"But we all have to learn to use it in the right way still. That means, pass the ball in the right moment, then immediately make another offer for him, open some passing options.
"That’s the job for the team, play the pass in the right moment, not on a dry pitch, not hard enough. Don’t do it in these moments.
"That was a mistake – no goalkeeper in the world should dribble in that situation. But he did it. But it’s all OK. For me, much more interesting is how the reaction was and it was good.
"Football is a game where a lot of mistakes happen over 90 minutes. But that one was a big one, decisive because we conceded a goal.
"But the reaction was brilliant and I’m completely fine. How I said, it was always clear it will happen. It needed to happen so that it will not happen again. I think today was the day."