Ademola Lookman fired The Foxes in front early in the first half, but a brace from Bruno Guimarães, with his second coming in the 95th minute, condemned Leicester to a heartbreaking late defeat.
Rodgers told the media in his post-match press conference: "You don't always get what you deserve in football at times and that was the case today.
"I thought we played very well in the game from a performance perspective. We started the game well and were in control."
Howe 'would've been happy with a point' before snatching the winner
When it came to both sides making substitutions deep in the second-half, the biggest indication towards Newcastle seemingly being happy to settle for a draw was when head coach Eddie Howe brought off striker Chris Wood and replaced him with a midfielder in Joe Willock.
Rodgers acknowledged that – and says that the late goal was a bitter pill to swallow after the performance his players produced.
He said: "Eddie [Howe] would've been happy with a point at the end, but for some reason we failed to control the ball right at the very end of the game with the last move of the game they break away and score.
"It's hugely disappointing in terms of the result, but lots of the performance was very good and that's what we'll take from it."
On the changes made after the PSV win
Rodgers made eight changes from the side that won late on in Eindhoven on Thursday night which saw them progress into the semi-final stage of the Europa Conference League.
The Northern Irishman said on his team selection: "It's just common sense.
"We've got lots of players coming back from injury and on the back of really tough game on Thursday, with the travel, we couldn't afford to take too many risks.
"Last weekend we made changes, brought good intensity to the game and I think we can see that again from the team today."
Massive praise for Newcastle and their fans
Since then, Howe has took charge following Bruce's dismissal, and has all but led The Magpies to safety – with today's win leaving them 12 points clear of the relegation zone.
The 51-year-old had a classy message for their head coach and their supporters.
"I’m so happy for the people here and the supporters," he said.
"You get a great feeling every time you come here. It’s one of the cities and grounds I really enjoy coming to. It’s great for the supporters because the passion here is incredible.
"Eddie [Howe]’s come in and made a very good start and made some really good signings. I think that gives great hope to the supporters.
"I’m pretty sure they’ll look to build on that in the summer.
"This is one of the iconic football clubs in Britain. It’s a special place.
"We’re just disappointed with the result but we say congratulations to Newcastle."