The Potters appeared in full control of the game until the 70th minute, with Jamie Vardy sent off and a two-goal lead courtesy of Bojan's first-half penalty and Joe Allen's tap-in just a few minutes later.
Despite cruising through the first half-an-hour of the second-half with only the returning Leicester goalkeeper Kasper Schmeichel preventing Stoke from extending their lead with some fine saves, substitute Leonardo Ulloa's header made it 2-1 with 16 minutes left.
From then on, Leicester piled on the pressure and eventually got their reward when Daniel Amartey met Christian Fuchs' 88th-minute cross to rescue a point and condemn Hughes' Stoke to a second successive home draw.
Hughes baffled after Stoke's late collapse
In his post-match press conference, Hughes admitted: "We're scratching our heads somewhat.
"First-half I thought we were excellent, not withstanding the sending off which obviously from their point of view changed the game in a negative way.
"In saying that, I think our play and inter-play and the goals that we scored were merited. Two-nil to the good at half-time, clearly Leicester were going to have a response and have a go. Why wouldn't they cause they had nothing to lose.
"We expected a response and in fairness, I thought we dealt with it reasonably well. They didn't really create anything of note and we were in control of the game and then they make a change, Ulloa coming on.
"We allowed the ball into the box and he gets a free header on the six-yard box, which we should deal with. If you get quality balls into the box, chances can be created against you. That was the lift they needed at 2-1.
"We became a little apprehensive in our play after that. We needed to be a little bit braver."
Stoke manager bemoans side's late fatigue
Hughes stated that his side suffered from the hangover of the work they put in to earn a point against Southampton on Wednesday night, having been down to 10 men themselves when Marko Arnautovic was dismissed after 23 minutes.
He added: "Whether or not [the final 15 minutes] that's a consequence of our efforts in mid-week, I think we had a huge effort. Towards the end of the game that came to bear to be perfectly honest.
"Leicester had an extra day's rest as well. They looked like they had more energy than us, given that they were a goal down.
"Credit to them, they had a go, they had nothing to lose and we tried to protect what we had and paid the penalty for that. It's one of those games.
"Wednesday, we drew and it felt like a win. Today, we've drawn and it feels like a loss. You can't have it both ways I suppose."
Vardy challenge was 'out of control', insists Hughes
The Welshman also revealed that he agreed with the decision of referee Craig Pawson to dismiss Leicester striker Jamie Vardy when asked about his controversial red card on 28 minutes.
He continued: "People are trying to say he was pushed and that's the reason he jumped in with two feet off the ground.
"But I've seen the incident and it looked to me like he had straight legs, was off the ground and out of control obviously.
"You can understand there's been a couple of incidents like that where players have done similar and got away with yellow cards but I think they were deemed to be wrong. This time, the referee got it right."