Spurs were not feeling sorry for themselves, though, and took the lead through a clinical Kane finish just before the hour – marking a brace of assists for Son on a crazy Manchester evening.
Riyad Mahrez looked like he had snatched a late point with a stoppage-time penalty but, unbelievably, Kane then scored his second of the game to swing the pendulum back in favour of Spurs.
It was a masterclass from Spurs manager Antonio Conte. The Italian set his team up to be organised and played an effective counter-attacking game against one of the best teams in Europe.
Story of the match
Spurs made wholesale changes from their disappointing defeat to Wolverhampton Wanderers.
Cristian Romero came into the back-three, Emerson Royal started at right-wing back, Pierre-Emile Hojbjerg came back to partner Rodrigo Bentancur while Kulusevski replaced Lucas Moura on the right of the three-pronged attack.
Conte saw this match as an opportunity to prove something and his players made the perfect start after just four minutes.
Son raced in behind City's defence after latching onto Kane's inviting ball. The South Korean knocked the ball across as Ederson advanced and Kulusevki slotted the ball into an empty net from the edge of the box. It was his first goal in a Spurs jersey since recently moving over from Juventus.
The early goal gave Spurs something to fight for as City started to find their rhythm and get a hold of possession.
Joao Cancelo tried his best to turn the tide but he couldn't quite get purchase in the final-third. The adventurous full-back first dragged an effort wide after a well-worked corner before bending a long-range strike wide of the post.
Gundogan went even closer, rattling the post after good work from Raheem Sterling. That combination play proved to be perfect foreshadowing for the equalising goal.
Sterling's cross looked to be meat and drink for Hugo Lloris but the Frenchman spilled what should have been a routine claim. The ball broke to Gundogan, who stroked the ball into the back of the net.
It was no more than City deserved but Conte will have been disappointed at the manner of the goal. His team were keeping Guardiola's pace-setters at bay until an individual error.
The second-half started in a similar vein to the first, with City dominating territorially and Tottenham sitting in.
Kevin de Bruyne almost caught City's opponents out with a cute clip to the front-post which Ruben Dias flicked across the six-yard box, but nobody in sky blue was there to provide a finish to the second-ball.
At the other end, Kane did really well to pick out the run of Son with a well measured ball in-behind. The winger did score the winner in there reverse fixture at the start of the season but, this time, his left-footed strike from the edge of the box lacked the power to worry Ederson.
However, Spurs were not to be denied again and it was a return of the Kane-Son partnership that did the business in the 59th minute.
Kane's initial punt forward wasn't dealt with by City and the ball was eventually recycled to Son. He took his time to assess the situation before clipping the ball back into the path of the Kane, who delicately guided the ball into the far corner after ghosting into the box.
The shell-shocked hosts almost fell even further behind minutes later when Son rolled Kane in on goal with a wonderfully weighted pass, but Ederson made an incredible one-versus-one save with his leg.
It was a big stop from the Brazilian and that seemed to jolt City back into some life.
On 66 minutes, Gundogan – the biggest threat for the home team throughout – forced a fantastic one-handed save out of Lloris, curling an audacious effort towards the top corner from 25-yards.
Guardiola reacted by bringing on Mahrez for Sterling as his team sought more penetration with the possession that they were having.
However, it was Spurs who were finding more joy and they thought they had killed the game off with a 74th-minute third.
Kane did well to bring down Kulusevski's skewed cross-cum-shot, before finishing past Ederson at his near-post. Conte was jumping for joy but VAR adjudged Kulusevski to have been offside in the build-up.
This second-half was the very definition of fast and furious: top-tier entertainment.
Mahrez – fresh off the bench – tricked his way into a shooting position but his deflected effort looped narrowly wide of the far corner.
It looked like Spurs were going to hold out but in the 90th minute, Romero handled Bernardo Silva's driven cross and – following VAR intervention – City were awarded a spot-kick. Mahrez was as cool as a cucumber – stuttering in his run-up before lashing the ball above Lloris.
Incredibly, with City then looking the likeliest to snatch a winner, it was Tottenham who went down the other end and scored themselves!
In the 95th minute, Kulusevski had all the time in the world to whip a ball in and Kane rose the highest to head the ball back across and into the far corner. The Etihad were stunned into silence just moments after getting back into the game. What an incredible finale!
The result keeps the gap at the top between City and Liverpool at six points, although the Merseyside outfit have a game in hand.
It was a huge win for Spurs in their quest for Champions League football. The gap is between themselves and the top-four has now closed to four points but they have two games on Arsenal who currently occupy that enviable spot.