Super-subs Ritsu Doan and Takuma Asano sealed what seemed to be a hugely improbable second-half Japan comeback.
Ilkay Gündogan's 33rd minute penalty was the least that Germany deserved, after a dominant first-half display.
However, some necessary adjustments during the break by Hajime Moriyasu brought a significant improvement in Japan's performance.
Doan and Asano both scored within the last 15 minutes, to cause a major upset in Group E. Germany now find themselves in serious danger of back-to-back World Cup group stage exits.
Here are the player ratings from the Khalifa International Stadium:
Germany
Manuel Neuer - 7/10
Prior to his team's collapse, he pulled off a fantastic save, to prolong Japan's eventual comeback. Could have done better for Asano's winner - Neuer was beaten at his near post, from close range.
Niklas Süle - 5/10
Often tucked inside when his team had possession, forming a back three with the two centre-backs, while Raum pushed on, high. Lost concentration by dropping below the defensive line, keeping Asano onside for the winner - a very costly error.
Antonio Rüdiger - 7.5/10
The only German outfield player who seemed like a very reliable defensive presence. Won all seven of his aerial and ground duels and was very composed in possession.
Nicolas Schlotterbeck - 5/10
Was beaten fairly easily by Asano and his movement in-behind the centre-back. Not an impressive performance, looked much inferior to Rüdiger.
David Raum - 7.5/10
Won the penalty for Gündogan's opener. Caused massive issues for Japan in the first half with his very advanced positions. A quieter second half.
Joshua Kimmich - 7/10
Showed signs of his clear quality throughout the evening. An effective contribution to his team's play in possession, particularly in the first half. Could have got his name on the scoresheet just before the break - his effort from just outside of the box was saved.
Ilkay Gündogan (67') - 7.5/10
A very active presence in the German midfield. Got away six efforts at goal, hit the woodwork once and, most importantly, put away the penalty to give his team the lead.
Jamal Musiala (79') - 6/10
A frustrating lack of end product was highlighted by him blazing an effort at goal high over the bar, following a fantastic solo run. Generally, an inconsistent performance with some moments of real quality combined with some frustrating ones.
Thomas Müller (67') - 6/10
Often played a part in Germany's attacking play. However, was not able to have any kind of significant impact on the game. As ever, a hard-working performance.
Serge Gnabry (89') - 6/10
Had numerous attempts at goal and will be disappointed to not have extended his team's lead; one of his efforts struck the woodwork. A similar story to most of his team-mates - a lack of end product will leave him feeling frustrated with his 89 minute performance.
Kai Havertz (79') - 5.5/10
Was the most inactive of the German front line - did not manage to have an attempt on goal. A quiet game, which many may have predicted, may mean that Flick may opt for Füllkrug or another option as the focal point of his team, from this point on.
Jonas Hofmann (67') - 6/10
Had a good opportunity to score, very shortly after his introduction. Could not have much of an impact, during the last quarter of the match.
Leon Goretzka (67') - 6/10
Managed to get away a powerful volley at goal during added time, which fizzed narrowly wide of the corner. Like Hoffmann, not enough time to get hugely involved in the action, other than that late half-chance.
Mario Götze (79') - N/A
Nicolas Füllkrug (79') - N/A
Youssoufa Moukoko (89') - N/A
Japan
Shuichi Gonda - 7.5/10
Made a total of eight saves - four of which were from efforts from inside of his box. A poor bit of play led to him fouling Raum and giving away a penalty, from a fairly innocuous position. More than made up for that error, from that point onwards.
Hiroki Sakai (74') - 6/10
Had plenty of defensive work to do and found it very difficult to deal with the threat of Raum down his side. To be fair, he may have been justified in expecting a bit more support from Ito in dealing with the German full-back. Missed a good chance, as he blazed a rebound over the bar, right before being subbed off.
Ko Itakura - 6.5/10
Him and Yoshida combined for a total of 16 clearances during the game. Made a few good interceptions and tackles. Was able to deal with the fairly constant German pressure well enough.
Maya Yoshida - 6.5/10
A good performance from the experienced captain. It probably suited the 34-year-old for his team to have to defend fairly deep throughout the match, as Gnabry - playing down the Japanese left side, was not allowed to the space to run in behind him and the 36-year-old Nagatomo.
Yuto Nagatomo (57') - 6/10
Dealt fairly well with the attacking threat down his left side throughout the first half. Had a brief spell at wing-back, after the change of formation during the interval, before being taken off just prior to the hour mark.
Wataru Endo - 7/10
As with the rest of the team, did not see much of the ball. More importantly, Endo got through an impressive amount of defensive work and was a hugely disciplined midfield presence for Japan. Won 11 of 16 aerial and ground duels
Ao Tanaka (71') - 6.5/10
Was, perhaps, the less impressive of the holding midfield pair. He did look fairly composed on the ball, but was unable to have a significant impact on the game.
Daichi Kamada - 7/10
It could have been a frustrating evening for the attacking midfielder, as his team being pinned back in their own half does not often suit a player in his position. However, Kamada covered good ground off the ball and was able to take care of the ball well, on the fairly few occasions when he received it.
Junya Ito - 6.5/10
Demonstrated his tremendous pace at times, during the first half. Was forced to get through a large amount of defensive work, where he, at times, struggled to deal with the marauding Raum. However, he was also tasked with being his team's primary threat on the counter, which made it a very difficult balance for Ito to try and follow.
Takefusa Kubo (45') - 5/10
Taken off during the break, after a very quiet first half. Only one completed pass in his 45-minute appearance, as his team really struggled to have any possession.
Daizen Maeda (57') - 5/10
Had the ball in the back of the net, after a first-time finish from close range. Unfortunately, he was in an offside position, after accelerating his run too early. Like Kubo, was only able to complete one pass all evening.
Takehiro Tomiyasu (45') - 7/10
A second-half clean sheet, following his introduction in the left centre-back position. Very impressive impact.
Kaoru Mitoma (57') - 7/10
A really impressive defensive impact. Covered the left side very well and was also confident with the ball. A must-start if Japan opt to persist with the wing-back system.
Takuma Asano (57') - 8/10
It was a fantastic bit of movement to find the space behind the German backline, for the eventual winner. Asano used his pace to get away from Schlotterbeck and find the space between Neuer and his near post to give his team a crucial win.
Ritsu Doan (71') - 7.5/10
Calmly slotted away a rebound to draw Japan level. It was his only attempt at goal during an impactful appearance off the bench.
Takumi Minamino (74') - N/A