The Frenchman opened the scoring with a delightful finish on 28 minutes, but the visitors were level within two minutes through Sardar Azmoun.
Atleti controlled the second-half, but without really testing Soslan Dzhanayev, and it wasn’t until the death when they snatched victory.
Griezmann helped the ball over the ‘keeper and into the net off the underside of the bar in the third minute of additional time.
The Spanish outfit now sit three points ahead of Bayern Munich in the group, and they welcome PSV Eindhoven to town in three weeks.
Frustrating start for hosts
The hosts came into the game knowing a win would seal their place in the knockout stages, and it was Atleti who went close first.
Griezmann controlled Koke’s pass and tried to lob the ball into the far corner, but the effort didn’t trouble Dzhanayev.
Christian Noboa struck a volley straight at Jan Oblak before a last ditch tackle from Timofei Kalachev down the other end prevented Yannick Carrasco from opening the scoring.
Griezmann puts hosts ahead
The final ball was letting the home side down, but the spark they needed came when Griezmann produced a moment of brilliance on 28 minutes.
Carrasco played a pass over the defence, who appealed for offside, and the Frenchman struck over the ‘keeper on the half turn.
However, Atleti were caught napping and Rostov were level within two minutes after a swift break upfield.
Rostov level within two minutes
Dmitry Poloz threaded the ball through to the advancing Azmoun, who made no mistake with the finish, sliding under the ‘keeper and into the bottom corner.
Diego Godin was left out of position for the goal, but could have made up for it soon after, but his header from Carrasco’s corner failed to hit the target.
Fernando Torres couldn’t help on Sime Vrsaljko’s fizzing pass before Fyodor Kudryashov forced Oblak into making a sprawling save on the stroke of half-time.
The hosts came out strong and could have gone ahead inside two minutes of the restart when Dzhanayev made a diving save from Koke’s fierce strike.
Rostov standing strong in defence
It wasn’t until after the hour mark when Atleti created their next opening, Koke drove at goal, but the ‘keeper saw the ball roll wide of the post.
Rostov were holding out, and despite the hosts having a lot of possession, they weren’t carving out too many chances.
Torres’ strike from the far side of the penalty area was a one of desperation and Dzhanayev collected easily.
Griezmann snatches victory in stoppage time
The hosts had shot on target, however, Dzhanayev wasn’t really troubled throughout the second-half, and right at the death Atleti snatched the victory.
Griezmann lofted the ball into the net off the underside of the bar after Rostov defender Alexandry Gatcan headed on into his path and Atleti maintained their 100% record in Group D with the win.