Sunderland had took an early lead through Aiden McGeady but sat back allowing Alex McDonald to equalise for the Gills. But Sunderland regained their lead just before half time from the penalty spot through Grant Leadbitter.
The second half was less action packed and didn't see a goal till stoppage time when Graham rounded Lee Burge and smashed a shot in the net to break Sunderland hearts.
Story of the game
Lee Johnson only made the one change ahead of this game, despite having brought in two new signings since Tuesdays win at Ipswich. It was an enforced change with Tom Flanagan replacing Bailey Wright who was still suffering from concussion. Winger Jordan Jones signed on loan from Rangers didn't gain international clearance in time to be in the squad. Johnson decided to stick with Callum McFadzean instead of putting Southampton loanee Jake Voskins straight in despite questions around the formers performances in recent weeks.
Steve Evans was only able to name six substitutes for the game. Tommy O'Connor had a slight groin issue and wasn't deemed fit enough to feature. Jack Tucker moved up from the bench to replace him leaving his spot on the bench vacant.
Sunderland had a really bright start to the game having most of the ball and hit the front after seven minutes. The ball was worked down Sunderland's left to Aiden McGeady, he fainted past a defender cutting inside and releasing an effort that via a deflection looped over the helpless Jack Bonham in the Gills net.
For the second home game in a row, Sunderland scored and then dropped back inviting pressure. Gillingham were happy to oblige and started to dominate possession.
Lee Johnson spoke about their threat from long throws in the week, perhaps choosing the Flanagan over Sanderson for his Aerial ability. It proved to be their best weapon and caused Sunderland all kinds of problems.
Sunderland were causing their own problems, giving numerous cheap free kicks away but had been able to deal with them thanks in part to Jordan Willis having one of his best performances in a red and white shirt, putting his body on the line on a couple of occasions to deny goalscoring chances.
But the rearguard action couldn't hold. A long throw was the undoing. Sunderland don't clear the ball far enough and from the edge of the box and Alex Mcdonald fires it back in beyond Burge via a slight deflection to restore the parity that the visitors deserved.
Gillingham and Mcdonald nearly had a second straight away. Another cheap free kick conceded by Sunderland isn't dealt with and the gills number sixteen smashed his effort off the underside of the crossbar.
It seemed to wake Sunderland up and they started to come out of their shell, having chances.
The first came from a corner. Flanagan found the ball at the edge of the box and fired a low shot across Bonham but he managed to get a hand to it and turn it round the post.
But Sunderland did have the lead just before half time. Aiden McGeady was the driving force for the hosts in the first half. He burst forward on the counter attack and was pulled down by a covering defender.
Grant Leadbitter stepped up, but his effort was saved excellently by Bonham however the ball looked over the line. The experienced midfielder made sure it was in following in to nod the rebound beyond a helpless Bonham to give Sunderland an undeserved but welcome half time lead.
Lee Johnson must have spoke to his players about their attacking intent, as they came out the traps and should have been two up before the fifty minute mark.
A great cross by Josh Scowen found the head of Charlie Wyke and with his recent form you expected the net to bulge, but he misjudged the height smashing his header off the crossbar.
The half settled from then, with neither goalkeeper being particularly tested for at least twenty minutes. When a keeper was tested it was Bonham getting down well to save a close range left footed effort from McGeady after good work from Wyke to lay him off.
Sunderland then had a couple of chances from Leadbitter corners with both Willis and Aiden O'Brien firing over from close range. Their failure to add to the lead meant Gillingham were always in the game.
They nearly grabbed the equaliser. Vadaine Oliver hitting the post with an overhead kick from yet another long throw. It seemed to put Sunderland on the ropes and Gillingham looked to find the goal commiting bodies to the cause.
They had brought on John Akinde on midway through the second half to unsettle Sunderland and he nearly did just that. Using his strength he managed to get a second of space and looked to pull the trigger but Leadbitter managed to get his toe to the ball to clear it away and cull the danger.
But the sucker punch came in stoppage time. Akinde had dominated in the air since he came on and he sucked in the defenders before flicking on for Jordan Graham who'd gambled and was in acres of space. All he had to do was round Burge and smash into the net to restore parity for the visitors late on for the second season in a row.
Man of the Match: Grant Leadbitter
When Grant Leadbitter first came in many doubted him, it looked like one final move before retirement and it was questioned what he could bring. He didn't help that early on, often looking off the pace and letting issues off the pitch affect him.
It is credit to him then that he has revived his Sunderland career. Paired with the younger Josh Scowen who is willing to do the running he has flourished. He was the springboard for many Sunderland attacks and while his penalty was saved he did well to follow in for the goal. He looked as if his clearance would keep all the points in the North East but it wasn't meant to be.
The message is starting to get through
While Lee Johnson has started to bring in players that suit his system, one of his main jobs is to make the current players buy in now. It has took a few weeks but it looks like it starting to happen. When they attack they move the ball much quicker and look dangerous.
But one thing he has to address is the inability to kill teams off. Especially at the Stadium of Light they seem to become scared of the lead and drop off allowing the opposition to dictate the game. Whether it is a confidence thing or other factors Johnson needs to get to the bottom of it.