The Gunners were shaky in the first half, but then put in a more professional display in the second period to draw 1-1. That was before Unai Emery's men reigned victorious in a penalty shoot-out.
Antonio Rüdiger had put Chelsea in front, before Petr Čech saved from Alvaro Morata's penalty. Alexandre Lacazette was then on hand moments from time to send it to penalties.
Čech then saved again in sudden death of the shoot-out as the North London side won 6-5 on spot kicks.
Defence still an issue
One of the main questions on Arsenal fans lips when Emery arrived at the club, was would he be able to sort out the defence?
There is not a definitive answer to that yet, but early signs are not exactly positive. Against Atletico, the Gunners' defence didn't look totally comfortable, with Rob Holding struggling. When they played a youthful PSG side, they looked far more assured.
Chelsea, however, gave the Arsenal back line an uncomfortable evening, especially in the first half. Hector Bellerin, a player whose development has stuttered over the last 18 months was run ragged by Callum Hudson-Odoi whilst Sead Kolasinac was tentative on the opposite side.
The acquisition of Sokratis from Borussia Dortmund this summer looks a solid one, but it will take time before we see his best performances. He is the no-nonsense type of defender Arsenal need. He loves to defend, he claimed, when he joined the club and that will only benefit the club in the long run.
The defence may still be an issue, but going into the new season things need to be given time. It could take a while before habits picked up under Arsene Wenger are eradicated.
Petr Čech can battle for number one spot
A positive of pre-season have been the performances of Čech, who after finally being given the number one shirt looks more ready than ever to battle for a place in the starting XI.
Bernd Leno has done little wrong since signing for the club this summer but Čech has been impressive. He was good in the shoot-out against Atletico Madrid, but saved his best display in recent memory for against his former side.
He made a great save from Hudson-Odoi in the first half, and then got down well to save Morata's penalty. He followed that up by saving from Ruben Loftus-Cheek in the shoot-out too.
A host of Arsenal fans will think he is past it, and they have a strong argument. Some of his performances last season weren't good enough but there is plenty to be encouraged about with the club bolstering some good options in goal ahead of the new campaign.
Mesut Özil looks ready
Considering all that Özil has dealt with this summer he looks like a man focused and raring to go this season. He took a host of backlash following a picture of him with the Turkish president Recep Tayyip Erdogan and then following Germany's disappointing World Cup campaign he was subject to racial animosity which meant he quit the national team.
Arsenal have been hugely supportive of him since and Emery trusted him with the captaincy when Arsenal took on PSG at the weekend. Özil then followed that up with an assuring display against Chelsea.
The German has got up to speed very quickly during pre-season and looks like one of the fittest members in the squad despite not having as long a pre-season as most of his team mates.
In the second half he was at his creative best and played a brilliant disguised ball for Arsenal's equaliser. Following a corner he slid the ball down the channel into Reiss Nelson, who then put the ball on a plate for Lacazette.
There will more than likely be a spotlight on Özil's performances again this season, but only time will tell whether he proves his doubters wrong.
Matteo Guendouzi continues to impress
The 19-year-old Frenchman has adapted to life at Arsenal more than any youngster in recent memory. He looks confident in every single way, involves himself in banter around the training ground and has been brilliant on the pitch.
What's more, he shares a budding bromance of sorts with Lacazette and Pierre Emerick-Aubameyang. He certainly isn't shying away from the big stars at the club.
He came to the club as an unknown quantity but is already showing signs that he's ready to take on a role in the first team after being on the fringes of Ligue 2 side Lorient last season.
Playing in a deeper midfield role, he's been able to protect the back four, getting stuck into challenges, but he's also not been afraid to get forward and play passes on the edge of the area.
It was this sort of performance that he showed against Chelsea, where he displayed his impressive passing range and then scored a cool penalty in the shoot-out.
Mohamed Elneny slipping down the pecking order
Elneny has always been a great squad member to have around the club, and he's not done much wrong in his time with Arsenal.
However, the arrival of Guendouzi, Emile Smith Rowe's performances in pre-season and Ainsley Maitland-Niles continuing to impress have seen Elneny slip down the pecking order.
His game time was limited last season but despite that was given the number four shirt coming into the new season. It's not necessarily a mistake, but it is perhaps a surprise for a player who has never really sustained himself in the starting XI.
His performance didn't stand out against Chelsea on Wednesday night, and was outshone by Guendouzi, a player who appears to be in a similar mould to the Egyptian.
Arsenal still have Granit Xhaka and Lucas Torreira to come into the frame as the new season gets underway and with the younger players coming through pushing for places, Elneny could struggle even more in 2018/19.
The Gunners have plenty of options in midfield and going into the new campaign it will interesting to see who Emery prefers in the middle.