Shkodran Mustafi's header had opened the scoring but Victor Camarasa was on hand to equalise moments before the break.
Pierre Emerick-Aubameyang put the Gunners back in front in the second half but once again Cardiff fought back through Danny Ward.
However, Alexandre Lacazette popped up late on to put Arsenal ahead for the third time to seal victory.
Bullet header opens proceedings
Arsenal were ahead within 11 minutes. A nervous Cardiff backline gave away a corner, and from the resulting set play, Granit Xhaka's ball in was headed home superbly by Mustafi.
The Gunners continued to press and should have been a few goals to the good. Matteo Guendouzi's crossfield pass wasn't dealt with by Bruno Ecuele Manga and Nacho Monreal should have scored but for the gloves of Neil Etheridge.
Lacazette, making his first start of the season, could have made it 2-0. He turned on the edge of the area and was allowed time and space to have a shot, but his attempt came back off the post as another positive move for Unai Emery's team went begging.
Cardiff hit back
Cardiff began to come back into the game well and after a strong period finally got their goal.
Xhaka played a terrible ball straight to Junior Hoilett and then the ball was worked out to Joe Bennett on the left. He swung a ball in and on the verge of half-time Camarasa was in the right place at the right time to fire home the host's first goal of the season.
It could have been a far worse half for Arsenal, though. Petr Cech gifted an opportunity to Harry Arter which wasn't taken in the opening ten minutes, whilst Bobby Reid was proving lively.
His acrobatic effort after 29 minutes went over the bar, but before that, he almost slipped in Ward as his fizzing cross only just evaded the striker.
Arsenal forwards down spirited Cardiff
The home side went behind again just after the hour mark, Aubameyang finding the net for his first goal of the season with a brilliantly curled finish from the edge of the area.
Mesut Ozil found space in the middle of the park to lay the ball into Lacazette who then clipped it into Aubameyang as he made it 2-1.
It didn't take long for Neil Warnock's men to fight back once more. A free-kick was given after Arter went down and then a well flighted set piece was headed back across the box by Sean Morrison. He picked out Ward who lept well to head past Cech and in off the post.
The game had a very familiar feel to it for Arsenal fans. On numerous occasions in recent season's under Arsene Wenger the Gunners struggled in these types of matches and they were feeling the pressure once more.
Cardiff almost took the lead for the first time in the game as they seized on a Mustafi mistake on 79 minutes. Reid couldn't take advantage though as he drove towards the area and planted an effort at Cech.
With just under ten minutes remaining Arsenal found a winner. Lucas Torreira, who had a brilliant cameo off the bench, slipped the ball down the right channel into Lacazette.
Sol Bamba failed to get close enough to him and that allowed the Frenchman to rifle a shot in at the near post.
Gunners claim the points
Arsenal had nine minutes to hold on and almost let their lead slip again in injury time.
Cech was dragged out of his goal by a ball to the back post and then once it had been worked to Morrison a few yards out, he headed over.
It was an incredibly gutsy performance from Cardiff but ultimately the class in finishing was the difference as Arsenal won for just the second time on the road in the Premier League this calendar year.