A Sehrou Guirassy penalty was enough to seal the win for the German side, but Arsène Wenger's team were still confirmed as group winners after BATE Borisov's 0-0 draw with Red Star Belgrade.
Coquelin comes closest for Arsenal
The performance was rather typical of Arsenal's last two games in the competition as the Gunners' attacking play lacked impetus and energy in attacking areas.
Francis Coquelin came the closest to breaking the deadlock, and that just about summed up the action.
First he tried an effort from distance which flew just wide of Timo Horn's goal, before gathering the ball from Olivier Giroud's ball into the box to smash a shot against the post after half-an-hour.
Arsenal were limited to efforts from the edge of the box, and struggled to create many clear cut opportunities. Danny Welbeck checked back inside with 33 minutes gone, only to see his shot curl wide of the far post.
Mohamed Elneny meanwhile then saw an effort fly over the bar moments before the break.
Mathieu Debuchy continued his role at centre-back alongside Per Mertesacker and Rob Holding and they were able to restrict Köln to only one chance in the opening 45 minutes.
Jhon Córdoba scored a fantastic goal in the tie at the Emirates and had the chance to test David Ospina after just six minutes played but the Colombian stopper was equal to it.
Guirassy penalty gives hosts the lead
It took until past the hour mark for either side to finally break the deadlock and it came against the run of play with Köln taking the lead.
Guirassy got ahead of Debuchy before tangling with the Frenchman inside the area. The result was a penalty as referee Vladislav Bezborodov pointed to the spot.
The man who was fouled stepped up and took the penalty himself, planting the ball down the middle to make it 1-0.
The Gunners defence had received a warning moments before that however. as Milos Jojic played in Guirassy but his shot was tame.
In an attempt to get back into the game Wenger brought both Reiss Nelson and Eddie Nketiah on as the game reached it's conclusion. Ultimately neither could help their team go on and claim a result.
Nelson came close with a minute remaining, dancing through the Koln defence before poking a shot goal bound which was denied by Horn.
Jack Wilshere meanwhile almost found the net earlier in the second half, seeing a long range attempt beaten away by Horn before heading over from Calum Chambers's cross.