Watford led through Daryl Janmaat's third-minute header before Tom Cleverley was sent off on 87 minutes for two yellow cards.
Five minutes later Palace had completed a miraculous comeback to take them out of the bottom three.
Both goals were created by the skill of Wilfried Zaha as substitutes Bakary Sako and McArthur scored to complete the comeback.
One in for Palace, two for Watford
Christian Benteke kept his starting place for Palace despite his penalty misdemeanor in the weekend draw against Bournemouth, whilst Scott Dann came in for the injured Mamadou Sakho.
Jose Holebas took the place of the suspended Marvin Zeegelaar for Watford after his red card was upheld by the FA, whereas Roberto Pereyra was dropped to the bench in favour of Sebastian Prodl as the visitors reverted to a 3-4-2-1 formation.
Watford's formation change comes up trumps early on
Watford made the perfect start as their first foray forward resulted in the opening goal.
The visitors had switched to a three-at-the-back system after a 4-2-3-1 formation on Saturday and wing-back Janmaat was the man who opened the scoring.
Janmaat was on hand at the back post to finish off Richarlison's cross after the Brazillian was given time and space by Timothy Fosu-Mensah on the left.
Watford's record signing picked out the Dutchman who had coasted in unmarked at the back post and his powerful header gave the Hornets the lead in just the third minute.
Watford sensed blood with Richarlison causing all sorts of problems on the left and it was he who almost doubled the lead.
Andre Carrillo got in behind on the right before finding the Brazillian on the penalty spot, but Dann was there to clear the danger from inside the six-yard box.
It took Palace over 20 minutes to register their first meaningful shot through Andros Townsend, who fired wide from range after some good pressing high up the field from the hosts.
Palace earned themselves a foothold in the game and if it wasn't for Janmaat's superb back-post clearance Benteke would have levelled the scores on the half hour after Dann's flick-on from a corner.
Watford wasteful from point blank range
However, it was Watford ruing their luck seconds later and it was Richarlison once again who laid the ball on a plate for Troy Deeney.
The winger's fantastic ball into the six-yard box found Watford's captain just three yards from goal but he could only scuff his effort wide with the goal gaping.
After a scrappy start to the second half Palace grew into the game as chances came and went for Ruben Loftus-Cheek and Jeffrey Schlupp.
Janmaat and then Richarlison went close for Watford as the game started to open up.
Late drama sees Palace turn things around
However the match was to explode after Cleverley was sent off on 87 minutes for a second yellow card after he chopped down Sako, who was to make him pay just two minutes later.
They say you don't get lucky when you're at the bottom, but this wasn't the case for Sako who'd been introduced as a 70th-minute substitute.
The winger's shot was saved by Heurelho Gomes but bounced back off of the Palace man and went in for his first league goal of the season.
The move had been created by a mazy run from Zaha down the left and he was at it again in stoppage time, driving at the Watford defenders and firing a ball across the box for McArthur to tap in and seal the points at the death.