The Serbia international, who missed a late penalty against Manchester City two days ago, met Andros Townsend's pass and superbly curled the ball across goal and into the bottom corner from 25 yards.
The Eagles had fought back from a goal down having gone behind courtesy of Shane Long's first half strike, his first goal in almost eleven months.
James McArthur equalised for Palace after half-time and the game appeared to be heading for a draw until Milivojević netted ten minutes from full-time.
Roy Hodgson's side now find themselves fourteenth with a two point gap between themselves and the bottom three. However, they can also begin to look up with just a three point margin between themselves and tenth-placed Watford.
Meanwhile, Southampton are above the relegation zone on goal difference.
Pellegrino now under pressure
Saints boss Mauricio Pellegrino will no doubt feel the full force of the defeat which leaves his side battling for their Premier League status.
The Argentine replaced the sacked Claude Puel in the summer and was expected to improve the club's league form, but Southampton are now without a top-flight victory since they defeated Everton 4-1 in November.
A clean sheet in the 0-0 draw with Manchester United in their previous outing was their first since October, and Pellegrino will be worried about the defensive issues that were evident tonight as they shipped two very avoidable goals. Palace's first goal came when a half-cleared free-kick was crossed back into the box by Townsend and met unchallenged by Christian Benteke, before McArthur was given too much room to volley past former Eagles goalkeeper Alex McCarthy on 69 minutes.
Milivojević was then given the freedom of an almost 10 yard diameter to run on to a Townsend pass and sweep the ball beyond McCarthy. Pellegrino will be furious at the amount of time and space his team allowed Palace to score their two goals and cannot aim blame at the congested fixture list given Saints had 24 hours extra rest than their victorious opponents.
The only positive for Southampton was that Ireland forward Long ended his goal drought. The former Reading man had failed to score since netting in a 4-0 victory over Sunderland last February, but today gave his side the lead when he met Jeremy Pied's low cross and finished via the post.
Apart from the goal, though, Southampton rarely tested Palace goalkeeper Wayne Hennessey. Pierre-Emile Højbjerg drew a good save from the Wales international but other attempts from Oriol Romeu and a couple from Long were wasted. A new striker in January has to be a priority, particular with Charlie Austin ruled out for a month and Manolo Gabbiadini failing to continue his goal-scoring form from last season into his first full campaign in a Saints shirt.
The main question will be whether Pellegrino will still be in charge when that signing is made. Southampton are currently in a similar predicament to the one Stoke City find themselves in: there are no other viable alternatives to their current boss. The only advantage that Southampton have over Stoke - who have struggled under Mark Hughes - is that they are prepared to appoint a foreign manager, whereas it is believed the Potters would prefer a British or Irish candidate to take over from Hughes. The Saints' latest defeat will certainly leave many question marks hanging over Pellegrino's head.
Victory a sign of improvement for Palace
It seems far longer ago than just four months when the Eagles lost their opening seven games of the season.
Hodgson's first match in charge was a 1-0 defeat to Southampton in September and his current side is one so far away from the dull and defensively abysmal team that began his reign in disappointing defeat.
The Palace boss deserves vast credit for the job he has done at Selhurst Park. The South Londoners are now an exciting team to watch and are generally defensively sound, and can put any mistakes tonight down to having played out an exhausting but very credible 0-0 draw with Man City two days ago.
Hodgson appears to have put an arm round the shoulder of many of Palace's squad players, including Bakary Sako who replaced Jeffrey Schlupp on 64 minutes and had a big influence on the game. Another, Patrick van Aanholt, was excellent against the Citizens on New Year's Eve and had a good second half against the Saints after a disappointing start to the campaign, in which he has lost his place to Schlupp.
Hodgson's biggest task, though, remains rekindling the form of Benteke, who still has just one goal this season. Palace's only fit recognised striker appears bereft of confidence in front of goal, but will take pride in setting up McArthur's equaliser. The Belgian also forced McCarthy into an excellent save moments before Palace scored their first of the evening.
Palace travel to bitter rivals Brighton & Hove Albion on Monday in the third round of the FA Cup. It is a certainty that they will go into that game with plenty of confidence having lost just once in eleven games. A stark contrast to their early season form.