Veteran defender Rolando sealed Marseille's place in the Europa League final with a 116th-minute goal.
Despite a dramatic comeback by the hosts to bring the aggregate score level at 2-2 it was Les Olympiens who prevailed.
A goal from Amadou Haidara and an own goal by Malang Sarr put Salzburg back in the tie but they were punished deep into extra-time.
After looking so comfortable in the first-half, Marseille seemed to crumble under the pressure of a Salzburg side dreaming of a place in the final.
Marseille will meet Atletico Madrid in the final to be held in Lyon after the Spanish side saw off Arsenal.
Haidara pulls one back
20-year-old Amadou Haidara gave the Austrian side a fighting chance after his second-half goal reduced the deficit by half.
It was a cool finish from the youngster who drove into the Marseille box and poked the ball beyond Yohann Pelé with his right foot.
Despite such a poor first-half display from the home side they gave themselves a chance through scoring early on in the second period.
All square
Only fourteen-minutes after the opening goal, Malang Sarr put the ball into his own net to equalise the tie.
After some dreadful Marseille defending, the ball was fired back across their box and the French youngster poked the ball beyond his own goalkeeper.
The initial shot from Xaver Schlager was going well wide but the Marseille defender's instinct took control as he prodded it in.
Rolando seals the deal
Against the run of play, veteran defender Rolando volleyed the ball home to send his side to the final.
A superb corner from Dimitri Payet fell straight at the feet of an unmarked Rolando who finished it like a veteran striker.
Marking Rolando's fifth ever European goal it will most certainly have been the most important of his career.
The substitute centre-back provided the travelling fans with a moment to savour despite the goal marking Marseille's first shot on target.
Deep into extra-time, there was no way back for the hosts.
Slow start from Salzburg
After a measured start by both sides, it was Salzburg who created the first attempt on goal.
Moanes Dabbur raced through the middle of the park to latch on to the ball but his shot was struck straight at Yohann Pelé.
Although it should have been an easy save for the French goalkeeper the ball escaped his possession and was scrambled clear by Luiz Gustavo.
Rudi García's men looked content with their two-goal aggregate lead and the frustration of the Salzburg faithful was only growing.
Les Olympiens back-line stood strong with no way through for the Salzburg forwards.
Despite the home fans creating such an electric atmosphere before the game their players were unable to replicate their energy on the field.
New lease of life
The 52nd-minute goal gave the home side a new lease of life and gave their fans something to cheer about.
The regained confidence transformed the game as Marseille immediately looked vulnerable at the back.
Dabbur had a golden opportunity to equalise the tie on aggregate but instead of opting for the easy finish his unnecessary showboating ended in complete embarrassment.
Suddenly the Salzburg side looked recognisable to the side that dramatically came back to beat Lazio in the previous round.
They were quickly granted their reward as an own goal by Sarr equalised the tie.
For a team which looked so comfortable in the first leg and for the first-half of this game, Marseille were in a state of shock.