Tiafoe had lost all three previous meetings against the South African and it looked like it would be four from four trailing by a set and 3-0 in the second before turning around their second-round clash.
The American broke a six-match losing streak against top ten opponents dating back to last February and he will face Italian veteran Andreas Seppi in the third round.
Anderson gets off to quick start in first set and a half
Anderson was quick off the mark, holding and earning a break point on Tiafoe's first service game only to see the American erase it with an ace to hold and make the score 1-1. It was the world number 39's turn to see a break point, but as he had done, saw it vanish with an ace.
Neither player had another look at a break chance until the tenth game with Anderson leading 5-4. After wasting a game point, Tiafoe was faced with a set point against him and a backhand crosscourt winner from the fifth game gave him the opening set.
Again, it was the South African off to a fast start in the second set, holding to love and getting to 0-40 on the American's serve. After dumping a backhand into the net, Anderson drove a forehand down the line for a winner and a 2-0 lead. He saved a break point in his next service game to extend his advantage to 3-0.
Tiafoe turns things around to level the match
The turning point in the match came when Tiafoe dug out a hold after being pushed to 30-30 on his serve and holding two break points in Anderson's next service game, he drilled a forehand and finished off the point with an overhead smash to get back on serve at 3-2.
Again, both men enjoyed a spell of easy service games until Anderson faced four break points in the ninth game. After saving the first three, he flubbed a backhand, sending it into the bottom of the net to give Tiafoe his first lead of the match at 5-4.
Serving for the set, the American blasted an ace to get to 40-0 and after another deep, powerful forehand that Anderson could only make a futile attempt to return with one hand, the match, which seemed to be under control by the South African, was suddenly one set all.
American grabs two sets to one lead
It was Tiafoe applying early pressure to begin the third set, holding a break point before Anderson used his forehand to climb his way out of trouble and post an opening hold. He was quickly ahead 40-0 only to see the American win six straight points, capped off when the fifth seed missed a backhand volley to give Tiafoe a 2-1 lead.
The world number 39 began to settle into a nice serving rhythm, holding with more ease and mixed up his plays, utilizing his backhand to great effect. Another weak forehand by the South African off of a strong serve put Tiafoe a game away from the third set at 5-3.
Anderson held to make it 5-4 and Tiafoe stepped up to serve for the set. At 30-15, the American fired an ace to bring up two set points and after shoveling a backhand crosscourt for a winner, the world number 39 held a two sets to one lead, on the verge of the biggest win of his career.
World number 39 claims memorable upset
The start of the fourth set saw Tiafoe create a break opportunity right away with another blistering forehand winner. Anderson eventually took advantage of some conservative returning by the American to hold on for 1-1.
Again, both men were settling into a nice serving rhythm, neither player coming close to a break for the majority of the set. Leading 4-3, the South African had a half-chance at 30-30, but more clutch serving got Tiafoe out of trouble and he held for 4-4.
Each competitor held to love to make the score 5-5. The decisive game came next with the world number 39 showing his touch, throwing in drop shots and volleys while staying aggressive. It paid off as Anderson double-faulted on break point to give Tiafoe a 6-5 lead.
Serving for the match, the American fell behind 15-40. After saving the first break point, he ripped a backhand winner, capping a 17-ball rally to get back to deuce. Two points later, Tiafoe played his first serve-and-volley point of the match to seal a memorable victory in two hours, 56 minutes.
"It means the world to me", Tiafoe said after the match. "I lost to Kevin three times last year. I was down a set and a break today. I went to a different place. I dug deep. It's just how bad you wanted it, and I wanted it bad".