It's the first time that Huddersfield have made it past the third-round since the 2017-18 campaign, where they were eventually knocked out in the fifth-round by Manchester United.
Story of the match
Burnley were without their manager Sean Dyche, as well as several first-team stars, who were out with positive coronavirus tests.
The return of Nick Pope was a big boost, however, as the patched up team looked to get back to winning ways.
Sitting sixth in the Championship, Huddersfield are certainly no mugs and they made a purposeful start to the match.
Koroma was looking lively on the left of an attacking trio and the 23-year-old curled an early shot just wide from outside the box.
The hosts have been struggling for form in the Premier League and it showed, but Rodriguez quickly started to make a nuisance of himself on his return to the line-up.
The former West Bromwich Albion marksman sent a low driven strike zipping past the far post after a neat tee-up from Chris Wood. He then forced Ryan Schofield into a smart save low down to his right after bursting into some space in the box.
Schofield then got into a mix-up with Tom Lees, colliding with his team-mate in an attempt to deal with Josh Brownhill's looping punt into the air, but Rodriguez couldn't bring the ball under control with the goal at his mercy.
In the end, Rodriguez did have his moment of glory. It was a well taken finish from the Burnley striker, spotting Ashley Westwood's first-time scoop over the Huddersfield defence before directing a header away from the onrushing Schofield.
Just minutes later, Wood thought he was going to be the recipient of another direct ball over a high defensive line but the New Zealander could only clatter into the unfortunate Schofield - attempting to sweep up the danger.
It proved to be one collision too many for the visiting goalkeeper, and young Nicholas Bilokapic was summoned off the bench for his professional debut.
Huddersfield hung on in and, on the stroke of half-time, they could have levelled.
Jack Cork came on for Wood at half-time, with the striker perhaps feeling the ill-effects of that Schofield collision, as Burnley added another man to the midfield.
Pipa dragged a shot wide after a quickly taken corner but the Terriers were struggling to create too many openings.
However, visiting manager Carlos Corberan made four second-half changes and Sobra Thomas, one of those substitutes, made a big impact.
After being played in behind following a mistake by Lowton, the Welshman drilled the ball across the six-yard box and Koroma couldn't miss with the goal gaping.
Pope made an excellent save to deny Danel Sinani after a ball deflected into an appetising position as the Terriers looked to turn the screw.
Burnley brought on 18-year-old Owen Dodgson for his professional debut and the youngster showed some nice glimpses in an unfamiliar left-midfield position. It did, however, highlight the skeletal nature of the playing squad and it did little to change the tide.
Momentum was with the visitors and, invariably, it was the underdogs who managed to snatch the game at the death.
Thomas whipped in a lovely corner and Pearson rose the tallest to plant a header into the corner. VAR was looking for a foul but all Burnley protests were rebuffed.