The Millers had hit the front in the first half through winger Ben Wiles but were unable to hold on to their lead.
The result leaves Carlos Corberan's side in 18th while Rotherham move up to 12th.
Story of the game
Town were nearly off to the perfect start after two minutes. Isaac Mbenza cutting the ball back towards Fraizer Campbell after a wayward shot from Harry Toffolo. Campbell looked set to convert but Rotherham defender Angus MacDonald was able to get his body in front of the shot and deflect it for a corner.
Monsoon-like rain had made the pitch slick and passes were skidding across the turf. Carel Eiting on his Huddersfield debut after arriving on loan from Ajax looked calm, making some good passes but as a team Town were unable to break down the Rotherham blockade and the game was stop start for long periods of the half.
Josh Koroma on his return to The New York stadium after helping The Millers to promotion last term was cutting a frustrated figure early on, being forced to swap flanks with Mbenza before the 20 minute mark.
Rotherham were playing exclusively on the counter breaking from Huddersfield's poor corners. Freddy Ladapo looked to be in from Eting's wayward cross but Juniniho Bacuna did well to recover and cut out the cross.
Town were overcompicating their play, Naby Sarr kept looking for the diagonal towards Koroma and Mbenza in attempt to play over the press but with a large height disadvantage it was baring little fruit.
The turf had an effect in the thirty third minute, a set piece is half cleared by Sarr on his Town debut, Huddersfield failed to get out to Ben Wiles whose strike zipped past an unsighted Ben Hamer into the bottom corner. It was Rotherham's first shot on target and the first chance of note in over 20 minutes. Not much Hamer could have done as the bodies in front of him meant it was impossible to anticipate the strike.
Rotherham tails were up and they looked to add a second. Ladapo who had a great game and was a nuisance to Town was released after a quick counter. The former Crystal Palace & Plymouth Argyle man looked to jink his way into the box before firing and forcing Hamer to make a good save tipping it over the bar.
At half time Huddersfield will have been disappointed. They started so well, dominating the first ten minutes but Rotherham were able ride out the storm and come into their own towards the end of the half and deserved the goal. Rotherham were well drilled and Town hadn't moved them around enough in an attempt to break their shape.
Town came out for the second half, having more of the ball but they struggled to test Jamal Blackman in the Rotherham goal. It took them twelve minutes to create a meaningful chance. Eiting played a ball over the top to Koroma to run onto all he had to do is make a clean contact or square it to Mbenza but he did neither and Blackman was able to collect.
Koroma wasted another chance a minute later, sliding to reach a Toffolo cross but he could only put it wide of the post.
Rotherham began to take their time, Blackman taking an age to take his goal kicks, lots of fouls. It was frustrating Huddersfield who were losing their composure and were unable to get into a rhythm.
It was then Bacuna who had a couple of chances. Firstly from a corner the ball found it's way to him but he was unable to get a sight of goal before Campbell was deemed to have fouled Blackman.
Town's number seven then received the ball just outside the box and tried to find the bottom corner but was only able to fire wide.
It took till the 82nd minute for Huddersfield's best chance of the half. Good play between Jonathan Hogg and Eiting released Spain U-21 international Pipa down the right, his fierce shot looked destined for the near post but Blackman was able to turn it round the post.
With seconds on the clock Town equalised. It was backs to the wall for Rotherham who had nearly completed the job but the pressure told. Pipa on the right unleashes a shot which takes a massive deflection off Angus MacDonald past a helpless Blackman in the Millers goal.
Full time: Rotherham 1-0 Huddersfield Town (Wiles 33')
Rotherham- Blackman (GK), Harding, Ihiekwe (C), A MacDonald, Mattock, Wiles, Crooks, S MacDonald (Lindsay 75'), Sadlier, Hirst (Smith 52'), Ladapo (Vassell 70')
Subs- Johansson (GK), Wood, Barlaser, Miller
Huddersfield- Hamer (GK), Pipa, Stearman, Sarr, Toffolo, Eiting, Hogg (C), Mbenza (Diakhaby 83') , Bacuna (Daly 78'), Koroma, Campbell
Subs- Pereria (GK), Olagunju, Duhaney, Brown, Crichlow-Noble
Man of the Match- Carel Eiting
In a dire game where quality was trumped by grit, Carel Eiting was a standout. Having arrived from Ajax on loan he comes with a pedigree which he lived up to. His passing was excellent, every half chance Town created he was involved in. He certainly has the ability to unlock a defence and could be crucial to Huddersfield throughout the season.
Without Karlan Grant it could be a long season for Huddersfield
The stats spoke today, 73% possession, one shot on target. Two goals in four games, these are relegation numbers. Fraizer Campbell up top struggled against the height and strength of MacDonald and Michael Ihiekwe and he no longer has the pace to turn them. Karlan Grant carried Town last season and was expected to do the same this term. His head has clearly been turned and the club needs to decide whether to force him to play knowing his heart isn't in it or sell him and reinvest the money. The concern is they won't be able to find a player near his quality who is willing to buy into Carlos Coberhain's project.