A second half penalty save to deny City's Charlie Wellings proved to be enough to ensure Liverpool returned to Merseyside with all three points.
Story of the match
Rinsola Babajide nearly gave Liverpool the lead in the opening few minutes after Furness played a fantastic through ball to the forward, her effort deflected off the inside of the post.
Leighanne Robe was forced into making a last ditch tackle in the box to deny a certain Robins goal after Ebony Salmon charged into the Liverpool box.
The visitors found a breakthrough on thirteen minutes as Furness drilled the ball into the back of the net from close range.
Melissa Lawley's cross was parried into the path of Niamh Charles who flicked it on to the midfielder at the far post.
Just shy of the 30 minute mark Olivia Chance ran at the Liverpool defence and her effort rattled off the crossbar, denying what would've been a fantastic goal.
Shortly after the Robins were forced into their first substitution as the injured Elise Hughes made way for debutant Megan Wynne.
It was a relatively quiet start to the second half, with City carving a decent opportunity from a free kick around the 60 minute mark.
Salmon lifted a fantastic effort just over the crossbar.
The hosts kept piling on the pressure, though, as moments later Salmon drew a save from Anke Preuss.
62 minutes in a scramble in the box saw Wellings fouled in the box to win a penalty for City.
Wellings stepped up to take but Preuss made the save and she then reacted well to get a hand the eventual rebound shot from Poppy Pattinson.
Liverpool were next to carve an opportunity on 71 minute as Babajide broke down the right hand side and cut inside onto her left foot, drawing a save from Baggaley.
Babajide was clean through on 80 minutes but was unselfish and attempted to cut the ball back to Lawley who had made a darting run into the box, but it was cleared by the City defence.
Disappointing for the hosts though after Meghan Sargeant was stretchered off late on with a head injury.
Soon after play got back underway the Reds had a chance with Furness who smashed an effort off the post.
Preuss was called into action once again on 90 minutes as she denied Wynne from close range in another scramble in the box.
Takeaways from the game
Furness scores first goal for Reds
Putting the ball in the back of the net is something Liverpool have struggled with this season, with only three goals to their name heading into the match.
New signing Furness made her second appearance for the Merseyside club on Sunday afternoon and gave them the lead on thirteen minutes.
The midfielder headed straight for the touchline towards the backroom staff in her celebrations, evidently a special moment for the 31-year-old.
Robins unlucky in front of goal
It wasn't to be for the hosts after a number of chances went begging across the 90 minutes.
They had the chance to draw level before half time after a fantastic solo effort from Chance rattled the crossbar.
The saved penalty in the second half and a fantastic late save from Preuss to deny Wynne exemplified a resilient defensive display from Liverpool.
Liverpool move above City
The reds earned their first league win of the season and move above City on goal difference.
Both teams sit on six points and return to action in the Women's FA Cup on Sunday January 26.
Bristol City host Durham whilst Liverpool welcome Blackburn Rovers.