Need to improve
The Black Cats looked on course to record their first league win of the campaign, with a brace from Defoe, giving them a seemingly comfortable lead over Alan Pardew's side.
However goals from Joe Ledley and James McArthur brought them level, and they sealed a glorious comeback with Christian Benteke's stoppage time header, and after the game Defoe shared his frustration at the late defeat.
"When you have a good performance and come away with no points it is a difficult one to take," Defoe told the Sunderland Echo. "We are improving but when you concede 30 seconds after you score a second, it is not good enough. All the players know."
He continued: "You have to win the game when you are 2-0 up, especially when you need the points."
Defoe concluded: "It is hard to take because it is not like we were outplayed or they were the better team. Second half we created a lot of chances," and the striker admitted "Even if we had drawn, I would have been disappointed."
Taking the blame
It is has been far from the start that Moyes will have hoped for in his tenure at the Stadium of Light, having just won two matches so far, both been in the EFL Cup, and he shared his disappointment at the defeat on Saturday.
"To be 2-0 up at home and lose 3-2 was disappointing. I thought we were a wee bit fortunate to be 2-0 up when we were but nevertheless we were and we had a chance." He told safc.com.
Moyes concluded: "We need people to take responsibility. We need the players to have a level of responsibility that not everything is led by me or the staff."
Sunderland AFC will take on West Bromwich Albion at the Stadium of Light on Saturday, October 1 with kick-off at 3pm BST.