A Richarlison header with 25 minutes to go proved to be enough to see all three points go to the visitors, even though Mbaye Diagne thought he had rescued a point for West Bromwich Albion late on, but his strike was ruled out for offside.
With this result, it now means that with 11 games remaining the Baggies remain in 19th position, nine-points adrift from safety.
"They should not have lost this game"
After the game, when speaking on how this defeat could impact their fight for survival Allardyce said: "It is a massive setback for us in terms of the result.
"A massive disappointment for the players in the dressing room, they know they should not have lost this game, but we did.
"A lot of it is we have to take the blame ourselves by not finishing opportunities or not making more opportunities."
He added: "But on the other side, Everton are riding high but created little or no chances tonight and the goal was only half a chance."
"But even then, we have scored a great goal and it was half a toenail offside sadly, so if you are on the right side of that, it brings you a point, if you are on the wrong side, you lose points."
"We have not got the finishing touch"
Following yesterday's result, it confirms that West Brom currently obtain the third-worst goalscoring record in the league, after only bagging 20 goals all season.
Then after missing a host of opportunities against the Toffees primarily through Diagne, Allardyce said: "We haven’t got the finishing touch, unfortunately, and without the finishing touch, it means that we have ended up losing this game that we should not have lost, and have not even drawn it".
"We are a bit hard done with Mbaye and his goal because it is that small margin. It is not a blatant offside when you first look at it.
"It looks like it is onside and them lines are so defined and there are so small margins in it, it is very disappointing when it goes against you."
The Baggies boss continued: "After that, a point would have been more than satisfying when you score it late on, and it was something that we really did deserve.
"We did not deserve to lose this game, but we did and that is something that is difficult to take on the performance that we gave tonight.
"But, we need to pick ourselves up for Sunday and try and deliver that performance and hopefully, that will give us the victory that we need."
VAR is not making life consistent
In the first couple of minutes of the game, the hosts had a massive penalty claim when Diagne went over under a challenge from Mason Holgate, but the appeals were waved away.
When commenting on this incident, which in the end proved to be defining, Allardyce said that VAR is making the Premier League more inconsistent than it was before it arrived.
"We feel hard done by, but most of it Is our own fault and some of it is the officials not giving us a penalty, particularly in the first few minutes on the tackle from Mason Holgate on Mbaye (Diagne).
"While I have not been managing over the last two years or so since VAR has come in, I have seen 50, 60 penalties given for a lot less than that.
"There was contact on his leg, no contact on the ball. He brings the player down and low and beholds it doesn’t even go to VAR"
The 66-year-old continued: "So, next week I will see two penalties given for similar incidents and until then, it boils down to the lack of inconsistency again.
"So VAR is not making life consistent. It is just bringing as much inconsistency than it did even before it arrived."