The man in charge awarded Spurs a debatable spot-kick in the first half, which Dele Alli scored to make it 2-0.
Marriner had made substitute Shane Long wait on the sideline, despite being ready to replace injured Manolo Gabbiadini, as the home side won the penalty.
The Birmingham-based referee also disallowed an onside Southampton goal in last month's EFL Cup final.
The Japanese international said, “We showed a good comeback in the second half.
“But first half they played much better and we should have played like the second half but from the beginning.
And the 28-year-old also bemoaned the goals his side conceded.
“We keep conceding a lot, which we have to stop now.
“The first goal was individual ability from Eriksen and the second one, I still have doubts about the penalty."
Speaking directly about the penalty incident, Yoshida said, "I couldn’t see the situation, so I can’t say exactly but I feel it is unfair.”
Chance for redemption
Southampton players must wait until after the international break to get back their league season back on track but Yoshida is confident that once the domestic campaign restarts again, the Saints will be ready.
“Now we break for the internationals and we can come back and have many games in April, so we will be ready.
“Nobody will be injured and we will be fresh.”