"We didn't play well and we let ourselves down against Birmingham - that is the bottom line," he says.
"We have got to stick together; we are all 100 % behind the gaffer, every single player, and we have to make sure we dig in and fight because we all want to do well for West Ham.
"All the players are disappointed but we are all prepared to work hard for the gaffer - we all know he is the right man.
"You don't go overnight from being a bad team, it is just that little things are not clicking for us this year and we have to get out of the situation we have put ourselves in.
"What more can we do? We are working hard and we have to keep on at the training ground to put things right."
Joe, watched by Sven-Goran Eriksson ahead of next week's international fixtures, is hoping - like the other England candidates - that an assessment of their form will not just be based on Saturday's defeat, but also on the win at Chelsea the week before.
Did Joe fell the Hammers had 'cracked it' then?
"It was only one game, 90 minutes," he says.
"We did well last week but we haven't this week."
As for the reaction to his being substituted in the second half, a decision which was met by a chorus of boos, Glenn Roeder says:
"People didn't know the full story - he did very well even to come out for the second half."
And Joe himself adds:
"It was the right thing to do because I had picked up a knock - a dead leg."
There was support for West Ham from one of Saturday's opponents; former winger Stan Lazaridis, who was so influential down the left flank, says:
"I just wish the club well; I have very happy memories of the place and I hope they stay up for a start.
"But they are at least a mid-table side without a doubt and I could still see them going for Europe."