Joe's younger brother Nicky is an ardent Chelsea fan and was cheering on the Blues from the East stand in the recent games at Stamford Bridge.
Sibling rivalry is one thing, but when your brother plays for the opposition it is stretched to the limit.
"He gave me a bit of stick last week," admits Joe, "but we'll just keep quiet and see how it goes at Upton Park before we say anything next week."
Joe admits pride was restored somewhat on Saturday, but he will not even dream of gloating until and unless victory is gained over Chelsea in the replay next Wednesday.
"I was trying to stay away from him last week because he has been caning me," says Joe.
"We will just have to wait and see until after the game and are just keeping our heads down.
"But I haven't got anything to be ashamed of, none of us have, now we have put on a good performance away from home."
Talking of familial strife, incidentally, there are those at the club who are surprised by a Sunday newspaper article on the day of the league game which featured an interview with Chelsea star Frank Lampard.
It said that his family were reluctant to go to Upton Park earlier in the season when, in fact, Frank senior did attend the game.
Meanwhile, Joe feels there was a marked improvement after half-time on Saturday, and says: "Jimmy scored a wonderful goal and you can't do anything about that, but I thought we were excellent in the second half when we just destroyed them and played some good stuff.
"They are a quality team and we will have to be on our game when we get them back to Upton Park next week.
"Now we will wait to see what happens - the saga continues.
"We can't be making any predictions, we'll just have to wait and see, but with a lot of big clubs out any team can fancy it."
Joe is full of praise for Fredi Kanoute's performance, and adds: "We kept at it and Fredi got a good, tidy goal - and nearly nicked it at the end.
"That's what we've got to get him doing more, facing the other way, facing their defenders, because when he is on form he is frightening.
"We've just got to keep pushing him to do that and he will hurt teams."
It was in marked contrast to the 5-1 defeat the week before, but at least new father Fredi could escape criticism for the league loss, thinks Joe.
"Little 'Ibu' has been keeping him up at night and I think that's what happened last week, which is why he wasn't on his game!"
Joe knows the job is only half done, and adds: "We know what they are all about with Frank, Jimmy, and Zola, and look at how much money Chelsea have spent compared to us.
"But we tried to play a system where we thought we could get the best out of our players and the longer the game went on the more it was working."
Joe was quietly impressed with Vladimir Labant's debut for West Ham, and adds: "He is a good, tidy player who has played Champions' League football, and he has a good left foot.
"We'll have to see how he develops in the next few months, but from what we have seen in training he looks an excellent player."