The Hammers have a history of beating the so-called "bigger clubs" before losing the next game against a supposedly inferior side.
Slaven Bilic's men could not have started the season any better, running out convincing 2-0 winners against Arsene's Wenger's title hopefuls at the Emirates.
Players and fans have been basking in the euphoria of the victory with media and neutrals tipping the east Londoners to have a fine season after seeing the performance.
But speaking to West Ham's official website, Senegal international Kouyate, who scored the first goal against the Gunners and was one of a number of star performers, is urging the club not to get too carried away by their fantastic start as they prepare to face Leicester City at Upton Park this weekend.
"Each weekend you have a match which is super difficult, there is no such thing as an easy game. We found that out last year.
“We have to do it on the pitch, not talk about it. We've got to prepare well this week in training, just as we did before Arsenal."
Right to be cautious
Kouyate was one of West Ham's best players last season and is a real fans' favourite at the Boleyn.
But he is right to urge caution as he has already seen what can happen when complacency creeps in at the club.
Last season should serve as a lesson for Bilic's side as they made their best ever start to a Premier League campaign to find themselves fourth by the new year only to end up finishing 12th.
The Hammers became only the third side, along with Southampton and Aston Villa, to be in the top four at Christmas and not finish there in the history of the Premier League.
Hammers must learn from the ghosts of teams past
Ten players have arrived over the summer which means expectations at West Ham are high on the terraces.
In fact they are probably the highest expectations have been since former Liverpool boss Gerard Houllier tipped Glenn Roeder's star-studded side including the likes of Joe Cole, Paolo Di Canio, Jermaine Defoe, Michael Carrick, Trevor Sinclair and Frederic Kanoute as dark horses for the Champions League ahead of the 2002-03 season.
And West Ham fans don't need reminding what happened at the end of that season.