The shock result had many pundits and fans tipping the east Londoners for big things this season with Bilic's new emphasis on attacking football.
But the Hammers were brought crashing back down to earth after the euphoria of the Arsenal result when Leicester City beat them 2-1 at the Boleyn Ground on Saturday.
So what did we learn about West Ham from the game?
1) West Ham MUST sign at least one top quality striker before the window shuts
West Ham could be in for a long hard season if they do not bring in at least one decent striker before the window closes. The club's co-owners David Sullivan and David Gold have added eight players to Slaven Bilic's squad this summer. But realistically only three of those are improvements on the first team in the shape of Dimitri Payet, Pedro Obiang and Angelo Ogbonna. Despite promising to beef up the striking department, though, it is the key area where the Hammers are sorely lacking. It is so bad that forgotten strikers Mauro Zarate and Modibo Maiga and now regularly featuring in the first team. While Zarate has proved with his performances that he has a place in the squad, fans are furious they are having to watch the likes of Maiga disgracing the claret and blue shirt in the final season at their beloved Boleyn Ground. The Hammers needed at least one new striker anyway, that was before Carlton Cole was released and Enner Valencia was ruled out for three months. What the Leicester game showed is that if sides can frustrate Diafra Sakho West Ham do not have much of a goal threat. If Sullivan and Gold truly want to see West Ham progress they must stop cutting corners when it comes to the most important part of the team and splash out for a quality forward before the close of the transfer window. Not a desperate loan, not taking a chance on a free transfer of some unknown from South America, but a proven player who will get them goals consistently.
2) Oxford is the future but Obiang is present
One of the stars of West Ham's win over Arsenal was Reece Oxford, the 16-year-old who shot to fame on his Premier League debut after keeping the likes of Mesut Ozil in his pocket. Could he go out there again against Leicester and show it wasn't just a flash in the plan was the next question being asked of the hugely promising starlet. Well in many ways the answer was no. While Oxford did not do anything particularly wrong, he looked a little lost in the central midfield role where he had shone so brightly at the Emirates. He gave the ball away in some dangerous positions and did not really get into his stride before being substituted at half-time.
In defence of a player who I have no doubt will go on to become one of West Ham's finest of a generation, he is a central defender playing out of position and everyone must remember just how young and raw he is while he was also playing at home with the added pressure that brings. While it may have knocked his confidence, Bilic did the right thing to withdraw him at the break with new signing Pedro Obiang making his Premier League debut. And what a debut it was, showing his supreme class on the ball and sparking a Hammers fightback as the east Londoners dominated proceedings after the break. Although fans were disappointed to lose to the Foxes, many were buoyed by Obiang's excellent performance and he will surely start against Bournemouth next time out.
3) Adrian's red leaves West Ham bang in trouble at the back
As if the 2-1 defeat wasn't a bad enough start to the last ever season at Upton Park, goalkeeper Adrian will now miss three games after his late sending off. Last season with the dependable and vastly experienced Jussi Jaaskelinen as back-up it would not have been cause for concern for the club. But now they have summer signing Darren Randolph to rely on after his free transfer from Birmingham City and he has been decidedly shaky so far. A string of mistakes and nervy displays in pre-season friendlies and the club's ill-fated Europa League qualifying campaign has left supporters with little or no confidence in the Irishman who will now be thrown into the deep end of the Premier League unless West Ham can re-sign former 'keeper Rob Green from Queens Park Rangers, as has been rumoured in the likes of the Evening Standard.
4) Bilic must sort out ill-discipline as it could cost the Hammers dear
If Slaven Bilic has a to-do-list at West Ham, sorting out the club's lack of discipline should be right near the top of it. Adrian was sent off for a high boot in the closing stages of the Leicester defeat, the club's fourth sending off in eight games under their new manager. While it could be argued that a couple of those sending offs have been somewhat harsh or debatable, what we must not forget is that last season the Hammers were the best behaved team in the Premier League, a fact which earned them a qualifying spot in the Europa League. While that particular prize won't be up for grabs anymore as FIFA have scrapped awarding clubs with European places for the Fair Play Rankings, it could end up costing West Ham crucial points in a season the players and supporters should be enjoying given it is the last ever at their Boleyn Ground home of 112 years.
5) In case there was any doubt, Payet is the star West Ham fans have been waiting for
After playing a starring role on his Premier League debut against Arsenal at the Emirates, star signing Dimitri Payet was once again top class for West Ham every time he touched the ball against Leicester City. He scored a composed goal to get the Hammers back in the game in the second half and looked a superb acquisition. Fans are salivating over his style of play and he is a player that very much epitomises the 'West Ham way' like Brooking, Devonshire, Berkovic, Di Canio and Benayoun before him. West Ham have already discovered that opposing teams will come to Upton Park – particularly after their headline-grabbing win over the Gunners – and sit back hoping to catch the Irons on the break.
When West Ham are struggling to break the visitors down, as they did against the Foxes, Payet will be crucial in helping unlock the door and it is vital they keep him fit as much as possible. Everything good West Ham did went through him but without some quality additions up front Bilic's side may become over reliant on the France international, which as long as he is producing performances like this will not be too much of a cause for concern.