Slaven Bilic's side were poor for long periods and once again conceded late on at the Bridge to drop points, just as they did last season.
West Ham looked like they may go on to snatch an unlikely point when James Collins smashed home a superb half volley in the box to make it 1-1 in the second half.
Some lackadaisical play in possession on the halfway line gifted Chelsea a counter attack and Costa punished the Hammers with a fine finish into the bottom corner from outside the box.
West Ham's Adrian remonstrates with Chelsea's Diego Costa after the incident
West Ham left counting the Costa and Kante survived too
There was a lot of controversy after the game, though, as to whether Costa should even have been on the pitch to score the goal after a poor challenge on Hammers 'keeper Adrian when he had already been booked.
Earlier new signing N'Golo Kante also stayed on the pitch after scything down compatriot Dimitri Payet while also on a booking.
Sutton was Chelsea's record £10 million signing back in 1999 but stayed for just one miserable season where he scored once in 28 games.
West Ham United's Dimitri Payet is scythed down by N'Golo Kante
And he has slammed ref Taylor's performance insisting his old club got away with it in a post on Twitter.
"Kante and Costa should've gone and yes Taylor bottled it," Sutton said.
"Referees should have to explain their thinking after games."
Kante and Costa should've gone and yes Taylor bottled it. Referees should have to explain their thinking after games⚽️— Chris Sutton (@chris_sutton73) August 16, 2016
Reffing headache: West Ham manager Slaven Bilic
Stamford Bridge unlucky for Hammers
It is not the first time the Hammers have left Stamford Bridge feeling hard done by.
Bilic's side were just seconds away from ending a 14-year wait for a win over the Blues in West London last season when Antonio was adjudged to have tripped Ruben Loftus-Cheek inside the box.
Replays showed the contact was way outside and Chelsea scored the penalty to somehow rescue a draw from a game were they had been second best throughout.
Thanks to a poor performance from both West Ham and the ref, that wait could now stretch to 15 years.