The Hammers will face Blackburn Rovers at Ewood Park in round five of the competition after being taken all the way by a much-changed Liverpool side, who also welcomed back a few of their more stand-out players to the action.
The first half saw the woodwork struck no fewer than three times as both sides had their chances but it was West Ham who struck the first blow just before half-time as Michail Antonio smartly finished to give them a half-time lead.
The visitors struck back shortly after the break as Philippe Coutinho played the percentages by stroking his free-kick under the wall for Liverpool to level, before he made way as Daniel Sturridge and Divock Origi both returned from injury.
Again both sides had their chances, notably Christian Benteke for Liverpool, while West Ham ended up going more and more direct as the game neared the end but ultimately Ogbonna had the last word with a terrific header from Dimitri Payet's excellent delivery.
Here are some of the best images from last night's action: