Sunderland fell behind against the Gunners when they allowed Joel Campbell to score.
The Black Cats, however, were otherwise very impressive during the contest - but they were unable to stop themselves from falling to a 3-1 defeat.
"The only thing that let us down, we slipped up for the first," he is quoted as saying.
Allardyce also bemoaned Sunderland’s lack of ruthlessness at the Emirates Stadium.
The Wearsiders had a number of presentable opportunities but the likes of Duncan Watmore, Steven Fletcher and Fabio Borini all missed their chances.
"Our real failing was not scoring at the times we created those chances,” Allardyce said.
"Borini could have scored and two more after that.”
In the end, Arsenal made Sunderland pay for their profligacy and their quality shone through.
Goals from Campbell, Olivier Giroud and Aaron Ramsey sealed the Gunners’ triumph but the Black Cats should still feel encouraged by their display.
Sunderland have improved considerably under Allardyce’s guidance and there is now genuine hope they will be able to survive the battle against relegation.
Next up for the Black Cats is a Premier League clash against Watford, who are in good form. Allardyce will have plenty of positives to take into that from the Arsenal game, though, despite defeat.