Allardyce's arrival at the Stadium of Light has increased optimism on Wearside that the Black Cats can get themselves out of trouble, and Merson fears that could spell disaster for Sherwood.
The former Arsenal midfielder claims that before Allardyce arrived, Villa would have been confident of finishing ahead of Sunderland, but says they can no longer feel assured of that.
"What will put more pressure on him is that Sam Allardyce has gone into Sunderland," Merson said. "Two weeks ago, the Villa board would have said 'we'll finish above Sunderland'.
"But now, you're sitting there going, 'Sunderland will probably get out of it, they've got Sam.'"
Sherwood's future may rest on securing a positive result against Swansea City in Aston Villa's next Premier League fixture.
Reports in the Daily Mirror have suggested that the Villa board are starting to lose faith in Sherwood, but Merson believes that sacking him would be a major error.
"It would be a joke, if they sacked him," he added. "He's only just come in at the back end of last season. Has he become a bad manager overnight? Surely, they have to stick with him.
"Swansea at home isn't an easy game, and they've got hard games after that, but they have got to stick with him."
Villa's frustrations with Sherwood stem from a terrible run of form, which has seen them fail to collect a victory since the very first day of the season.
The Claret and Blues were triumphant over Bournemouth in their opening match, but their eight-match winless run has left them four points adrift of safety, with relegation fears rising at the Midlands club.