Tottenham are seven points behind top-of-the-table Chelsea after losing 2-1 to the Blues at Stamford Bridge on Saturday night.
The 21-year-old attacker has come in for some criticism this season as he struggles to scale the lofty heights which he managed so easily last term.
So far, the England international has scored three and set up one goal in 17 appearances across all competitions for Spurs - not the return that he or Tottenham fans would have been expecting following a fine first season.
The four teams above Mauricio Pochettino's side - Chelsea, Man City, Liverpool and Arsenal - all have some fine creative sparks in their ranks, and Townsend tells BBC Radio Five Live that is where the Lilywhites are suffering in Alli.
“Who is the special player behind Harry Kane for Spurs? Who’s the special one? Any of the top teams they have more than one outlet to make the difference.
“I think Spurs have so many very good players, I just don’t think they have that extra, that somebody else.
“Son is a terrific player, but I mean like a No.10. [Alli is] not a No.10. He doesn’t play with his back to goal – he’s great at running beyond people and he can score a goal. He’s great at that.
“[Alli] doesn’t play like a number ten like Hazard, he’s not that special player.”
In Alli's defence, he is not the only Spurs man who is struggling to reach the standards he set last season, with Erik Lamela and Christian Eriksen both dipping in and out of form, while Son Heung-min has gone a little quiet following his early-season heroics.
Pochettino's boys face Swansea in a must-win match this weekend. Defeat could see them fall seven points behind off the pace in the race for a top-four spot - and they face Manchester United at Old Trafford the following week.