The punditry duo were full of criticism for the midfielder following the loss to Manchester United on Sunday.
The midfielder was part of the side that were beaten 4-2 by their fierce rivals on Sunday afternoon, leaving City fourth in the Premier League table and trailing United.
The Ivorian was unable to make a positive impact in the defeat, failing to help his side deal with the presence on Marouane Fellaini in the midfield area, with the Belgian scoring a header to give his side a 2-1 lead and generally running riot throughout the game.
That has led to criticism of Toure from both Souness and Neville following the game.
“He’s a real conundrum, a real problem,” said Souness.
“He doesn’t want to take part in going back towards his own goal and doing the dirty side. I think that’s a problem they’ve had right throughout the last couple of seasons and maybe that’s one of the big reasons they’ve not done as well as they should have in European football.”
Despite being one of the best players in the Premier League for City over recent seasons, he has never truly looked comfortable in the central midfielder position, with him often having looked lost while playing there.
Instead he has always looked more at home in a more attacking position where he can have a bigger influence upon games, although in the past that has meant moving David Silva to an unfamiliar left wing role.
Neville agrees with Souness that Toure cannot play in midfield, and has suggested a system that could work in the future.
“I don’t think it’s a conundrum anymore; Yaya Toure cannot play in a two in midfield against opposition of any quality,” added Neville.
“There’s too many examples from watching him that show he cannot do his job. Mancini used to play De Jong and Barry behind him to protect him and that might be a way to go if he’s to stay at Manchester City.”