The former Tottenham star also insisted Mauricio Pochettino be given time.
Ferdinand was brought into the club as a coach under Harry Redknapp, having made nearly 120 appearances as a player, and joined Tim Sherwood on the bench as an assistant during the former Blackburn captain's few months in charge at White Hart Lane last season.
And after Sherwood was dismissed in the summer, Ferdinand and fellow coach Chris Ramsey were also released.
But the changes in the coaching setup, with Southampton boss Mauricio Pochettino being brought in as head coach, have not helped the club settle down after a couple of years of upheaval in at both management and playing level.
The former England striker claimed that it has led to Spurs losing their identity and direction.
"You feel for Pochettino because it seems Tottenham for a long time are trying to change their identity and they have done that now," said Ferdinand.
"No one knows what they do and how they are doing, what style they are playing and what they are doing at the moment."
Ferdinand is eager for the Argentine currently in charge of the club to avoid the same fate he and Sherwood suffered under Daniel Levy, despite questions already being asked of the former Espanyol defender.
"Pochettino has come in and inherited players that perhaps he wouldn't have chosen himself but they were there and he has had to work with them and hopefully he will ge given the time.
"There is unrest there at the moment because results are not going the way they want but he's come in and needs to change what he is doing and get his own players and once he has done that then you can judge him.
"It's difficult to judge him on the strength of what he has come in to."