The Citizens took a 2-0 lead through Sergio Aguero and Aleksandar Kolarov before goals from Tom Cleverley and Carlos Sanchez appeared to rescue at least a point for Villa.
However, moments after Sanchez’s late equaliser, City midfielder Fernandinho put his side back in front with a last minute winner but not before the game’s most controversial moment.
Villa striker Christian Benteke was onside when he rounded England goalkeeper Joe Hart and was subsequently brought down, though play was brought back.
Replays prove that the Belgian international was onside – it should have been a penalty to Tim Sherwood’s side and an ensuing red card for Hart, but it wasn’t to be.
Speaking in the Sky Sports studio after the match, Redknapp felt sympathy towards the visitors but stated their bad defending is what ultimately cost them at least a point from the Etihad stadium.
"The referee's official has had a bad day and I can understand Tim's frustration because they put so much into the game,” Redknapp commented.
"But the goals they conceded were really poor. Once you get back to 2-2, I think then you've got to try and hang on for at least a point.”
"The goals they've conceded have cost them and I'm sure that will be something Sherwood will be trying to sort out in training for sure."
The defeat leaves the Villans in 15th spot in the Premier League table, just two points above the drop zone with their relegation rivals having played a game fewer.
Up next for Villa are two back-to-back home games against Everton and West Ham United, fixtures Sherwood will be targeting at least four points from in order to move away from the bottom three.