The Hammers were interested in signing Austin from Queens Park Rangers in the summer of 2015, but they did not go through with the deal due to a knee injury sustained by the striker earlier in his career, according to The Daily Mail.
The West Ham co-chairman has also claimed that he would have made a move for the 26-year-old had he known that he was available for £4 million.
According to The Guardian, Southampton paid QPR £4m as transfer fee in order to secure the services of Austin last month.
“Our next league game is away at Southampton and I just want to clarify my views on the Charlie Austin situation,” The Daily Mail quotes Sullivan as saying in the programme for West Ham's 2-0 win against Aston Villa on Tuesday.
“Contrary to reports, he is a player I have always greatly admired and his climb through the divisions is inspiring. I once said at £12m he was a gamble, with his injury record.
“However, if his financial demands are reasonable, Southampton have got fantastic value at £4m. Had I known he was available at that price, I would have taken the gamble myself, and I am sure half the Premier League clubs would have as well.”
Austin is a proven goalscorer, and found the net as many as 18 times and created 23 chances in 35 Premier League games for QPR last season.
Despite his injury problems this campaign, the former Burnley striker scored 10 goals and created 17 chances in 16 Championship matches for Rangers before his move to Southampton in the January transfer window.
Austin has already scored for Southampton, finding the winner in the 1-0 victory over Manchester United away from home at Old Trafford in the Premier League on January 23.