'You would expect to see the demise of some weak players in the market and then their clients would basically start migrating to other, stronger, banks'.
"The least profitable banks in my view in Europe will face this challenge (of scrutiny over their profitability) on a continued basis so I think consolidation in the European market is going to happen despite all attempts to try and prevent it by providing some national solutions, Weber told CNBC in an interview.
European banking stocks have slumped since the start of the year amid concerns that loose monetary policy is hurting its profitability.
And smaller players will continue to struggle, he believes.
"Europe is over-banked so you would expect to see the demise of some weak players in the market and then their clients would basically start migrating to other, stronger, banks," he told CNBC on the sidelines of the International Monetary Fund's (IMF) spring meeting in Washington.
"Now a lot of the rescue programs that are in place prevent that from happening so whilst all the banks are probably no longer discussing insolvency issues, profitability should lead to the exit of some non-profitable banks or less profitable banks, he added.
He added however that new regulation made it tough for bigger banks to acquire smaller rivals.
"We are already facing a reduction in our balance sheet," he said.
He argued that stronger players should be able to take over weaker peers and integrate them into their own businesses. That would give them a larger client base, create cost synergies, and create more profitability.
"And in that sense, Europe needs to move on and the banking sector needs to consolidate, Weber said.
The changes needed to stabilize the European economy go well beyond the banking sector however, he said, pointing to the need for all European countries to enact structural reforms.
At the moment, Europe is too focused on ultra-loose monetary policy, Weber said.
"We all know that structural reforms have long term benefits but come at a short term cost...so what we actually need to do is ...embark on the structural reforms and only to the degree that they have counter effects in the short term," he said.
"We need to bridge those counter effects by more expansionary fiscal and more expansionary monetary policy. So it's getting the sequence right."