Speaking at a Brexit legal debate, Mark Boleat, policy chairman at the City of London Corporation, said UK-based financial institutions would lose access to the single market, while some would eventually consider relocating elsewhere in the EU.
Thousands of jobs in the City's clearing houses — which sit between two sides of financial trades — could be jeopardised because the EU is unlikely to allow this function to continue outside of its borders.
The UK would also have to adhere to EU regulation without taking part in discussions which set the standards.
Boleat added that the City could be burned when Britain thrashes out a new trade deal with the EU following a Leave vote.
"Financial services are the part of the economy where trade negotiations stand the smallest chance of success," he said.
"There is no doubt that both the UK and the EU would have an interest in securing a trade deal as quickly as possible. However this would matter more to the UK than the EU."
But Vote Leave hit back, saying London's standing as financial capital of the world would be enhanced if Britain left the EU.
"Mark Boleat said we would be disadvantaged if we didn't join the euro. Now he’s making the same flimsy predictions about the referendum. He was wrong then and he is wrong now," Matthew Elliott, chief executive of Vote Leave, said.
"London is the financial capital of the world because of its global talent pool, its tax and legal system and the dynamic nature of our economy, not because of our EU membership. After we Vote Leave all those characteristics will be enhanced as we take back control."