He told a conference of bankers in Germany this week that, in case they hadn't noticed: "If you go to a party these days, you're asked what you do and you say you're a banker, people go all quiet."
(Read More: Bankers Told to Watch What They Say at Bars )
Spare a thought for bankers this office party season, then. When they hit the bars of Manhattan or London, there could be even more potential for social awkwardness than usual. And with more job cuts and a muted bonus season looming, there isn't even as much cash for champagne to go around any more.
Bankers might hope that, five years after the credit crisis began, there would be worse professions to admit to - euro zone politician, perhaps.
Giles Keating, head of research at Credit Suisse (XETRA: dbk-de) , joked he usually tells people he's an economist or a researcher, although he'll call himself a banker sometimes. But he argued that the criticism is a little unfair.
"We all know that there was leverage, a bubble, and the banks were at the heart of that and it's natural to say that therefore bankers were at fault," Keating told CNBC. "There were a lot of other bad things going on the economy, as well. Bankers certainly were not angels, but governments borrowed too much, we consumed too much oil, and all sorts of things went well."
Manoj Ladwa, head of trading at TJ Markets, jokingly said he describes himself as a bookie at introductions. He thinks that the reputation of bankers will be restored if the focus shifts to a more traditional, retail customer-focused model.
(Read More: How Can Banks' Reputations be Fixed? )
"The banking sector has shifted back in the last couple of years. With the increasing likelihood of banks having to hive off the investment sides of their business, it's back to that more traditional retail banking, customer-focused approach. That's the way it has to go," he told CNBC.
-By CNBC's Catherine Boyle; Follow Her on Twitter @cboylecnbc