Who is going to be held accountable ?
"So, the question for Citigroup is: Who is going to be held accountable at Citigroup? And I'm not talking about someone way far down the totem pole. I'm talking about who with the public face to investors, to shareholders, is going to be held accountable for this mishap at Citigroup," he said.
On Wednesday, the Federal Reserve rejected Citigroup's capital plan and barred it from increasing both its dividend and stock repurchases.
Read More Fed rejects Citigroup capital plan
On CNBC's " Halftime Report ," Mayo took direct aim at Gerspach's track record.
"Well, it's baseball season, and three strikes, you're out. So, I think it's John Gerspach, the CFO, needs to go. Citigroup needs to change the CFO, bottom line," he said.
"John Gerspach, the CFO, was the chief accounting officer going into the financial crisis, when Citigroup had financial mishaps. Two years ago, Citi was turned down for its capital plan by the Fed. That's strike two. And then yesterday, being turned down again is strike three."
Citigroup did not immediately respond to CNBC's request for comment.
Mayo also called on Chairman Michael O'Neill to reach out beyond the company.
"Mike O'Neill, chairman of Citigroup, go ahead and call the large investors today," Mayo said.
"Why isn't Citigroup having a conference call today? Where's Citi's conference call? If this was Jamie Dimon - for anything I might've said about JPMorgan and Jamie Dimon - at least Jamie Dimon would have a conference if he had a mishap like this," Mayo added.
Despite his criticism, Mayo said he was still positive on the stock, on which he holds a $58 price target and a "buy" rating.
"We all know it's cheap, but they have some nice businesses," he said. "They generate a 2 percent return on assets in their global consumer basis and in their transaction-processing business. The investment bank has great potential to be streamlined, and the legacy assets from the financial crisis are running off, and as that runs off, they will have a nice benefit to earnings over time. So, the potential is there, and the sum of the parts is worth at least more than 50 percent over where the stock is trading now. It just needs someone to release that trapped value from Citigroup."
Mayo does not hold a position in Citigroup or JPMorgan.
Another expert who expected financial stocks, especially Citigroup, to do well was Bill Miller, former chairman and CEO of Legg Mason Capital Management.
"They're cheap historically," he said on CNBC's Closing Bell Thursday, noting that dividend growth from bank stocks would be among the best in the market over the next few years.
Citigroup, he added, was a value today.
"Citibank's value didn't change today from yesterday 5 or 6 percent. And more importantly, Citi's going to have a $58 tangible book by the end of this year," he said. "They'll probably earn $7 in two years. They'll trade, in my opinion, at probably 10 or 11 times."
Miller said that he expected Citi shares to hit $70 in 24 months.
Read More Bill Miller: I'm investing in bitcoin
"I think it's going to beat the market," he added.