If they don't, she expects that shareholders will be the ultimate losers.
"What you saw this quarter as you look at these banks is there's some group of them that are earning single-digit ROEs (Return on Equity) and they are destroying shareholder value every day they go to work," she said on "Squawk on the Street" Wednesday. "They really need to figure out how to get those returns up consistently."
Krawcheck says that there are two big questions for banks right now: First, can they grow through a cycle? And second, can they earn their cost of capital back?
She said that in recent years, the "go-to" strategy has been cost cutting. "That's a very difficult way to become great. What company every cut its way to greatness?" she asked. "What they have to do is recognize the changed environment."
(Related: Reading Through Bank Earnings Tea Leaves )
Krawcheck said that there were some positives in this past quarter, including individual investors coming back in the market, but the low rate environment posed challenges. "If you take this quarter as a pretty representative quarter, how do you grow in this environment? You've cut costs so far, but can you innovate?"
She added that what everyone previously thought was growth was actually just "increased risk, wrapped in complexity, riding volatility."
"This quarter, we really had the ability to catch our breath. There are no excuses anymore," she said. "How do you earn that cost of capital in a pretty good environment? The answer is: It is not increasing leverage."
(Related: Still Waiting for Fed's Emergency Lending Plan )
She said that banks need to go back to business school 101. "You're allowed to earn a lower ROE when your volatility comes down. All things being equal, volatility comes down, ROEs can come down too. You don't just have to increase your leverage," she said. "Increasing leverage and therefore increasing ROE is a false increase in ROE."
"The real question is can they regain their customer and clients' trust, can they do more for them, better for them and grow the business the real old-fashioned way, as opposed to just increasing risk either through leverage or risky products," she said.
Right now, she said, much of the innovation in the financial space is coming from smaller institutions and start-ups - Sallie Krawcheck currently consults for some of them. Without the bureaucracy, regulatory scrutiny and entrenched processes, they are "taking what was the little crumbs that fall off the table, but now are pieces of bread and sometimes a hunk of meat as well."
- By CNBC's Paul Toscano. Follow him on Twitter and get the latest stories from "Squawk on the Street" @ToscanoPaul