(Read more: Frail EMs hot topic for G-20 meeting )
"I'm not sure less regulation is the answer, smart regulation is the answer," Cohn said at an event held in Sydney ahead of the G-20 meeting. "Right now we have different regulatory regimes out there and if we could come up with one smart harmonized regulatory environment that would be the best thing for everyone."
In the wake of the global financial crisis in 2007-2008, the banking sector has seen a wave of regulation.
"A lot of what we have done over the last 4-5 years has been good regulation, I don't want to undo that -- raising capital standards, liquidity," Cohn said. "What we're looking for is greater harmonization and greater consistent implementation around the world."
(Read more: Second wind for regulators leaves banks feeling bruised )
Asked about getting rid of the notion that banks are "too big to fail," Cohn said this was a big objective for regulators.
During the financial crisis, the U.S. government pumped $250 billion into a number of banks including JPMorgan, Citigroup, Bank of America and Goldman Sachs. European countries such as the U.K. also took up stakes in some banks to help contain the crisis.
Goldman paid back the $10 billion it received from the government's Troubled Asset Relief Program in 2009.
"It's a big objective of the banks, none of us want to ever be a burden on anyone. We want to be in a position to take care of ourselves and I believe we at Goldman Sachs are in that position," he said.
On the subject of recent volatility in emerging markets, Cohn said he would not describe heavy selling as a "rout."
"What we've seen is a new pricing, we've seen... the risk premium come back to the market, volatility come back, these are all natural phenomena as we return to a more normalized market environment where we don't have central banks of the world in a zero-interest rate policy," he said.
In a note to clients in December, the U.S. investment bank said emerging markets would continue to disappoint, recommending investors with a "moderate" tolerance for risk reduce their exposure by one-third of overall portfolios.
(Read more: Goldman: Cut your emerging markets exposure by a third )
- Writing by CNBC's Dhara Ranasinghe. Follow her on Twitter at @DharaCNBC