"I wish Jamie and I had been able to work out our issues and that it didn't have to end up in a break up, because it was a very good relationship," Weill said.
Dimon was fired by Weill shortly after the Travelers and Citicorp merger in 1998-abruptly ending a 15-year partnership that saw them build a financial services empire like no other at the time.
Weill's appearance Wednesday on CNBC's " Squawk Box " came after he was named No. 21 on the CNBC First 25 list of business movers and shakers, compiled in conjunction with CNBC's 25th anniversary.
Reflecting on his decades-long career, Weill said the other big mistake he made was choosing Chuck Prince to succeed him as Citigroup boss because the pick led to departures of many top executives.
Prince could not immediately be located by CNBC for comment.
On the issue of banking, Weill said regulators should help strengthen the financial industry and get banks to follow the rules rather than be adversaries of the companies they oversee.
Case in point, the latest stress test results embarrassed Citigroup CEO Michael Corbat, who was blindsided by news the bank failed the test while he was out of the country, Weill said.
Regarding Dimon's handling of JPMorgan during this ligitious envirnoment, Weill said: "I think he's being shot at by everybody. He was too successful."
But Weill didn't fault Dimon for the bank's $13 billion settlement with federal and state authorities to resolve allegations of sales of shoddy mortgage securities by Bear Stearns and Washington Mutual, which JPMorgan purchased during the 2008 financial crisis at the behest of the government and the Federal Reserve.
"You can't fight the government," Weill said. "I think Jamie did a great job managing JPMorgan through the [2008 financial] crisis. He bought two companies that helped the country."
In a 2012 interview on CNBC, the man who invented the financial supermarket called for the breakup of big banks . But a year later, he told CNBC that big institutions don't have to be split if the "right regulation" is in place.
On Wednesday, Weill said: "The model of having a broad-based institution is still a good one," provided the government doesn't keep second-guessing why banks do business and how much they pay their leaders.
Looking ahead at the next 25 years, he said he's an "incredible optimist" about the United States. "We're acting like a young country."
"We are coming up with more new ideas, more new entrepreneurs, in spite of all this stuff [in Washington]," said Weill. He added that changing the nation's immigration policy would help keep overseas workers who were educated in the U.S. here to apply their skills.
He sees U.S. growth about 3 percent for the rest of the year, as the economy makes up for the impact of the harsh winter weather at the beginning of 2014.
"I think things look very good. Companies are doing well," he said. "[They have] a lot cash. Companies balance sheets are very strong. Interest rates are very low."
Weill said the stock market is not rigged-as Michael Lewis asserted in his new book, "Flash Boys." Lewis' claims touched off heated debate on Wall Street in recent weeks about the role of high frequency trading in the market.
"Our markets are really very, very efficient. Warren Buffett was on [CNBC] talking about the cost of doing business has gone down a lot," Weill said. "When I was a broker originally, stocks traded at an one-eighth of a point or a quarter of a point differential. Now they trade at one-tenth of 1 cent differential."
Read More Rigged markets? Mary Jo White says no
He's not bothered by high-frequency traders who make money getting ahead of the trade. "To pay a half a cent more ... that is meaningless compared to what you should be investing. You should be really be investing for the long term."
-By CNBC's Matthew J. Belvedere