JC Penney stock wavered after-hours: Spiking 10 percent after the news of Johnson's departure, then sliding more than 10 percent. What's the stock doing now? Click here for the latest after-hours quote.
Ullman, who was CEO at the retailer from 2004 to 2011 before Johnson was tapped to succeed him, told Reuters that he was approached by the board to return to the helm of the troubled retailer over the weekend.
He said he wasn't sure if he'll stick to Johnson's shop-in-shop strategy but said, "There's nothing wrong with the idea of having attractions."
"We are fortunate to have someone with Mike's proven experience and leadership abilities to take the reins at the company at this important time," Thomas Engibous, chairman of the company's board, said in a statement. "He is well-positioned to quickly analyze the situation JC Penney faces and take steps to improve the company's performance."
For sure, Ullman has his work cut out for him.
"I think this announcements suggests to us that things are very much in flux at JC Penney," said Liz Dunn of Macquarie Capital. "I don't think they can simply return to the old strategy and all will be well. They've got some big questions to answer, and I think the release sort of hinted at that when Ullman said he's got some work to do to even figure out what's salvageable."
According to former JCP CEO Allen Questrom, Mike Ullman is "best choice right now" to lead the company, but not for the long term.
"Ullman knows the company, knows the people," said Questrom . "[It's] possible to get JC Penney back on track, but won't happen overnight."
(Read More: 'No White Knight' for JCP: Analyst )
Since Johnson took over in 2011, JC Penney shares are down 51 percent and the company's market cap has fallen $6.84 billion to $3.49 billion.
Johnson, a former Apple executive, had come under fire for his plans to revitalize the retailer by instituting a no-sale strategy and selling more fashionable merchandise. That strategy hasn't worked out.
JC Penney has lost money in each of the last four quarters. In the fiscal year ending February 2012, JC Penney revenues were $12.99 billion, down 27 percent from the last full fiscal year before Johnson took over (ending Feb. 2011).
On Friday, hedge fund manager William Ackman said Johnson, whom he handpicked to turn around the struggling retailer, has made "big mistakes" and the impact has been "very close to a disaster."
(Read More: Ackman Turns Up Heat on JC Penney CEO )
"One of the big mistakes was perhaps too much change too quickly without adequate testing on what the impact would be," Ackman said at an investment conference sponsored by Thomson Reuters.
Dunn told CNBC that there was probably disagreement on the board about bringing Ullman back. "I think it's probably pretty awkward. Bill Ackman has made a point in many presentations saying how the stock did not perform well under Ullman's tenure at the company," Dunn said.
Ullman told Reuters that Ackman's earlier criticism of his tenure was not "entirely balanced," pointing to the recession and subsequent drop in mall traffic.
But Ackman wasn't the only one. Disappointment in Ullman's appointment started trickling in shortly after the announcement.
"Mike may be a friendly face to some. However, I think Ullman offers the kiss of death wrapped in a smile," Brian Sozzi, CEO and chief equities strategist at Belus Capital Advisors, said in a note to clients.
"Ullman is someone I have dubbed Mr. Promotion," Sozzi said. "One has to believe that heading into the holiday season, he will ramp up discounts to lure back shoppers. Why is this bad? Simple, JC Penney has already plowed TONS of shareholders funds into shop in shops, so an environment of Ullman's 1980s promotional strategy sets the stage for poor returns on shop in shops investments right out of the gate," Sozzi said.
- Reporting by CNBC's Scott Wapner.
- Reuters contributed to this article.
- ‘No White Knight’ for JCP: Analyst
- Don't Jump Into JCP Stock: Pros
- Ackman Turns Up Heat on Johnson
- JC Penney CEO Received No Cash Bonus in 2012
- JC Penney Tries to Get House Together
- JCP's C-Suite Feels Hit After Dismal Year
- JC Penney Says Turnaround May Be Longer, More Costly
- Ron Johnson Out As JC Penney CEO: Source