The transaction is expected to close in the first quarter of 2014, subject to approval by Nokia's shareholders, regulatory approvals and other closing conditions, according to a statement.
Once the transaction is approved, several executives will transfer to Microsoft, including Nokia CEO Stephen Elop, Microsoft and Nokia said.
"For Microsoft, this transaction is the key next step in furthering the company's transition to a devices and services company," the statement added.
In 2011, Nokia teamed up with Microsoft and uses Microsoft Windows software to run its mobile phones. Finnish firm Nokia has struggled in recent years to keep up with rivals such as Apple and Samsung.
"It is a big surprise for the market to digest. About a year ago there was some serious discussion about whether Microsoft would buy Nokia's handset business and it didn't happen," Bob O'Donnell, program vice president, Clients and Displays at IDC in San Francisco told CNBC Asia's " Cash Flow ."
"To revisit that now is a huge deal. It is good for Microsoft in the sense that they control the hardware and they control the software - it's a very Apple like strategy," he added.
The start of the week has seen a couple of large deals in the telecommunications sector. Verizon on Monday agreed to pay $130 billion to buy Britain's Vodafone out of its U.S. wireless business.
"I think it [the Microsoft/Nokia deal] is a continuation of the M&A spree we've seen since the start of the year," Roger Nightingale, a strategist at RDN Associates, told CNBC.
"There is still a lot of liquidity and companies are using their borrowing capacity to buy up competitors. This trend is likely to continue," he added.
(Read more: Vodafone CEO: We will invest in ourselves )
Analysts said the reaction of other large tech firms such as South Korea's Samsung to the Microsoft-Nokia deal was worth watching.
"My concern is what about the other guys that have been trying Windows phones? We've seen Samsung, HTC , a couple of other large players are probably not going to be quite as interested now that Microsoft owns this new business," IDC's O'Donnell said. "That's the big question mark."
Samsung is expected to launch its much anticipated smartwatch later on Wednesday.
(Read more: Samsung seeks 'iPod moment' with smartwatch launch )
Some analysts were skeptical about whether the deal would be beneficial for Microsoft.
"This is too little too late. Bringing in a struggling company like Nokia and thinking it can rejuvenate Microsoft into a mobile company -- I doubt that will happen," said Trip Chowdhry, the managing director of equity research at Global Equities Research.
-By CNBC.Com's Dhara Ranasinghe; Follow her on Twitter @DharaCNBC