That little idea, of course, evolved into one of the biggest tech companies on the planet and made Ma a billionaire many times over.
Just like the garages where Hewlett-Packard , Apple and Google were started, the apartment in Hangzhou is now a shrine-another sacred spot in the tech world. People frequently journey to the site to see if they can sneak a peek, or perhaps set foot inside.
When CNBC was given an exclusive tour last week, a South Korean journalist was lurking about trying to get past an Alibaba tour guide/protector. But there is one major difference between Jack's place and all of those garages that are now protected by historical societies and have been turned into museums: Jack Ma's apartment is very much a working shrine. No dust is being collected there.
Years ago, Ma decided that he would send his most-talented programmers, Web designers and product managers back to the place where it all began in an effort to help them channel their inner Jack Ma. His hope was that anyone working at the 16-1 Lakeside Gardens apartment would be inspired to create the next best Alibaba product. It's not such a crazy idea. Some of the biggest and most successful products were launched in the apartment. Alipay, TMall and Taobao have made billions of dollars for the company.
The six-room apartment is a far cry from Alibaba's new campus, with all of its high-tech trappings. Employees there can use the state-of-the-art workout facility, indoor and outdoor basketball courts, foosball and pool tables, and eat at a multitude of restaurants and cafes that are on-site. The on-campus Starbucks was overflowing with employees when we were there covering Singles' Day on Nov. 11.
But according to the young people we spoke with at Lakeside Gardens, they were happy to work in Ma's old apartment rather than at the brand-new campus.
Read More Jack Man tops Forbes' China rich list
CNBC's David Faber asked product manager Chen Hang about that last week when we were at the apartment.
"Do people like working here? In the apartment? The headquarters is very nice. You got a gym, you can play pingpong, pool. There's nothing here," Faber asked.
"It's an honor for all people to work here," was Chen's quick and easy response.
The group of 30 will spend 10 months working and eating very closely together, in the same tiny kitchen where Ma and his wife ate their meals.
It was a surreal image-these brilliant, creative minds standing in the same spot where that other brilliant, creative mind came up with what is now Alibaba.
When CNBC was there, people were sitting shoulder to shoulder in front of laptops. A few people had their heads down on the table, recharging to get through the long day. Chen told Faber that his group works approximately 13 hours a day, seven days a week. They do some work outside the apartment on weekends.
There are pictures of Ma's early days hanging on the wall, clearly meant to inspire everyone in the apartment. While he doesn't show up that often there is a note on the wall, handwritten on March 15, 2008, by the founder and chairman. The rough translation, "The core of the company is to grow and grow."
Prophetic words from the man who went from this modest little apartment on Wen Yi West Road to become the richest man in China.