The answer to that question is crucial to the health of the struggling tech giant and the revitalization of the company's stock. A series of missteps - not the least of which was Microsoft's failure to recognize the mobile revolution until it was too late - had left the stock languishing for more than a decade.
And while Microsoft stock is up more than 40% in 2013, that climb was stifled over the summer by numerous downgrades.
With the company at a crossroads, whoever becomes the next Microsoft CEO will need to possess the ability to shake the tech giant out of its doldrums so it can better compete with the likes of Apple Inc. (Nasdaq: AAPL) and Google Inc. (Nasdaq: GOOG).
Not that it will be easy. The next Microsoft CEO would only be the third person to hold this position in the firm's 38-year history.
Microsoft's situation calls for one of two leader types: a turnaround expert capable of changing the culture and focusing on business development, or a technology visionary capable of guiding the firm in a rapidly changing world.
So far, the Redmond, Wash.-based company has cut down the list of potential candidates from 40 to "about five."
As talk on the Street continues to heats up over who that will be, I had a chance to speak with Money Morning Defense & Technology Specialist Michael A. Robinson about the narrowing list of candidates.
Robinson says there is one person with the turnaround experience and leadership knowledge to lead the software giant into the next decade.
And that person is going to need every ounce of experience and wisdom to rejuvenate this tired tech titan...
Challenges Facing the Next Microsoft CEO
The truth is, Microsoft is desperate for change.
Ballmer's looming departure has fueled negative views about the company's future. And it's deeply affecting business decisions.
Concerns about a CEO transition has factored into company downgrades by a number of Wall Street analysts, including Bank of America Merrill Lynch and UBS. Numerous reports suggest the company won't even make final decisions on major capital investments until it selects its next CEO.
But this is a short-term problem. It's the long-term transition to new consumer markets - primarily the red-hot domain of smartphones and tablets - that poses the greatest challenge to Microsoft and is delaying the search.
"The big question for MSFT is how to go mobile," Robinson told me. "Android and iOS pretty much own the market."
Robinson argues that the company has finally got mobile right. But now it's a question of implementation...
The $7.3 billion acquisition of Nokia's (NYSE ADR: NOK) devices and services unit, announced just a week after the news of Ballmer's departure, is expected to be a major catalyst that sparks Microsoft's true mobile surge.
"[Microsoft] made a huge step with the Nokia deal," Robinson said. "Now they have to integrate and execute. And pronto."
Meanwhile, Microsoft still needs to clear up some problems with the latest version of its flagship product, Windows 8. Customer complaints soured its launch, while the company's competitors continue to attack its grip on the office software market.
The Top Choices for Next Microsoft CEO
Before we get to who we think would be the best option for the next Microsoft CEO, let's take a quick look at the current runners-up.
Former Nokia chief and soon-to-be-returning Microsoft executive Stephen Elop is one reasonable choice, given his knowledge of the mobile market. The shortlist also includes Skype chief and current Microsoft business development head Tony Bates, as well Microsoft cloud and enterprise boss Satya Nadella. In addition, there is one more "mystery" candidate who Chairman Bill Gates has not disclosed.
But Robinson said the best choice would be current Ford Motor Co. (NYSE: F) CEO Alan Mulally, based on his knowledge of Microsoft and his reputation as a turnaround specialist.
"The challenge for MSFT is to find a high-profile CEO with lots of turnaround experience, but with some tech in his background," Robinson said. "Mulally has all of the above and understands consumer tech, since that's the route Ford has chosen with its new models that are clobbering the competition here in tech-centric California."
For a company desperate for a cultural change and a steep shift into new business areas, Mulally has the chops. He has been praised for his ability to change his current company's culture with his One Ford program. He greatly improved product quality, reversed Ford's financial losses, and avoided a federal bailout during the financial crisis.
In addition, Mullaly's focus on brand perception changed consumer sentiment on Ford products, something Microsoft could use in the wake of its Windows 8 debacle.
And after seven years in the auto business, and a stint as president and CEO of Boeing Commercial Airplanes, a division of The Boeing Company (NYSE: BA), Mullaly now may be seeking a serious challenge in the tech world.
Sure, Mullaly doesn't have significant experience in the software industry, but that isn't a vital requirement here. Mulally's role as the next Microsoft CEO would focus on managing and implementing change in a large organization with multiple business lines - something he's proven he can do.
And Mullaly already has some experience in changing Microsoft's culture, as he is said to have helped mastermind Ballmer's recent reorganization effort.
Although Mulally has not shown any sign he would leave Ford, his agreement expires at the end of 2014. Becoming the next Microsoft CEO may be a perfect challenge for a man who, at the age of 17, found himself eternally motivated by engineering's greatest challenge: President John F. Kennedy's goal of putting a man on the moon.
Don't be like Microsoft and get left behind in this next big tech revolution. Ultimately it will become a $14 trillion market - and it's only just getting started. That means there's still a lot of money to be made. Let Michael Robinson tell you all about it...