Why Good Ideas Fail
Sometimes an excellent software product comes to market, gets rave reviews, and then disappears. This happens regardless of a clear need and want for the product. The problem is not specific to any particular software domain; BeOS, and Lotus Improv were both great products but seem to have nothing in common apart from being good ideas that failed.
Lotus Improv introduced a radical spreadsheet metaphor, and although BeOS was not as revolutionary it too brought plenty of good ideas with it. Both products received extensive media coverage and favorable reviews, but made no impact on the software market.
Why does software that everyone seems to like and want die? Surely, the ideas behind the products and the implementations can not be at fault. Ideas from such software have been recycled successfully in vastly inferior forms by competing products: Microsoft Excel's pivot tables were inspired by Lotus Improv, but have been a big success. BeOS too has yielded many nice ideas; some of these ideas have been copied and others are being copied.
Was poor marketing the reason for the demise of BeOS and Lotus Improv? Marketing is always a factor in the success of a product; however, these products did receive extensive favorable media coverage. Better marketing might have enabled these products to do slightly better, but it is unlikely that marketing alone could have rescued them. So, what exactly did go wrong?
Software users tend to stick with software they are familiar with. This is not irrational behavior; software is complex to learn and use productively, and people prefer not having to relearn computer skills over and over again. Most people are inquisitive about new products but can not see straight away if these products carry any productivity benefits. Moreover, the utility of new functionality often becomes apparent only after individuals get accustomed to it.
People do adopt new software but may require a very long time to do so. Rapid adoption of new software occurs only if there are no pre-existing substitutes for the software. When the PC software industry was young many software companies went through rapid expansion because their products had no counterparts. As the PC software industry matured, software companies found it much harder to sell new ideas. For instance, Lotus 123 was the first good spreadsheet available on the PC and it became an instant hit, but MS Excel took a long time to displace Lotus 123, even though it was a superior product. Excel would have taken even longer to displace Lotus 123 if Windows 3.0 had not come along. The popularity of Windows 3.0 changed the market in favor of Excel, and thus accelerated Excel's acceptance.
Good ideas fail mostly because they are not allowed sufficient time to succeed. More time allows an idea to mature, and also enables it to benefit from favorable marketplace changes. Additionally, the backers behind the idea get to learn from failure and are able to market the idea better.
The time needed for a new product to establish itself is mainly a function of the effort required of users to switch to the new product and be productive. Image viewers and the like can be instant successes, but more sophisticated applications need many years to gain a foothold. The experience of Linux suggests that a new OS can require more than a decade to gain credibility.
Lotus and Be both blundered by not allowing their products sufficient time on the market. Be was financially constrained and had to call it quits; however, Lotus acted in an incredibly stupid manner by killing the company's only product with any chance of success.
The practice of discontinuing good ideas prematurely is widespread in the software industry. The story of BeOS and Lotus Improv has been repeated so often that innovation in the PC software industry has stalled. Software companies are convinced that users are indifferent to good ideas, and money spent on developing new ideas is a waste. The OS scene is one big casualty of this attitude of the software industry.
The lack of innovation in the OS scene is certainly not due to a lack of ideas. Good ideas are so abundant that people need not even look for them. An entire book, Unix Hater's Handbook, is dedicated to pointing out all the things Unix got wrong. Some of the shortcomings mentioned in that book have been addressed, but others simply cannot be addressed in the framework of current generation operating systems. Of course, no software company has the courage to explore these ideas.
Things are hardly any better in the application software market. Only recently, a new startup, Quantrix, reintroduced the ideas pioneered by Lotus Improv (see the Quantrix tour for an overview of Lotus Improv ideas) to the Windows market. For almost a decade nobody had the courage to touch an insanely cool idea.
Software companies love to blame ideas instead of management for product failures. The industry simply does not understand that ideas need time to succeed, and has a hit or miss attitude towards products. This attitude is a consequence of the initial success of the PC software industry. The PC software industry is barely 25 years old and in the initial 10 years of its inception many products became instant hits. This initial phase of the software industry established a culture of impatience. Although, the PC software market has changed and products rarely become instant hits nowadays, the old habits continue to persist.
Only Microsoft seems to be immune from this culture of impatience. Unlike, Lotus and most other software companies who achieved instant success, Microsoft went through 5-6 extremely lean years after the founding of the company; this seems to have shaped the management culture at Microsoft. Unfortunately, Microsoft has primarily limited itself to copying successful ideas, and is not interested in radically different ideas.
Apart from commercial software another plausible source of innovation in the software industry is non-profit software development; however, non-profit development is biased towards cloning commercial software. Non-profit software development is about creating the most amount of utility for the most amount of people, and this goal is best served by copying successful ideas. Going for radical innovation is a suboptimal use of the very limited resources available for non-profit development. Linux exists because Linus wisely chose not to be ambitious. Had Linus tried fancy ideas, Linux might not have existed today.
The future of software looks bleak; Microsoft and other big software companies are unwilling to back good ideas, and non-profit developers are unable to do so. Can the software innovation stalemate be broken or is software innovation dead for good? Part II of this article will examine the issues the software industry needs to tackle in order to bring innovation back to the software scene.
by Usman Latif [Feb 29, 2004]
Why Good Ideas Fail, Part II
OSNews Discussion for "Why Good Ideas Fail"
Lotus Improv from Wikipedia
Ars Technica BeOS Station
Spreadsheet Analysis from Winners, Losers, and Microsoft
Unix Hater's Handbook,