Beyond Software Development - Dancing with the stars

Predicting Software Success

Why nobody knows anything, and what you can still do about it.

What people want

Predicting the success of software applications boils down to the question of “what people really want?”, and how can we possibly predict it.
It’s easy to see why most people view the prediction of taste as an art. Historically, companies relied on the brilliance of tastemakers to predict and shape what people would buy. If KPMG and Ernst Young said CRM is the way to go, it did. Feelings were more critical than data. Harry Cohn, the founder of Columbia Pictures, believed he could predict how successful a movie would be based on whether his backside squirmed as he watched (if it did, the movie was no good).

Is new technology the answer ?

In the past, neither the creators nor the distributors of products have used analytics — data, statistics, predictive modeling — to determine the likely success of their offerings. Today companies have unprecedented access to data and sophisticated technology that allows even the best-known experts to weigh factors and consider evidence that was unobtainable just a few years ago, before, during or after the production lifecycle.
So, is the balance between art and science shifting ?

The short answer

Short answer first : "No"!

All software applications are radically different products. The success of one says little about the likely success of another.
Practically all expenditures on software development are made upfront. The cost must be sunk before anyone can tell anything about what can attribute to its success.
Software development is a work of art made by the contribution of many talents, whose efforts must be carefully coordinated. Slight changes in talent/coordination can give rise to non trivial changes in the final result.
Software developers really do not know the full nature of their applications until end users experience them. The basic problem is that end users don’t know what they like until they see it. Testing an application for the first time is a process of discovery, for both users and developers.
There is a tremendous high variance of the impact of a software development budget and application success. While sales tend to rise with the budget (especially marketing budget), many applications with big budgets have been major flops, while many with small budgets have been blockbusters (see open source).

Nobody knows anything

In the absence of an ability to know much about the demand for any given software application, the best prediction for any particular application is, more or less, a total guess. Economist Arthur De Vany coined the term "Nobody knows anything" to challenge the view that the success of movies is predictable, and he was once again correct with regards to software applications. The following is an updated version of the "nobody knows anything" principle of Richard Caves, a Harvard economist:

"Software Development Executives know a great deal about what has succeeded commercially in the past and constantly seek to extrapolate that knowledge to new projects. But their ability to predict at any early stage the commercial success of a new software application is almost nonexistent"

Bundling

Under market conditions of risk and uncertainty, one way to mitigate software development risks producing a number of software products and then selling them in bundles. The expectation can be that most of the applications developed for the bundles will be failures.
The portfolio profits from the few hits can more than offset losses from the many flops - It’s just nobody knows in advance which is which...

Take IT Farther