Duncan Watts at the MIT Technology Review, recently posted an article called ‘The Scientific Method in Business.’
He argues that business is frequently run on existing data or on the instincts of its leaders, both of which have drawbacks.
Replicating the conditions of a controlled experiment is often difficult or impossible in business or policy settings, but increasingly it is being done in “field experiments,” where treatments are randomly assigned to different individuals or communities. For example, MIT’s Poverty Action Lab has conducted over 400 field experiments to better understand aid delivery, while economists have used such experiments to measure the impact of online advertising.
One application he does not mention is in the development of products and services.
The approach designers take to their work has a lot of the scientific method in it. A designer might not tell that they go about creating a hypothesis and testing that hypothesis against other conditions, controlling for all variables except the one in which you are interested. They do, however, conduct a great many experiments.
We do gather understanding about a space. We also build a set of rules or an understanding of that space. Then we start to make guesses about what would happen if we introduced something new to that space.
We build prototypes and test them. We make models and put them in people’s hands. We walk through a process or a job in a new way to see how it turns out. Then, we act on what we learn—frequently by creating new models and trying again.
Learn. Conceive. Create. Test. Go back to step one.
This is the same king of thinking Eric Reis argues for in the Lean Startup.
As Duncan Watts points out, many other kinds of business decisions do not allow the luxury of that kind of experimentation.
Maybe that is why some companies avoid it in product development as well.