Modeling, prediction and validation are matters of choice

Today, at the protein folding meeting at IMA, there was an interesting discussion on the relative roles of theory, computation and experiments. I am impressed by the omni-presence of this general question. It shows up in discussions with every sort of scientist in a wide variety of domains – hard and soft! As always, there was some griping about the politics of the situation. However, I was much more impressed by a comment made by a young faculty member from Wisconsin who pointed out that most of her students were simultaneously doing theory and experiment so that the whole question becomes somewhat irrelevant.

In the same discussion but on a different note, Sorin Istrail (a computer scientist and one of the organizers of the meeting) made an interesting point. Borrowing the phrase from a paper (Oxford Economic Papers, 32(3):353, 1980) by Amartya Sen, he pointed out that description (i.e., modeling) is a matter of choice – as is the process of using the description to make predictions and validating those predictions by experiment.

To understand the essential issue, consider the following (from Sen’s paper):

A description can be accurate without being a good description. It could be unhelpful, even useless. We question the expert on the level of factory ages in India. He answers: “Oh, it varies from place to place.” True enough, of course. We ask for more description, demanding precision. The expert now goes into details. “The integer approximation of the national average wage in rupees,” he says, “is a prime number”. I won’t belabour the point further. Clearly, truth isn’t a sufficient condition for a description to be good.

Of course, in this example the absurdity is clear. However, there is a lot of theoretical work that is meaningless in exactly the same way. And, as a corollary, there are many ‘experimental’ results that are equally pointless. I am beginning to become increasingly convinced that the best way to avoid these dual traps and to do work ‘that matters, somehow’ is to be a theorist and an experimentalist at the same time – so that the questions are honest and answers rigorous.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s