wisdom and the limitations of science

In perusing my blogroll I came across two different posts that, when combined with my experience of being a mother, make an interesting illustration for the importance of wisdom when applying and interpreting evidence.

Alice Callahane, at Science of Mom wrote about researching and trying to understand sleep choices for infants.  I'm far from an expert on this subject, but I believe this is a fairly well researched, but highly polarized subject.  In regards to this, she says,

"In the real world, lots of factors determine how different families handle sleep. Infant temperament is one of them. Safety is another. There are also cultural expectations, family dynamics, work schedules, parenting styles, and feeding methods to factor in. Where baby sleeps is a complex parenting decision, one that is exceedingly difficult to study in a quantitative manner. ..I also recognize that this parenting decision – and most of them for that matter – can never be reduced to science and statistics..." (emphasis mine)

I too often fall into this trap of believing that decisions can be reduced to science and statistics. It's easy to read a study and then conclude that this study must necessarily apply to everyone I know.  If the study design was good, and the statistics were also pretty clear, why shouldn't I be able to generalize the results to everyone? I believe there are both extrinsic and intrinsic reasons against being too liberal in my study generalizations.

The intrinsic reason against applying a study too generally is that population-based studies are, just that, population based.  Because of this, they show what is best for the population, and not necessarily for a particular individual.  If I make too many particular conclusions from a population based study I commit the whole to part fallacy, what is true of the whole (population) is not necessarily true for every part (an individual).  Population based studies don't, and can't, look at every factor that effects every person, like in the sleep example above.

The extrinsic reason against over-generalization is closely related to the intrinsic reason, but instead of focusing on the limitations due to the actual science, the extrinsic reason focuses more on your personal experiences and preferences.  

The importance of considering personal experience in decision making was driven home to me when I read a post from Kyte, a friend from high school, where she candidly wrote about some of her experiences of being a new mom. My experience has been very different, but it's not that hers is the "right" experience and mine the "wrong", or vice versa. They're simply different experiences.  And, I'm sure, at least partly because of these different experiences, we'd make different decisions for our families, even if we had the same exact evidence available to us.  That's just the way decision are, even evidence-based ones.

These are some of the reasons I've been thinking about the limitations of science and how this effects evidence-based decisions.  Next time I'm tempted to judge someone for making a different decision than I would have, I'll try a little harder to remember that decisions cannot be reduced to science and statistics and that wisdom dictates that I factor in individual experience.