Superstition, Statistics, and Science

So a funny thing happened to me Thursday morning.  Some who rides the Caltrain shuttle with me in Palo Alto asked me if I was "Superstitious? Or just a fan?".   I was wearing my Giants cap, black leather jacket, black cords and orange shirt.   I almost replied "just a fan" but admitted, "Superstitious.".  She replied that that she was superstitious but not about baseball, and I admitted that I was *only* superstitious about baseball.

Friday morning, the day after the game 5 loss.  I felt bad; almost hungover.  I thought of her thinking how awful it must feel to be like this all the time.  Doing crazy stuff (and there have been dozens if not hundreds on comments on it here -- including "stay in the hospital", "don't ever go to another game", etc. etc.) because you feel desperate to have SOME influence on an event that carries so much meaning for you.

And why is that?  I'm a pretty rational dude.  But I have always been superstitious about baseball; even when I spend half the time talking to myself about how silly it is.  But I have worn my Giants hat every playoff game day -- and not worn it at all during the year except for a couple of ball games I actually went to.

Lets go back a step.  Why are people superstitious at all?  It seems pretty self-evident (provided a cursory knowledge of evolution and human psychology) that superstitious behavior is at least related to reason and inference.  Something happens to us -- good or bad -- and our brains try to force and understanding or predictability on future like events.  Now in the case where there is a readily apparent pattern - "if it rains, I will get wet".  "If there are dark clouds, it often rains".  "If I see dark clouds, and want to avoid getting wet, I should seek cover in case it rains".    Where it really becomes classical "superstition" is when the reinforcement is intermittent.  I am not the first person to point this out - in fact, Wikipedia entry on superstition summarizes none other than BF Skinner.

Originally, in Skinner's animal research, "some pigeons responded up to 10,000 times without reinforcement when they had originally been conditioned on an intermittent reinforcement basis."[9] Compared to the other reinforcement schedules (e.g. fixed ratio, fixed interval), these behaviours were also the most resistant to extinction.[9] This is called the partial reinforcement effect, and this has been used to explain superstitious behaviour in humans. To be more precise, this effect means that, whenever an individual performs an action expecting a reinforcement, and none seems forthcoming, it actually creates a sense of persistence within the individual.[10] This strongly parallels superstitious behaviour in humans because the individual feels that, by continuing this action, reinforcement will happen; or that reinforcement has come at certain times in the past as a result of this action, although not all the time, but this may be one of those times. From a simpler perspective, natural selection will tend to reinforce a tendency to generate weak associations. If there is a strong survival advantage to making correct associations, then this will outweigh the negatives of making many incorrect, "superstitious" associations.[11]

Skinner might have been wrong about the pigeons, but I think his theory of human superstition is essentially correct.  Now - what is the difference between superstition and religion?  Well, essentially the model of reality!  In religion - some supernatural forces and/or gods control the outcomes.  Our actions can influence the gods in their control over events.   Note that this is actually no difference between religion and science here!  They are both different models of reality!   So - what is the difference between superstition and statistics?  The only real difference is that in statistics, you keep better notes!   In fact, for 90% of statistics as applied to the real world - be it baseball or genetics or meteorology the underlying model is not relevant.  It is a technique of pure outcome prediction; quantitative superstition, if you will.  (I feel compelled to point out for those who don't know; I am a big fan of statistics in baseball.)

So that brings us back to baseball.  All those playoff stats?  Superstitions.  Pitcher/Batter splits?  Superstition.  11-0 at home?  Superstition.  Just because you count something doesn't make your (non) model predictive.  And that is why baseball is so superstitious.  It is the absolute KING of intermittent reinforcement.   You have guys throwing a 3.5" irregular sphere 60' at 90 mph trying to carve a special arc through an invisible box while another guy takes a rounded oddly shaped club and tries to hit it where 7 other guys ain't.   There are literally hundreds of sources of human error / extreme performance on every pitch; and about 330 pitches per game.  It's amazing that it's even recognizable on a day-to-day basis as the same activity.   We might as well just throw a big bag of quarters on the field and decide who gets to go to the World Series (Giants get 2 bags).

So why does it hurt so damn much when they lose?  There's an evolutionary biology answer for that one too... but I think I have rambled on enough.  But I will leave you with this.  We fans don't always bet on our teams with our wallets.  But we bet with our hearts, minds, and spirits. 

Rollin' the dice

Just

One

More

Time.