What’s Your Null Hypothesis?

Around this time last year I journaled about the family trip to the Pacific Northwest and the wonderfulness of it all.  By my own estimates, I was the only one who could one-up me.  I tried this holiday season and while my probability was low, my effect size was quite large!  My tested experiment that skiing in negative degree temperatures with 30 mph winds gusting could prove as blissful as eating chocolate popcorn in a one-of-a-kind, state-of-the-art, movie theater, proved false.  That driving across only three states with dog in tow would be easy peasy might prove to be the case if you’re talking about the 2.5 hours it takes to go from New Haven, CT, to Boston, MA, via Providence, RI, not when your starting place is Texas and you head west.  And seeing snow for the first time this season quickly loses its charm when you’re seeing it parked on a shutdown interstate because you got caught in the winter storm your wife told you was coming three days before you set out on your journey.  Trust me, the hour you get back from Mountain Time doesn’t make it any better when you arrive before 7 pm because your day started 15 hours before!

I was reading this morning in one of my subscription journals and it shaped this entry while reminding me of a recent experience.  The American Educational Research Association (AERA) produces several publications and one that dates back to 1972 is Educational Researcher (ER).  Its latest edition, a special issue, is entitled “Randomized Controlled Trials Meet the Real World:  The Nature and Consequences of Null Findings.”  Who can resist a cover to cover read after that gem of a title but as a bonus carrot I’ll share some of what you’ll find when you click this link:

“Because we know the probability values and the effect sizes, we can estimate standard errors of the effect sizes…the standard error of the difference between two estimates is the square root of the sum of their squared standard errors.”

So I’ll admit the editors of ER probably aren’t too invested in clickbait or cliffhanger headlines and to dedicate an entire volume on research that well, proved nothing substantive seems a little much.  For context, and more to the point I need to get to quickly :-), time and time again educational interventions are evaluated in an ocean of existing influencers and making meaning of any one “thing” often is unlikely.  Thus, the estimated impact seems little to none; the experiments produce null findings.

At church this past Sunday while everyone was standing during the opening hymn a young man walked in and immediately drew attention to himself.  He couldn’t have been more than 20 years old, had a disheveled appearance, and then started hugging each person he approached.  When he got to me I said something along the lines of “I have no idea who you are but I guess bring it in” as I became new hug friend #5 or so along our back row.  My wife was next as were a couple more while the ushers looked on confused, so much so that my wife took the young man and planted him next to her in an open seat.  She partnered with him so he could see in the hymnal as we finished “Lord of the Dance” (I’m not making this up!) and then guided him through the reading of the affirmation of faith.  Then, as we prepared for offering, we got the 411 from the 5-0.  As our visitor was escorted out by the police officer, it reminded me as we had just recited who shall come to judge the quick.  In a badly designed experiment, my null hypothesis that day was that people who come to church dress the same.  I had to reject that hypothesis and was left with more questions than answers.  I was left with null findings that taught me once again when it comes to people, we may never know the full story but we should always be willing to have our theories challenged.  Some of the best findings can be those we have yet to consider…