Start by assuming you’re wrong

Every semester in introductory statistics, I teach my students null hypothesis significance testing (NHST). It’s one of the most misunderstood topics in all of statistics — and for good reason; there’s a lot of issues with it!

But one real benefit of learning NHST is, in my mind, philosophical. It all starts from an assumption that you’re wrong. You begin with the null hypothesis, a statement of no effect, no difference, and just random noise rather than anything meaningful. Many people forget this, since we often use NHST language as if we are “supporting” our hypothesis, when instead the root logic of the method is challenging the starting assumption that we are wrong.

In other words, NHST is a formalized act of intellectual humility. And philosophically, I believe this to be a core requirement of being a faculty member in higher education. That’s what this op-ed is about (surprise!) — not statistics, but rather what it means to be a faculty member.

We’ve forgotten our foundations

Somewhere along the way, I think researchers stopped treating the null hypothesis as a genuine possibility and started treating it as an obstacle. The goal became to prove our hypothesis, rather than to genuinely consider that we might be wrong. The result is a culture that rewards p < 0.05 and incentivizes HARK’ing. That’s not intellectual humility, it’s the opposite.

The irony is that we baked genuine epistemic humility into the very structure of our scientific method through the NHST, then we built a culture that systematically ignores it and instead rewards “being right” at all costs.

Maybe I’ve just had a few less-than-positive experiences, but I think this goes far beyond conducting a statistical test. Too often, I see or hear of vitriolic critiques, adamant defensiveness, and a tone of “how dare you question me” or “how could you possibly think this way” — treating intellectual challenges as personal attacks. Alvesson (2020), in his critique of leadership research, calls these “subtribe orientations” where scholars safeguard their position "like designing a fortress against possible invaders" to protect from reviewers' criticisms and dismiss trivial ones.

It’s become embarrassing to admit that you’re wrong, when in fact, that’s the starting position that we all as scholars should be taking.

What “start by assuming you’re wrong” actually feels like

Intellectual humility has a long history in psychological research. It’s comprised of both an internal dimension (how we think and our attitudes) and an external dimension (how we behave), directed at both ourselves and at others. In other words, it goes beyond recognizing your own intellectual fallibility, it also incorporates behaving as if you could be wrong and recognizing that others might be right.

This means that it goes beyond a static personality trait, it’s a behavioral practice that one can embody both in their thinking and acting. And my argument here is that we, as scholars, ought to strive for this.

That’s uncomfortable. It’s much more comfortable to walk into a conversation, a dataset, or a peer review assuming that you have something to teach rather than something to learn. But the people I respect the most are the ones who seem genuinely open to having their minds changed.

To be clear, I don't think intellectual humility means being wishy-washy or refusing to take positions. You can have strong, well-reasoned views and still hold them loosely enough to revise them when the evidence warrants it. That's the goal I hope we all aspire to.

But it really does require learning from the basics, like the NHST. Start with the skeptical default, let the evidence speak, then update accordingly.

Where we can go from here

First, let me model this: I might be wrong in this op-ed. Perhaps my experiences are the outliers, and most scholars are truly intellectually humble — so we already are “good to go” on this front. Or perhaps intellectual humility shouldn’t be the goal, and there’s a good reason for why we ought to stand stronger in our arguments in the face of threats to scientific research.

If you disagree, I’d love to hear why.

But if you agree, let me ask everyone: What would it look like for you to genuinely apply null hypothesis logic to your own most confident beliefs? What would it look like to engage in both teaching and research (and service/leadership!) from this position of always being open to being wrong? I look at the Adversarial Collaboration Project as a prime example of a tangible way for scholars to model intellectual humility in research, by collaborating on a study with people they vehemently disagree with.

Philosopher of science Imre Lakatos put it bluntly: intellectual honesty isn't about trying to prove your position, it's about specifying precisely the conditions under which you'd be willing to give it up.

Every semester, I ask my students to start from the assumption that they're wrong. It’s time we all started asking that of ourselves.