I know this might be a boring title. Who wants a research role that’s all about data cleaning, data merging, and managing very messy data from so many different sources and contributors?
And yet, that’s what I found most interesting about my most recent research publication: Think political leader — think he or she? published in The Leadership Quarterly (one of our top research journals!).
Through the Psychological Science Accelerator (PSA), I served on the leadership team of this massive study with almost 23K participants across 42 countries. My role was coordinating the data coming in from over 150 research labs around the world —each with their own IRB, Qualtrics surveys, and logistics — then merging, applying standardized cleaning practices, and preparing the data for analysis.
I'll cover the findings briefly, but this post is mostly about what it took to get there, and why I keep coming back to projects like this one.
The study: Gender bias in political leadership
We were specifically interested in gender stereotypes in political leadership, sometimes called the “think manager, think male” stereotype. This shows up in everyday language: Chairman rather than Chairperson, or mental recall of images of male leaders. Recent studies have promoted gender-fair language as an easy and small intervention that could have large impact on subtly nudging gender stereotypes in order to reduce bias.
We tested this linguistic nudge through this “big team science” approach in order to test its effects across multiple countries, cultures, and even languages (e.g., comparing gendered languages to non-gendered languages). Participants rated man vs. woman political leaders on various trait adjectives (e.g., warmth, assertiveness), and they were randomly assigned to receive gendered language and gendered pronouns vs. non-gendered. The entire study was a registered report, so our hypotheses and analysis plan were peer-reviewed and accepted prior to data collection.
Overall, we found (a) unsurprisingly, people’s stereotypes of political leaders leaned male, but then (b) the gender-fair wording had no detectable effect on this bias. Given the large sample size and a priori power analysis, even small effects were not detected, and this held across students vs. community samples, paid vs. unpaid participants, and men vs. women. Unfortunately, our takeaway is that these subtle linguistic changes are not very effective as an intervention to reduce gender bias.
Why the “boring” task of data cleaning was so important
It’s hard enough to keep one dataset clean and organized, ready for someone else to replicate one’s analysis and reproduce one’s findings. Trying to do this across 150 labs around the world — many with language and institutional policy barriers — was no small task. And while at times it was certainly mundane (the number of times I had to fix an embedded data tag in an URL to ensure we properly captured lab IDs…), it was critical to ensuring transparency of our research and documenting every step taken to reach our final results.
I got to learn so many things:
Creating an interactive dashboard to automatically pull in data from a hundred different Qualtrics surveys, clean them, and keep everyone updated on their participant counts
Testing multiple iterations of longstring, fraud detection, attention checks, and more — and justifying each choice made
Merging data from different formats, particularly for some labs that couldn’t use Qualtrics for policy reasons, while ensuring uniformity in column names and other metadata
Leading and coordinating researchers, many of them with decades of experience more than me, to align us all on the same goal
One of the first tasks I have research assistants learn how to do in my lab is to clean data. It may seem mundane, but for me, it became my biggest contribution to this research project.
Get involved in big team science
In some ways, I was just one small cog in a large wheel with over 200 coauthors on this paper. That may seem discouraging to some — of course, we all want to lead our own major publication.
But I previously argued that big team science is the path forward for the future of social science research, and I stand by that. Without these initiatives, studies will remain isolated, lacking in generalizability, usually relying only on a small sample of data from a very specific population (see this great article about “WEIRD” research), rarely having impact for the real world.
I encourage students and early career researchers in particular to get involved. If you’re pursuing an academic career, it can’t be the only research you do, but it’s a valuable addition to your overall research portfolio. And if you’re skilled in data, the role I held in project management is a great way in. You'll learn reproducible workflows, documentation habits, and how to communicate with collaborators across cultures and time zones. Those skills transfer to almost any research career. The PSA, the Advancement of Replications Initiative in Management, and the Berkeley Initiative for Transparency in the Social Sciences are all good places to start.
A huge thank you to Kim Peters for leading this project for the past few years, and of course to every lab and translator who made it possible!
Read the paper: https://doi.org/10.1016/j.leaqua.2026.101991
Data, materials, and code: https://osf.io/gjv2f/overview?view_only=df334ebb0c7a4fa4802da9f739e7c047
