Ten Years of Alumni Data: The Case for Treating Mentorship Like an Experiment

Science celebrates rigor in almost every domain except the one that produces scientists. We measure effect sizes, we preregister hypotheses, we replicate until a finding holds — yet the practice of mentoring, the mechanism by which the entire enterprise reproduces itself, is largely governed by anecdote and tradition. At McKnight Lab, we decided a decade ago to treat that inconsistency as a problem worth studying, and we have been quietly collecting the data ever since.

This piece is an opinion, but an opinion with a spreadsheet behind it.

What We Tracked

Beginning in 2015, we recorded structured information on every trainee: undergraduates, rotation students, graduate students, and postdocs. Career outcomes were logged through annual follow-ups — positions, sectors, publications, and self-reported satisfaction. Just as importantly, we logged the mentoring practices they experienced: frequency of one-on-one meetings, involvement in project design, authorship discussions, conference travel, and exposure to career paths beyond academic research.

The sample is modest — fewer than a hundred alumni — and we offer it with appropriate caveats. But patterns emerged that we did not expect.

What Predicted Success (and What Didn't)

The strongest correlate of long-term career satisfaction and advancement was not publication count. It was project ownership: trainees who designed at least one experiment themselves, from question to interpretation, reported markedly higher confidence and were more likely to describe their training positively a decade later.

The second strongest predictor surprised us: explicit career conversations. Trainees who had at least two structured discussions about careers inside and outside academia — including the 40 percent who ultimately left the bench for industry, policy, or teaching — reported the strongest mentorship relationships, regardless of where they landed. Perhaps counterintuitively, discussing departure did not accelerate it.

What did not correlate strongly: daily supervision. High-frequency oversight predicted short-term productivity but not long-term outcomes, and in some cases correlated with lower self-reported independence.

The Incentive Problem

Here is the uncomfortable part. Nothing in our funding, our evaluation, or our publication record rewards any of this. A grant renewal asks about aims accomplished, not people launched. A faculty promotion dossier counts papers, not mentees who thrived. The publish-or-perish economy pushes mentorship into the margins precisely because it is the lab's most valuable and least measurable output.

We believe this is a solvable design problem. Institutions could weight mentoring statements with the same seriousness as research statements. Funders could require trainee outcome tracking the way they already require data management plans. Journals could treat mentoring practices as reportable methods. None of this requires new science — only new accounting.

A Modest Proposal

We are not arguing that mentorship should be graded like a Western blot. Human development resists the tidy quantification we impose on molecules. But the absence of perfect measurement has too often served as an excuse for no measurement at all.

Our ten-year ledger is imperfect, small, and undoubtedly biased in ways we have not fully anticipated. It is also the only systematic record of its kind we know of at a lab of our size, and it has already changed how we operate: fewer meetings, more ownership, earlier and more honest career talk.

Mentorship is not a soft skill appended to the real work of science. It is a method — arguably the method by which all other methods survive. It deserves the same discipline we bring to everything else on the bench, and trainees deserve mentors willing to be held to that standard.

We welcome colleagues who keep similar records to compare notes. The N is never big enough until everyone contributes.