In the last piece of this series I argued that scientifically validated is not a label, it is a position on an evidentiary map — and that the reader's job is to locate the claim on that map rather than take the label at face value. This piece picks up where that one ended. Once you know that replication is the step at which a finding starts to look reliable, you need a practical way of checking whether the replication has actually happened, and whether it has happened in conditions that make it informative.

Replication, here, does not just mean the same result has been reported more than once. It means the result has been reproduced by people who had nothing to gain or lose from how it came out. That second condition is the one that does most of the work, and it is also the one that gets quietly skipped in the way many clinical literatures are summarised. A method can have twenty published trials and still not have been independently replicated, in any sense that the word usually carries.

The good news is that the information needed to check this is mostly in the open. Investigator affiliations, correspondence addresses, funding statements, conflict-of-interest disclosures — the parts of a paper most readers skim past — are exactly where the answer lives. With a few minutes on PubMed and a stable habit of looking at metadata before reading abstracts, a reader can develop a fairly reliable sense of where any given clinical claim actually sits.

Two dimensions of independence

Independence in clinical research has two practical dimensions. They are related, but they are not the same.

The first is investigator independence. When the people running the trial are also the people who developed the method, who teach the certification course, or who sell the equipment, the trial is not independent in any meaningful sense — even when it is conducted with full methodological care, even when the authors are individually scrupulous. Inventor-linked evaluation is a recognised category in the methodology literature, and it is not a moral judgement about the investigators. It is a structural observation about who is in a position to detect a negative result, and who is in a position to publish it (Higgins et al., 2024).

The second is funding independence. When the trial is paid for by the entity that profits from a positive result, the structural pressure on the design, the analysis, and the reporting moves in a known direction. The most carefully documented synthesis on this point is the Cochrane methodology review by Lundh and colleagues, which pooled 75 studies comparing industry-sponsored trials to non-industry-sponsored ones across a range of biomedical areas. The conclusion was consistent: industry-sponsored studies report more favourable efficacy results and more favourable conclusions than non-industry-sponsored studies of the same interventions, even after adjusting for methodological quality (Lundh, Lexchin, Mintzes, Schroll, & Bero, 2017). The effect is not large in any single trial, but it is reliable in the aggregate, and it accumulates.

Both dimensions matter independently. A study can be funded by a public agency and still be run by the method's developers. A study can be commercially funded and still be carried out by investigators with no prior relationship to the sponsor. The reader's task is to check each, not to assume that one implies the other.

Where to find this information in a paper

Most of what a reader needs is in the first and last pages of any published article. Three places, specifically.

The affiliations list, immediately after the author names, tells you where each author works. Pay attention to author names that appear repeatedly across the literature on a method, and to the institutions named — particularly when an institution is itself a training centre for the method being evaluated. When the same handful of names appears in study after study, with the corresponding author affiliated with the centre that delivers the certification, you are looking at a closed-network literature.

The corresponding author line, usually toward the end of the front matter, is the person to whom data requests and editorial correspondence are addressed. In inventor-linked evaluation, this is very often the same person across multiple trials. The pattern is visible at a glance.

The funding and conflict-of-interest statements, almost always at the end of the paper, are where commercial relationships are declared. The International Committee of Medical Journal Editors (ICMJE) publishes a uniform format for these disclosures, and most major biomedical journals require it (International Committee of Medical Journal Editors, 2023). The disclosures cover financial relationships in the three years preceding submission — consulting fees, speaker honoraria, equity, patents, paid travel, advisory-board memberships. None of these are disqualifying in themselves. All of them are informative.

Reading these three sections takes less than a minute. The reflex of looking at them before the abstract is the most efficient evidence-literacy habit I know.

A worked example: reboxetine and the value of the unpublished

A clean way to see how independence and replication interact — and how easily the published literature can mislead when both are weak — is the history of reboxetine.

Reboxetine is a selective noradrenaline reuptake inhibitor that was authorised in several European countries in the late 1990s for the treatment of major depression. The trials supporting its approval and its early clinical recommendation appeared in the published literature in the late 1990s and early 2000s. Read at face value, those trials suggested that reboxetine was effective for depression and broadly comparable to other antidepressants.

In 2010, an independent group at the Institute for Quality and Efficiency in Health Care (IQWiG), the German health-technology assessment agency, published a meta-analysis in the BMJ that approached the same question differently (Eyding et al., 2010). Rather than restrict the analysis to the published trials, the authors obtained data on all trials submitted to the German regulator, including those that had never been published. They then compared the published-only meta-analysis to the all-trials meta-analysis.

The contrast was striking. About 74% of the patient data on reboxetine had not been published. When only the published data were analysed, reboxetine appeared modestly effective. When the unpublished data were included, the effect against placebo disappeared, and reboxetine performed worse than several other antidepressants in head-to-head comparisons. The published literature, in other words, was systematically more favourable to the drug than the full literature warranted.

The instructive feature of this case is not that reboxetine turned out to be a disappointing antidepressant. Drugs underperform their early promise often enough; that is not the lesson. The lesson is that the published trials, taken at face value, would have led an attentive reader to the wrong conclusion. The information that overturned the conclusion existed all along, in the regulatory file. It became part of the scientific literature only when an independent team with access to that file did the analysis.

Two practical points follow. First, the absence of replication by independent teams is not always reassuring evidence that there is nothing to find — sometimes it is evidence that the question has simply not been asked outside the closed network. Second, when the full evidentiary picture is reconstructed by an independent group, the conclusion can change in ways the published-only literature would not predict. The skill of evidence reading is partly the skill of remembering this.

Citation chains and closed networks

A related signal, once you start looking for it, is the structure of citation networks within a literature.

Healthy clinical literatures have a recognisable shape: results from one group are cited and re-evaluated by other groups, who add their own data, sometimes confirming and sometimes complicating the initial finding. Citations cross institutional and national lines. Reviews from different perspectives reach overlapping but not identical conclusions. The literature has the texture of an ongoing conversation.

Closed-network literatures look different. The same authors cite each other. New trials cite the foundational work of the network and rarely engage with critical commentary from outside it. Reviews are written by members of the network and conclude that more research is needed — which is true, but does most of its work as a rhetorical move rather than as a research agenda. When you map the citations, the graph stays small.

Spotting this pattern does not require formal bibliometric tools, although those exist. A practical heuristic is to take the five most-cited papers in a given clinical literature, list their authors, and check what proportion of the subsequent supporting literature is authored by overlapping members of that list. If the proportion is high after fifteen or twenty years of work, the field has not, in any meaningful sense, been independently evaluated. That is a finding worth carrying into any reading of claims about the method's evidentiary status.

When the absence of replication is the finding

A natural objection to all of this is that absence of independent replication might simply mean a method is new, or specialised, or technically demanding to reproduce. That is sometimes true and worth weighing fairly. A novel imaging technique that requires equipment available in only a handful of centres will accumulate independent replications slowly; a behavioural intervention that requires extensive practitioner training will diffuse into the broader literature only as that training spreads.

What changes the picture is time. A method that has been in clinical use for two decades and is still evaluated almost exclusively by the network that developed it is not in the early-replication phase. It is in a different phase, one the methodological vocabulary does not have a polite name for, and one that the phrase more research is needed tends to paper over.

I want to be careful here. The point is not that closed-network literatures are necessarily wrong, or that their findings should be dismissed. Plenty of effective interventions began life inside the work of a single laboratory and only later attracted broader independent attention. The point is that the evidentiary status of such literatures is not the same as the evidentiary status of literatures that have been examined from many angles by groups with no stake in the result. Treating those two situations as if they produced equivalent claims is the kind of small slippage that does most of the work of claim inflation in clinical marketing.

A reader's PubMed walkthrough

To make this concrete, here is the sequence I run when I encounter a clinical claim for a method I want to evaluate.

1. Search PubMed for the method's name combined with the condition of interest. Note the number of results and the date range. If the literature is older than fifteen years but smaller than fifty results, the field is either highly specialised or under-investigated. Either way, the finding shapes how you read what follows.

2. Filter for systematic reviews and meta-analyses. Read those first. Note who wrote them: are the authors of the syntheses themselves members of the network that ran the primary trials, or are they independent?

3. Examine the primary trials. For each, check the affiliations of the first and corresponding authors. Track whether the same names recur across trials. If they do, the literature is concentrated, and that concentration matters for interpretation.

4. Read the funding statements. Note any commercial sponsorship, training-centre support, or equipment-provider relationships. Read the conflict-of-interest disclosures carefully — they are often more informative than the funding line.

5. Look for evidence of independent replication. A trial run by a team with no prior publications on the method, no commercial relationship to its developers, and no affiliation with a training centre that delivers the certification, is the kind of evidence the field most needs and most often lacks.

6. If you cannot find such a trial after several searches, consider that finding as part of your evaluation. Replication has not been attempted independently is information.

None of this requires special access to databases or paid tools. PubMed, an open browser, and the willingness to spend ten minutes on metadata before reading the abstract are enough for most purposes.

Conclusion

Independent replication is one of those concepts that sounds straightforward until you try to check it in a specific case. Then it becomes clear how many of the everyday markers of a confident clinical literature — number of publications, prominence of authors, conviction of conclusions — can coexist with a near-absence of the structural independence that gives replication its weight.

The PubMed habits sketched above are not a guarantee against error. They will not let you distinguish a strong method from a weak one in fifteen minutes. They will, more modestly, let you distinguish a literature that has been examined by people with something to lose from a literature that has not. That distinction is small in any one paper. It is decisive across a field.

Replication, like validation, is patient work. It happens slowly, in places far from the original laboratory, on patients who were never told about the method by the person who developed it, by investigators who would as cheerfully publish a negative result as a positive one. When that work has happened, the literature looks unmistakeable. When it has not, the literature has a particular texture — same names, same centres, same conclusions, same call for more research — and learning to recognise that texture is most of what evidence literacy actually consists of.

The most useful question to keep close, when a clinical claim is being made, is the one that follows naturally from this piece and the last:

Who has tried, and failed, to make this method fail?

References

• Eyding, D., Lelgemann, M., Grouven, U., Härter, M., Kromp, M., Kaiser, T., Kerekes, M. F., Gerken, M., & Wieseler, B. (2010). Reboxetine for acute treatment of major depression: systematic review and meta-analysis of published and unpublished placebo and selective serotonin reuptake inhibitor controlled trials. BMJ, 341, c4737. https://doi.org/10.1136/bmj.c4737

• Higgins, J. P. T., Thomas, J., Chandler, J., Cumpston, M., Li, T., Page, M. J., & Welch, V. A. (Eds.). (2024). Cochrane Handbook for Systematic Reviews of Interventions (version 6.5). Cochrane. https://training.cochrane.org/handbook

• International Committee of Medical Journal Editors. (2023). Recommendations for the conduct, reporting, editing, and publication of scholarly work in medical journals. https://www.icmje.org/recommendations/

• Lundh, A., Lexchin, J., Mintzes, B., Schroll, J. B., & Bero, L. (2017). Industry sponsorship and research outcome. Cochrane Database of Systematic Reviews, 2(2), MR000033. https://doi.org/10.1002/14651858.MR000033.pub3