Research evidence versus clinical evidence

A finding is not a fact about people until it has been tested in people, under conditions designed to reveal it as false if it is false. Everything before that point is a reason to keep looking, not a reason to conclude. This article sets out the four evidence levels used throughout this site and explains what each one can and cannot support.

In-vitro evidence

In-vitro work studies cells, tissues or purified biochemical systems outside a living organism. Its strength is control: a single variable can be isolated in a way that is impossible in a whole animal. Its weakness follows directly from that strength. A cultured cell is not connected to a circulatory system, an immune system, a liver that metabolises compounds, or a kidney that clears them.

Concentration is a recurring problem. Effects demonstrated in a dish frequently require concentrations far above anything achievable in a living organism. A paper reporting activity at high micromolar concentrations has demonstrated that the compound can do something to those cells at that concentration, which is a narrower statement than it usually sounds.

Animal evidence

Animal studies restore the whole-organism context: absorption, distribution, metabolism, excretion and systemic responses all operate. They remain limited by species differences that can be substantial and are not always predictable. Rodents differ from humans in metabolic rate, receptor distribution, immune function and lifespan, and a compound handled one way by a mouse may be handled quite differently by a person.

Reporting quality varies widely. Randomisation, blinding and sample size calculations are frequently absent from published animal work, which inflates the apparent size and reliability of effects. Reporting guidelines exist to address this; papers that follow them are easier to assess than papers that do not.

Early human evidence

Early human research includes first-in-human and early-phase studies, small single-arm studies, and observational work. These designs are generally intended to examine tolerability, pharmacokinetics or the existence of a signal. They are usually short, often uncontrolled, and rarely large enough to detect anything but a substantial effect.

Observational studies carry a distinct difficulty: people who choose to do something differ from people who do not, in ways that are frequently related to the outcome being measured. Statistical adjustment reduces this problem but cannot be assumed to remove it.

Established clinical evidence

Established evidence means adequately powered, controlled trials in humans, replicated across more than one group, with outcomes that matter rather than surrogates that are easier to measure. In practice this level is usually reached only for compounds that have gone through regulatory assessment, because the assessment process demands exactly this kind of data.

Reading the gaps

Absence of evidence is reported inconsistently and is easily misread. Three situations look similar in summary but mean different things:

1. Nobody has looked. No study exists. Nothing follows about whether an effect exists or a risk is present.
2. Studies exist and found nothing. This is informative, and its weight depends on whether the studies were large enough to have found an effect had there been one.
3. Studies exist, found something, and have not been replicated. A single positive result is a starting point. Independent replication is what turns it into a finding.

Publication bias makes the third case more common than it should be: positive results are more likely to be written up and accepted than null results, so the published record overstates how often things work.

For this reason, every compound profile on this site states evidence level by level, and says so explicitly where a level is empty. A blank section would be ambiguous. A sentence saying no human evidence was identified is not.