Beware the "super well" - why the controls in psychology research are often too healthy

Many studies in clinical psychology and psychiatry are making the mistake of using healthy controls who are too healthy. That's according to a thought-provoking opinion piece by Sharon Schwartz and Ezra Susser - experts in the epidemiology of mental health.

Schwartz and Susser invite readers to consider a hypothetical study that samples participants from a wider group made up of people exposed to a virus prenatally and people not exposed to that virus. Imagine that a psychiatric registry is used to identify all the participants from this wider group who are diagnosed with schizophrenia, and they are compared with a slice of healthy participants recruited from the same source. The aim is to see what proportion of the participants with schizophrenia were exposed to the virus and what proportion of the healthy controls were exposed to the virus. If the history of exposure is higher among the schizophrenia participants, then this would suggest there may be an association between the virus and the later development of schizophrenia. In Schwartz and Susser's hypothetical scenario, there is no difference between patients and controls in rates of virus exposure and so the virus seems unassociated with schizophrenia. So far, so good - this is a classic case-controlled study.

The problem identified by Schwartz and Susser is that many such studies apply an exclusion criterion or criteria to the healthy controls that they don't also apply to the patient group. For example, they might rule out healthy controls with an alcohol problem, or depression, or even a physical disorder. The motivation for this is often the fear that these other disorders will obscure the potential link between the cause of interest and the condition of interest (virus exposure and schizophrenia in our ongoing example).

But to apply such exclusion criteria in a one-sided fashion (to the controls but not the patients), creates a serious confound. In our example, imagine that depressed "healthy" controls are excluded and imagine too that there is an underlying association between the virus exposure and depression. Excluding healthy controls with depression in this scenario would distort the results such that the virus appeared wrongly to be associated with schizophrenia (check out the full paper for the data behind this).

"With all the potential sources of bias in a biologic case-control study, why do we focus on the use of well controls?" the researchers asked. "We do so because the use of well controls is a common, and often recommended, method to select controls. Yet it is time-consuming and expensive, can cause considerable bias and does not improve study results."

If researchers include patient participants with other co-morbid diagnoses in their case-controlled studies, Schwartz and Susser went on to explain, then they must also include "healthy" controls who happen to have these other conditions. On the other hand, if researchers want to exclude other conditions, so as to clean up their investigation, then they must exclude both patient participants and controls with these other diagnoses.
_________________________________

Schwartz, S., and Susser, E. (2011). The use of well controls: an unhealthy practice in psychiatric research. Psychological Medicine, 41 (06), 1127-1131 DOI: 10.1017/S0033291710001595

This post was written by Christian Jarrett for the BPS Research Digest.