Chapter 4.2: Confounding, Chance, and Bias (Video Transcript)

Confounding, Chance, and Bias

Title: Confounding, Chance, and Bias

Presenter(s): Cochrane Austria (Department for Evidence-based Medicine and Evaluation, Danube University Krems)

The goal of clinical studies is to investigate the efficacy and safety of treatments. They leave room for three important sources of error that could lead to false study results: confounding, chance, and bias. Confounders confuse us in interpreting study results and may lead us to infer incorrect conclusions about cause and effect relationships. For example, in one study, we observed that those who drink a lot of coffee are at an increased risk for coronary heart disease. Is coffee dangerous for heart health, or could another factor be responsible for this relationship? In our example, smoking is responsible for the observed relationship. Smoking is a risk factor for coronary heart disease, and people who smoke tend to be people who drink a lot of coffee. So, taking the confounder, smoking, not into account, one would draw the wrong conclusion that drinking coffee is a risk factor for heart disease. We are dealing with the confounder.

Properties of confounders (= confounding factors)

If the following properties are present, the confounder must be related to the outcome regardless of the exposure. In our coffee example, this means that smoking is an independent risk factor for coronary heart disease, regardless of whether someone drinks coffee or not. The confounder must be related to the exposure. In our example, this means that people who smoke often often drink coffee. The confounder should not be on the causal pathway between the exposure and outcome. This condition would also be fulfilled in our example: drinking coffee does not automatically lead to smoking. How do you deal with confounders?

How do you deal with confounders?

You can get a grip on confounders that you know in advance in the planning phase. For example, the inclusion criteria of a study are defined so narrowly that the influence of confounders is eliminated. This procedure is called restriction. One can also consider confounding in the analysis. For example, with stratification, in our coffee study, we would stratify all individuals into groups; that is, divide them into smokers and non-smokers and examine in both groups if there is a connection between coffee drinking and heart disease. If only people who smoke are at an increased risk for heart disease, then we would see that drinking coffee is not the cause of heart disease. We are mostly dealing with several confounders here. Multivariate analyses can be calculated, which adjust for several confounders. But what do you do with confounders that are unknown to us?

What do you do when a confounder is unknown?

The only way to deal with unknown confounders is randomization. The study participants are randomly divided into study arms. The randomization leads to an equal distribution of known and unknown confounders in the study groups. If one observes a difference between the groups after an intervention, we can assume that this was caused by the intervention and not because influencing factors were already distributed differently between the groups at the beginning. Randomization is the only effective remedy against unknown confounders.

Chance

Another source of error in studies is chance, also called random error. The result of a study may coincidentally differ from the true effect in the population just by chance. This random deviation has no definite direction. If you were to do many of the same studies, some would overestimate the effect, others underestimate it, and others would estimate it correctly. One can minimize the influence of chance by having a large sample size.

Imagine you have a bowl of 500 green and 500 red gummy bears. You draw two samples, one with ten gummy bears and one with a hundred. In the small sample, it can be easy for you to draw eight green and two red gummy bears. With 100 gummy bears, you are probably already much closer to the 50-50 distribution. It’s similar with studies. In very small studies, it may happen, despite randomization, that the study groups are not similar. A non-evidence-based rule of thumb states that studies with fewer than 300 study participants are susceptible to random variability. Additionally, a low event rate is prone to random error. A large study size can minimize random errors.

Bias is another source of error in studies. Bias is a systematic deviation from the true effect that can be caused through the design, conduction, or analysis of a study. In contrast to random error, bias would always distort the results in the same direction if the study were performed multiple times. There is a variety of bias subcategories, and here are four types of bias that play an important role in interventional studies: selection bias, performance bias, measurement bias, and attrition bias.

Selection bias

Selection bias describes the problem of systematic differences in the allocation of study participants. For example, imagine a study that examines the effects of a physical exercise program versus no intervention, and study participants are allowed to choose which group they want to be in. It is very likely that more sports enthusiasts and health-conscious individuals would opt for the sports group. As a result, two groups would be formed that are not comparable from the start in some aspects. If, at the end of the study, it emerges that the sports group had better results, this cannot be automatically attributed to the intervention itself, as the group already had better starting conditions. Selection bias can be avoided by employing randomization and ensuring secrecy of the randomization sequence, also known as allocation concealment.

Performance bias

Performance bias occurs when, apart from the intervention under investigation, there are systematic differences in the treatment and care of patients. For example, the type of care and attention changes with the intravenous administration of a medication. So, if you compare two drugs that are administered differently, it remains unclear whether a different effect is due only to the drug or to the other care and attention given. In order to prevent performance bias, it helps to have a standardized treatment concept for all study groups and to blind all people involved in a study. Blinding means that the participants in a study are not aware of which people receive which treatment.

Measurement bias

Measurement bias exists when there are systematic differences in measuring outcomes. For example, if a person evaluates their knee pain, knowing if they were in the intervention or comparison group can influence the perception of the pain. To prevent measurement bias, those who measure the outcomes should not know which individuals receive the intervention and which receive the control intervention. This allows them to assess the outcomes unaffected and objectively.

Attrition bias

Premature exit from the study results in systematic differences between study groups. In general, it is normal for some people to leave studies early. However, it becomes problematic if this happens systematically and not accidentally. Suppose those who feel particularly unpleasant or have many side effects leave the study. If, at the end of the study, we only look at those in the analysis that remained in the study until the very end, we would ironically believe that there were very few side effects. One can minimize the influence of attrition bias in the data evaluation by providing an intention-to-treat analysis that includes all individuals originally randomized to a study.

Risk of bias

When we critically evaluate studies, we try to estimate the risk of bias. Bias cannot be measured directly. The risk of bias can only be assessed indirectly through evaluation of the study design and the execution of studies. In addition, the risk of bias between outcomes may vary. For example, while subjective outcomes such as pain can be strongly influenced by lack of blinding, this has no effect on hard outcomes such as mortality.

Components of a study result

The aim of studies is to map the true effect of an intervention as well as possible. That is, the distorting influence of confounding, chance, and bias should be contained as much as possible. A large sample can minimize the influence of random error. The study design, the good execution of the study, and the analysis can minimize the influence of confounding and bias.