Tips for Interpreting Scientific Studies

When researching and evaluating a scientific study, various factors should be considered. While the results of a study may sound promising, they are only relevant if the study was conducted with high-quality standards, and the results were accurately communicated. Therefore, reading and evaluating studies with a critical eye is important. Tips to help readers decipher the legitimacy of scientific studies include the following:

Determine who performed and paid for the study.

If the financial backers of a study have a vested interest in the outcome, the study may be compromised. For example, if a pharmaceutical company pays for a study about the safety and effectiveness of a medication that they manufacture, it is in their best interest for the study to have positive results. This conflict of interest may influence the way the study is conducted and the accuracy of the reported results.

Independent studies are conducted by independent researchers who do not have anything to gain or lose from the results of the study; these studies are generally more trustworthy than studies funded by companies that have a financial stake in the results.

Consider the participants in the study.

Here are some basic questions to answer about study participants:

• Was the study conducted with animals or humans? While many tests are conducted on animals, the results may not be applicable to humans.
• If the study was conducted using human participants, how diverse was the population? A study that includes participants of different ages, genders and ethnicities is more applicable to the general population. A study that involves similar participants is more applicable to that subgroup of the population.
• What medical condition did the participants in the study have? For studies involving medical treatments, the condition for which the participants were being treated should be noted. It is also important to note if any individuals were excluded from the study and if participants dropped out of the study, especially if they had the condition being studied.
• How many participants were involved? If too few participants are included in a study, the results may be incorrect or insignificant. Many researchers work with a statistician to determine the appropriate study size. For clinical trials, some experts recommend 20-80 participants for phase I trials and 100-200 participants for phase II trials. Some studies are case studies, which often focus on one participant; this is especially true for rare conditions where a larger study size is not possible. Case studies often provide anecdotal evidence, which is based on an individual’s experience.

Identify the way the study was conducted.

Most often, a study is either an observational study or an experimental study. An observational study can determine an association between two factors, such as medication use and improved symptoms, but association only shows correlation, not causation. Correlation shows a relationship, but cannot show that one factor causes another (causation) because other uncontrolled factors may affect the association.

In an experimental study, other factors are controlled so that causation can be shown. The gold standard for an experimental study is a placebo-controlled, double-blind randomized clinical trial. This type of study is often used to determine the effectiveness of a new medication. The medication is tested against a placebo (an artificial form of inactive, harmless “medication” that looks exactly like the real thing).

Study participants are randomly assigned to take either the placebo or the authentic medication; neither the participants nor their doctors know whether they were given the placebo or the authentic medication until the study is over (the blind part). This is done to compare the effect of the medication with the effect of the placebo. This type of study also takes into account the placebo effect, which occurs when conditions improve after taking a placebo.

When reading a scientific study, noting whether the study was an observational or experimental, whether the results show correlation or causation, and whether the trial was a double-blind randomized study helps individuals determine the validity of the study.

Consider the results and the way the results are communicated.

Questions to consider regarding study results and the communication of those results include the following:

• If the study involved a new treatment or medication, were there any side effects? Side effects should be clearly communicated and taken into consideration.
• Are the findings statistically significant? The complex statistics of research studies are often difficult to understand; however, the term “statistically significant” implies an accepted level of confidence in the results.

Applying these tips while evaluating a study helps to understand the motivation behind the study, the significance of the results of the study, and how the study applies to one’s own situation.