Systematic Reviews

Modern medical research generates so much literature, and fills so many journals, that a traditional literature review could take months, and still be out of date by the time that the research is designed and performed.

In addition, researchers are often guilty of selecting the research best fitting their pre-conceived notions, a weakness of the traditional 'narrative' literature review process.

To help medical professionals, specialist compilers assess and condense the research, entering it into easily accessible research databases. They are an integral part of the research process, and every student of medicine routinely receives a long and extensive training in the best methods for critically evaluating literature.

Systematic Reviews - Addressing the Deficiencies in Narration

The problems with narrative literature came to light a couple of decades ago, when critics realized that reviewers looking at the same body of evidence often generated completely different findings. They drew conclusions based upon their specialty, rather than the compelling evidence contained within the body of research.

It is unclear whether this was a case of conscious or subconscious manipulation (bias), but this particular finding was worrying, especially in a research area where life and death could be at stake. To address this issue, medical authorities developed a new protocol of systematic reviewing, based upon a structure as strict as the scientific method governing empirical research programs.

The Protocols Underpinning Systematic Reviews

• Define a research question, in a similar way to formulating a research question for a standard research design
• Locate and select relevant previous research studies, with no attempt at evaluation at this stage. Ideally, research in languages other than English should be used, and the researcher should try to find papers and reports unpublished in journals, such as conference speeches or company reports.
• Critically evaluate the studies. The reviewer should assess each study upon criteria based upon quality, strength of the findings and validity. For safety, this process should include at least two independent reviewers, although a greater number is advisable.
• Combine the results. This is the process of combining all of the findings, sometimes qualitatively, but usually quantitatively, using meta-analysis.
• Publish the results. As with any research, the results have to be written and published, usually with a system of independent review. Discussion of the conclusions, as with any research, allows the validity of the findings to be verified.

The Reasoning Behind Systematic Reviews

The principle behind the systematic reviews process is that the researcher critically evaluates previous studies, in a much more comprehensive and systematic way than a standard literature review.

In many cases, statistical meta-analysis tools are used to give the review a quantitative foundation, allowing correlations to be documented and conclusions to be drawn. Whilst the techniques are mainly used by medicine and psychology, there is a growing trend towards using systematic reviews in other disciplines. Many branches of science are becoming increasingly fragmented and anarchic, so this layer of analysis aggregates all of the disparate elements.

Systematic reviews, and meta-analysis, are regarded as a cornerstone of healthcare research, essential where it is impractical or unethical to keep repeating old research.

In addition to the potential risks of repeated research upon patients and volunteers, there are now laws in many countries prohibiting excessive research using animals. Systematic reviews are a great way of reducing the amount of suffering caused by vivisection.

Addressing the Disadvantages of Systematic Reviews

As with most systems, despite the protocols, systematic reviews do have some inherent weaknesses.

The main problem is the rapid advancement of medical research and technology, often meaning that many reviews are out of date before they are even published, forcing researchers to update their findings constantly. The development of specialist organizations for finding and evaluating data minimizes the effects of this particular shortcoming.

As with any subjective review, there is the problem of selection bias, where contradictory research is jettisoned, although most medical researchers are adept at following the proper procedures.

Funding and research grants cause researchers to try to find results that suit their paymasters, a growing problem in many areas of science, not just medicine. The specialist reviewers sidestep this problem, to a certain extent, by producing independent research, uncorrupted by governmental or private healthcare funding, curbing the worst excesses.

Often, a blind system is used, and reviewers are unaware of where the papers they are reviewing came from, or who they are written by. This lessens allegations of favoritism and judging research by the reputation of the researcher rather than on merit.

Ultimately, the onus is on the reader to draw their own assessments, using their own experience to judge the quality of the systematic review. Whilst not a perfect system, systematic reviews are far superior to the traditional narrative approach, which often allows a lot of good research to fall through the cracks.