Meta-Analysis

Social scientists can have difficulty designing and implementing true experiments, so meta-analysis provides them a quantitative tool to statistically analyze data drawn from a number of past studies. The fields of medicine and psychology increasingly use this method as a way of avoiding time-consuming and intricate studies.

What is Meta-Analysis?

Studies in the social sciences often use small sample sizes, so any statistics used generally give results containing large margins of error. Small sample size can be a problem when interpreting and drawing conclusions, because it can mask any underlying trends or correlations. Conclusions from small studies are tenuous at best, and leave the research open for criticism.

Meta-analysis, on the other hand, is the process of drawing from a larger body of research, and using powerful statistical analysis to come to conclusions. It can be thought of as a “study of studies.” This gives researchers a much larger sample population and is more likely to generate meaningful and usable outcomes.

The Advantages of Meta-Analysis

Meta-analysis is an excellent way of simplifying the complexity of research. A single research team can reasonably only output so much data in a given time. But meta-analysis gives access to possibly more data than that team could produce in a lifetime, and allows them to condense it in useful ways. As we make technological developments in computational power, new database programs have made the process even easier.

For rare medical conditions, meta-analysis allows researchers to collect data from further afield than would be possible for one research group. This allows them to conduct meaningful statistical analyses when a small local sample would have told them nothing about the disease.

When professionals working in parallel can upload their results and access all known data on a topic, there is a built-in quality control. The effects of error or bias in studies are kept in check. Meta-analysis also ensures there is no unnecessary repeat research and allows researchers to pool resources and compare methods. As papers can often take many months to be physically published, instant computer records ensure that other researchers are always aware of the latest work and results in the field.

A meta-study allows a much wider net to be cast than a traditional literature review, and is excellent for highlighting correlations and links between studies that may not be readily apparent as well as ensuring that the compiler does not subconsciously infer correlations that do not exist. Perhaps best of all, meta-studies are economical and allow research funds to be diverted elsewhere.

The Disadvantages of Meta-Analysis

There are nevertheless disadvantages to meta-analysis, of which a researcher must be aware before relying on the data and statistics it generates. The main problem is the potential for publication bias and skewed data.

Research generating results that don’t reject null hypotheses may tend to remain unpublished, or risk not being entered into a database. If the meta-study is restricted to research with positive results, then the validity of the entire endeavor is compromised.

The researcher compiling the data must also make sure that all research is quantitative, rather than qualitative, and that the data is comparable across the various research programs, allowing a genuine statistical analysis. It’s important to pre-select the studies carefully, ensuring that all the research used is appropriate and of sufficient quality to be used. Just one erroneous or poorly conducted study can place the results of the entire meta-analysis at risk.

On the other hand, setting almost unattainable standards for inclusion can leave the meta-study with too small a sample size to be statistically relevant. Striking a balance can be a little tricky, however the field is in a state of constant development, incorporating protocols similar to the scientific method used for normal quantitative research.

Current meta-analysts are skillfully developing library-based techniques that use data science to extend the research powers of human scientists. They make it possible to find information buried in government reports or forgotten conference data, ancient or rare materials, or extremely large data sets. As this field grows, meta-analysts are developing the knack of assessing the quality of sources quickly and effectively.

Conclusions and the Future

Meta-analysis is here to stay as an invaluable tool for research, and is rapidly gaining momentum as a stand-alone discipline, with practitioners straddling the divide between statisticians and librarians.

Provided that the disadvantages are taken into account, the benefits of meta-analysis are too obvious to ignore. A correctly conducted meta study can reduce the need for long, expensive and potentially intrusive repeated research studies.