Social scientists have great difficulty in designing and implementing true experiments, so meta-analysis gives them a quantitative tool to analyze statistically data drawn from a number of studies, performed over a period of time.
Medicine and psychology increasingly use this method, as a way of avoiding time-consuming and intricate studies, largely repeating the work of previous research.
What is Meta-Analysis?
Social studies often use very small sample sizes, so any statistics used generally give results containing large margins of error.
This can be a major problem when interpreting and drawing conclusions, because it can mask any underlying trends or correlations. Such conclusions are only tenuous, at best, and leave the research open for criticism.
Meta-analysis is the process of drawing from a larger body of research, and using powerful statistical analyzes on the conglomerated data.
This gives a much larger sample population and is more likely to generate meaningful and usable data.
The Advantages of Meta-Analysis
Meta-analysis is an excellent way of reducing the complexity and breadth of research, allowing funds to be diverted elsewhere. For rare medical conditions, it allows researchers to collect data from further afield than would be possible for one research group.
As the method becomes more common, database programs have made the process much easier, with professionals working in parallel able to enter their results and access the data. This allows constant quality assessments and also reducing the chances of unnecessary repeat research, as papers can often take many months to be published, and the computer records ensure that any researcher is aware of the latest directions and results.
The field of meta study is also a lot more rigorous than the traditional literature review, which often relies heavily upon the individual interpretation of the researcher.
When used with the databases, a meta study allows a much wider net to be cast than by the traditional literature review, and is excellent for highlighting correlations and links between studies that may not be readily apparent as well as ensuring that the compiler does not subconsciously infer correlations that do not exist.
The Disadvantages of Meta-Analysis
There are a number of disadvantages to meta-analysis, of which a researcher must be aware before relying upon the data and generated statistics.
The main problem is that there is the potential for publication bias and skewed data.
Research generating results not refuting a hypothesis may tend to remain unpublished, or risks not being entered into the database. If the meta study is restricted to the research with positive results, then the validity is compromised.
The researcher compiling the data must make sure that all research is quantitative, rather than qualitative, and that the data is comparable across the various research programs, allowing a genuine statistical analysis.
It is important to pre-select the studies, ensuring that all of the research used is of a sufficient quality to be used.
One erroneous or poorly conducted study can place the results of the entire meta-analysis at risk. On the other hand, setting almost unattainable criteria and criteria for inclusion can leave the meta study with too small a sample size to be statistically relevant.
Striking a balance can be a little tricky, but the whole field is in a state of constant development, incorporating protocols similar to the scientific method used for normal quantitative research.
Finding the data is rapidly becoming the real key, with skilled meta-analysts developing a skill-set of library based skills, finding information buried in government reports and conference data, developing the knack of assessing the quality of sources quickly and effectively.
Conclusions and the Future
Meta-analysis is here to stay, as an invaluable tool for research, and is rapidly gaining momentum as a stand-alone discipline, with practitioners straddling the divide between statisticians and librarians.
The conveniences, as long as the disadvantages are taken into account, are too apparent to ignore, and a meta study can reduce the need for long, expensive and potentially intrusive repeated research studies.