This is before looking at the statistics required, and studying the preferred methods for the individual scientific discipline.
Every experimental design must make compromises and generalizations, so the researcher must try to minimize these, whilst remaining realistic.
For ‘pure’ sciences, such as chemistry or astrophysics, experiments are quite easy to define and will, usually, be strictly quantitative.
For biology, psychology and social sciences, there can be a huge variety of methods to choose from, and a researcher will have to justify their choice. Whilst slightly arbitrary, the best way to look at the various methods is in terms of ‘strength’.
Experimental Research Methods
The first method is the straightforward experiment, involving the standard practice of manipulating quantitative, independent variables to generate statistically analyzable data.
Generally, the system of scientific measurements is interval or ratio based. When we talk about ‘scientific research methods’, this is what most people immediately think of, because it passes all of the definitions of ‘true science’. The researcher is accepting or refuting the null hypothesis.
The results generated are analyzable and are used to test hypotheses, with statistics giving a clear and unambiguous picture.
This research method is one of the most difficult, requiring rigorous design and a great deal of expense, especially for larger experiments. The other problem, where real life organisms are used, is that taking something out of its natural environment can seriously affect its behavior.
It is often argued that, in some fields of research, experimental research is ‘too’ accurate. It is also the biggest drain on time and resources, and is often impossible to perform for some fields, because of ethical considerations.
In other fields of study, which do not always have the luxury of definable and quantifiable variables - you need to use different research methods. These should attempt to fit all of the definitions of repeatability or falsifiability, although this is not always feasible.
Opinion Based Research Methods
Opinion based research methods generally involve designing an experiment and collecting quantitative data. For this type of research, the measurements are usually arbitrary, following the ordinal or interval type.
Questionnaires are an effective way of quantifying data from a sample group, and testing emotions or preferences. This method is very cheap and easy, where budget is a problem, and gives an element of scale to opinion and emotion. These figures are arbitrary, but at least give a directional method of measuring intensity.
Quantifying behavior is another way of performing this research, with researchers often applying a ‘numerical scale’ to the type, or intensity, of behavior. The Bandura Bobo Doll experiment and the Asch Experiment were examples of opinion based research.
By definition, this experiment method must be used where emotions or behaviors are measured, as there is no other way of defining the variables.
Observational research is a group of different research methods where researchers try to observe a phenomenon without interfering too much.
Observational research methods, such as the case study, are probably the furthest removed from the established scientific method. This type is looked down upon, by many scientists, as ‘quasi-experimental’ research, although this is usually an unfair criticism. Observational research tends to use nominal or ordinal scales of measurement.
Observational research often has no clearly defined research problem, and questions may arise during the course of the study. For example, a researcher may notice unusual behavior and ask, ‘What is happening?’ or ‘Why?’
Observation is heavily used in social sciences, behavioral studies and anthropology, as a way of studying a group without affecting their behavior. Whilst the experiment cannot be replicated or falsified, it still offers unique insights, and will advance human knowledge.
Case studies are often used as a pre-cursor to more rigorous methods, and avoid the problem of the experiment environment affecting the behavior of an organism. Observational research methods are useful when ethics are a problem.
In an ideal world, experimental research methods would be used for every type of research, fulfilling all of the requirements of falsifiability and generalization.
However, ethics, time and budget are major factors, so any experimental design must make compromises. As long as a researcher recognizes and evaluates flaws in the design when choosing from different research methods, any of the scientific research methods are valid contributors to scientific knowledge.