What is the difference between true experimental and quasi experimental research designs

Question 1 Explain the difference between true experimental and quasi-experimental research design. Provide examples in your answer. (265 words) True experimental and quasi-experimental research designs are the two most common forms of research design. While they both share similar elements, such as measuring participant results to test the hypothesis, there are also significant differences between true experimental and quasi-experimental research designs (The Regents of the University of Michigan, 2013 ). True experimental research design includes random selection and group assignment of participants, manipulation of variables and observing the effect that the manipulation has on the dependant variable to establish whether a cause and effect relationship is present between the variables (Follmer Greenhoot, 2003, pp. 92-93). An example of a commonly used true experimental research designs, are experiments used for testing the effects of new pharmaceuticals. Quasi-experimental research, which was once considered ‘flawed’ and less superior to true experimental research, has become increasingly popular over the last three to four decades in many areas, especially the social sciences (Coolican, 2014, p. 121). It differs from true experimental research design in the way that quasi-experimental research doesn’t involve random assignment of participants to groups and often has less control over the independent variable (Follmer Greenhoot, 2003, p. 94; Coolican, 2014, p. 121). An

Inhaltsverzeichnis Show

What is a quasi-experimental design?
What is an experimental design?
When to choose an experimental design over a quasi-experimental design?
When to choose a quasi-experimental design over a true experiment?
Further reading
Differences between quasi-experiments and true experiments
Example of a true experiment vs a quasi-experiment
Types of quasi-experimental designs
Nonequivalent groups design
Regression discontinuity
Natural experiments
When to use quasi-experimental design
Advantages and disadvantages
Frequently asked questions about quasi-experimental designs

Table of Contents Show

What is a quasi-experimental design?
What is an experimental design?
When to choose an experimental design over a quasi-experimental design?
When to choose a quasi-experimental design over a true experiment?
Further reading
Differences between quasi-experiments and true experiments
Example of a true experiment vs a quasi-experiment
Types of quasi-experimental designs
Nonequivalent groups design
Regression discontinuity
Natural experiments
When to use quasi-experimental design
Advantages and disadvantages
Frequently asked questions about quasi-experimental designs

Here’s a table that summarizes the similarities and differences between an experimental and a quasi-experimental study design:

	Experimental Study (a.k.a. Randomized Controlled Trial)	Quasi-Experimental Study
Objective	Evaluate the effect of an intervention or a treatment	Evaluate the effect of an intervention or a treatment
How participants get assigned to groups?	Random assignment	Non-random assignment (participants get assigned according to their choosing or that of the researcher)
Is there a control group?	Yes	Not always (although, if present, a control group will provide better evidence for the study results)
Is there any room for confounding?	No (although check Manson et al. for a detailed discussion on post-randomization confounding in randomized controlled trials)	Yes (however, statistical techniques can be used to study causal relationships in quasi-experiments)
Level of evidence	A randomized trial is at the highest level in the hierarchy of evidence	A quasi-experiment is one level below the experimental study in the hierarchy of evidence [source]
Advantages	Minimizes bias and confounding	– Can be used in situations where an experiment is not ethically or practically feasible – Can work with smaller sample sizes than randomized trials
Limitations	– High cost (as it generally requires a large sample size)– Ethical limitations– Generalizability issues – Sometimes practically infeasible	Lower ranking in the hierarchy of evidence as losing the power of randomization causes the study to be more susceptible to bias and confounding

What is a quasi-experimental design?

A quasi-experimental design is a non-randomized study design used to evaluate the effect of an intervention. The intervention can be a training program, a policy change or a medical treatment.

Unlike a true experiment, in a quasi-experimental study the choice of who gets the intervention and who doesn’t is not randomized. Instead, the intervention can be assigned to participants according to their choosing or that of the researcher, or by using any method other than randomness.

Having a control group is not required, but if present, it provides a higher level of evidence for the relationship between the intervention and the outcome.

(for more information, I recommend my other article: Understand Quasi-Experimental Design Through an Example).

Examples of quasi-experimental designs include:

What is an experimental design?

An experimental design is a randomized study design used to evaluate the effect of an intervention. In its simplest form, the participants will be randomly divided into 2 groups:

A treatment group: where participants receive the new intervention which effect we want to study.
A control or comparison group: where participants do not receive any intervention at all (or receive some standard intervention).

Randomization ensures that each participant has the same chance of receiving the intervention. Its objective is to equalize the 2 groups, and therefore, any observed difference in the study outcome afterwards will only be attributed to the intervention – i.e. it removes confounding.

(for more information, I recommend my other article: Purpose and Limitations of Random Assignment).

Examples of experimental designs include:

When to choose an experimental design over a quasi-experimental design?

Although many statistical techniques can be used to deal with confounding in a quasi-experimental study, in practice, randomization is still the best tool we have to study causal relationships.

Another problem with quasi-experiments is the natural progression of the disease or the condition under study — When studying the effect of an intervention over time, one should consider natural changes because these can be mistaken with changes in outcome that are caused by the intervention. Having a well-chosen control group helps dealing with this issue.

So, if losing the element of randomness seems like an unwise step down in the hierarchy of evidence, why would we ever want to do it?

This is what we’re going to discuss next.

When to choose a quasi-experimental design over a true experiment?

The issue with randomness is that it cannot be always achievable.

So here are some cases where using a quasi-experimental design makes more sense than using an experimental one:

If being in one group is believed to be harmful for the participants, either because the intervention is harmful (ex. randomizing people to smoking), or the intervention has a questionable efficacy, or on the contrary it is believed to be so beneficial that it would be malevolent to put people in the control group (ex. randomizing people to receiving an operation).
In cases where interventions act on a group of people in a given location, it becomes difficult to adequately randomize subjects (ex. an intervention that reduces pollution in a given area).
When working with small sample sizes, as randomized controlled trials require a large sample size to account for heterogeneity among subjects (i.e. to evenly distribute confounding variables between the intervention and control groups).

Differences between quasi-experiments and true experiments

There are several common differences between true and quasi-experimental designs.

	True experimental design	Quasi-experimental design
Assignment to treatment	The researcher randomly assigns subjects to control and treatment groups.	Some other, non-random method is used to assign subjects to groups.
Control over treatment	The researcher usually designs the treatment.	The researcher often does not have control over the treatment, but instead studies pre-existing groups that received different treatments after the fact.
Use of control groups	Requires the use of control and treatment groups.	Control groups are not required (although they are commonly used).

Example of a true experiment vs a quasi-experiment

Let’s say you are interested in the impact of a new psychological therapy on patients with depression. Example: True experimental designTo run a true experiment, you randomly assign half the patients in a mental health clinic to receive the new treatment. The other half—the control group—receives the standard course of treatment for depression.

Every few months, patients fill out a sheet describing their symptoms to see if the new treatment produces significantly better (or worse) effects than the standard one.

However, for ethical reasons, the directors of the mental health clinic may not give you permission to randomly assign their patients to treatments. In this case, you cannot run a true experiment.

Instead, you can use a quasi-experimental design.

Example: Quasi-experimental designYou discover that a few of the psychotherapists in the clinic have decided to try out the new therapy, while others who treat similar patients have chosen to stick with the normal protocol.

You can use these pre-existing groups to study the symptom progression of the patients treated with the new therapy versus those receiving the standard course of treatment.

Although the groups were not randomly assigned, if you properly account for any systematic differences between them, you can be reasonably confident any differences must arise from the treatment and not other confounding variables.

Types of quasi-experimental designs

Many types of quasi-experimental designs exist. Here we explain three of the most common types: nonequivalent groups design, regression discontinuity, and natural experiments.

Nonequivalent groups design

In nonequivalent group design, the researcher chooses existing groups that appear similar, but where only one of the groups experiences the treatment.

In a true experiment with random assignment, the control and treatment groups are considered equivalent in every way other than the treatment. But in a quasi-experiment where the groups are not random, they may differ in other ways—they are nonequivalent groups.

When using this kind of design, researchers try to account for any confounding variables by controlling for them in their analysis or by choosing groups that are as similar as possible.

This is the most common type of quasi-experimental design.

Example: Nonequivalent groups designYou hypothesize that a new after-school program will lead to higher grades. You choose two similar groups of children who attend different schools, one of which implements the new program while the other does not.

By comparing the children who attend the program with those who do not, you can find out whether it has an impact on grades.

Regression discontinuity

Many potential treatments that researchers wish to study are designed around an essentially arbitrary cutoff, where those above the threshold receive the treatment and those below it do not.

Near this threshold, the differences between the two groups are often so minimal as to be nearly nonexistent. Therefore, researchers can use individuals just below the threshold as a control group and those just above as a treatment group.

Example: Regression discontinuitySome high schools in the United States are set aside for high-achieving students, who must exceed a certain score on a test to be allowed to attend. Those who pass this test most likely differ systematically from those who do not.

However, since the exact cutoff score is arbitrary, the students near the threshold—those who just barely pass the exam and those who fail by a very small margin—tend to be very similar, with the small differences in their scores mostly due to random chance. You can therefore conclude that any outcome differences must come from the school they attended.

To test the impact of attending a selective school, you can study the long-term outcomes of these two groups of students (those who barely passed and those who barely failed).

Natural experiments

In both laboratory and field experiments, researchers normally control which group the subjects are assigned to. In a natural experiment, an external event or situation (“nature”) results in the random or random-like assignment of subjects to the treatment group.

Even though some use random assignments, natural experiments are not considered to be true experiments because they are observational in nature.

Although the researchers have no control over the independent variable, they can exploit this event after the fact to study the effect of the treatment.

Example: Natural experimentThe Oregon Health Study is one of the most famous natural experiments. In 2008, the state of Oregon decided to expand enrollment in Medicaid, America’s low-income public health insurance program, to more low-income adults.

However, as they could not afford to cover everyone who they deemed eligible for the program, they instead allocated spots in the program based on a random lottery.

Researchers were able to study the impact of the program by using the enrolled individuals as a randomly assigned treatment group, and the others who were eligible but did not succeed in the lottery as a control group.

When to use quasi-experimental design

Although true experiments have higher internal validity, you might choose to use a quasi-experimental design for ethical or practical reasons.

Ethical

Sometimes it would be unethical to provide or withhold a treatment on a random basis, so a true experiment is not feasible. In this case, a quasi-experiment can allow you to study the same causal relationship without the ethical issues.

The Oregon Health Study is a good example. It would be unethical to randomly provide some people with health insurance but purposely prevent others from receiving it solely for the purposes of research.

However, since the Oregon government faced financial constraints and decided to provide health insurance via lottery, studying this event after the fact is a much more ethical approach to studying the same problem.

Practical

True experimental design may be infeasible to implement or simply too expensive, particularly for researchers without access to large funding streams.

At other times, too much work is involved in recruiting and properly designing an experimental intervention for an adequate number of subjects to justify a true experiment.

In either case, quasi-experimental designs allow you to study the question by taking advantage of data that has previously been paid for or collected by others (often the government).

Advantages and disadvantages

Quasi-experimental designs have various pros and cons compared to other types of studies.

Higher external validity than most true experiments, because they often involve real-world interventions instead of artificial laboratory settings.
Higher internal validity than other non-experimental types of research, because they allow you to better control for confounding variables than other types of studies do.
Lower internal validity than true experiments—without randomization, it can be difficult to verify that all confounding variables have been accounted for.
The use of retrospective data that has already been collected for other purposes can be inaccurate, incomplete or difficult to access.

Frequently asked questions about quasi-experimental designs

What is random assignment?

In experimental research, random assignment is a way of placing participants from your sample into different groups using randomization. With this method, every member of the sample has a known or equal chance of being placed in a control group or an experimental group.