True Experimental Design How to transform market research insights into action?​

True Experimental Design - Types & How to Conduct

SHARE THE ARTICLE ON

Table of Contents

True-experimental research is often considered the most accurate research. A researcher has complete control over the process which helps reduce any error in the result. This also increases the confidence level of the research outcome. 

In this blog, we will explore in detail what it is, its various types, and how to conduct it in 7 steps.

What is a True Experimental Design?

True experimental design is a statistical approach to establishing a cause-and-effect relationship between different variables. This is one of the most accurate forms of research designs which provides substantial backing to support the existence of relationships.

There are three criteria that need to be met in order to carry out this type of research:

1. The existence of a control group: The sample of participants is subdivided into 2 groups – one that is subjected to the experiment and so, undergoes changes and the other that does not. 

2. The presence of an independent variable: Independent variables that influence the working of other variables must be there for the researcher to control and observe changes.

3. Random assignment: Participants must be randomly distributed within the groups.

See Voxco survey software in action with a Free demo.

An example of True Experimental Design

A study to observe the effects of physical exercise on productivity levels can be conducted using a true experimental design.

Suppose a group of 300 people volunteer for a study involving office workers in their 20s. These 300 participants are randomly distributed into 3 groups. 

  • 1st Group: A control group that does not participate in exercising and has to carry on with their everyday schedule. 
  • 2nd Group: Asked to indulge in home workouts for 30-45 minutes every day for one month. 
  • 3rd Group: Has to work out 2 hours every day for a month. Both groups have to take one rest day per week.

In this research, the level of physical exercise acts as an independent variable while the performance at the workplace is a dependent variable that varies with the change in exercise levels.

Before initiating the true experimental research, each participant’s current performance at the workplace is evaluated and documented. As the study goes on, a progress report is generated for each of the 300 participants to monitor how their physical activity has impacted their workplace functioning.

At the end of two weeks, participants from the 2nd and 3rd groups that are able to endure their current level of workout, are asked to increase their daily exercise time by half an hour. While those that aren’t able to endure, are suggested to either continue with the same timing or fix the timing to a level that is half an hour lower. 

So, in this true experimental design a participant who at the end of two weeks is not able to put up with 2 hours of workout, will now workout for 1 hour and 30 minutes for the remaining tenure of two weeks while someone who can endure the 2 hours, will now push themselves towards 2 hours and 30 minutes.

In this manner, the researcher notes the timings of each member from the two active groups for the first two weeks and the remaining two weeks after the change in timings and also monitors their corresponding performance levels at work.

The above example can be categorized as true experiment research since now we have:

  • Control group: Group 1 carries on with their schedule without being conditioned to exercise.
  • Independent variable: The duration of exercise each day.
  • Random assignment: 300 participants are randomly distributed into 3 groups and as such, there are no criteria for the assignment.

What is the purpose of conducting true experimental research?

Both the primary usage and purpose of a true experimental design lie in establishing meaningful relationships based on quantitative surveillance. 

True experiments focus on connecting the dots between two or more variables by displaying how the change in one brings about a change in another variable. It can be as simple as having enough sleep improves retention or as large scale as geographical differences affect consumer behavior. 

The main idea is to ensure the presence of different sets of variables to study with some shared commonality.

Beyond this, the research is used when the three criteria of random distribution, a control group, and an independent variable to be manipulated by the researcher, are met.

What are the advantages of true experimental design?

Let’s take a look at some of the advantages that make this research design conclusive and accurate research. 

Concrete method of research:

The statistical nature of the experimental design makes it highly credible and accurate. The data collected from the research is subjected to statistical tools. 

This makes the results easy to understand, objective and actionable. This makes it a better alternative to observation-based studies that are subjective and difficult to make inferences from.

Easy to understand and replicate:

Since the research provides hard figures and a precise representation of the entire process, the results presented become easily comprehensible for any stakeholder. 

Further, it becomes easier for future researchers conducting studies around the same subject to get a grasp of prior takes on the same and replicate its results to supplement their own research.

Establishes comparison:

The presence of a control group in true experimental research allows researchers to compare and contrast. The degree to which a methodology is applied to a group can be studied with respect to the end result as a frame of reference.

Conclusive:

The research combines observational and statistical analysis to generate informed conclusions. This directs the flow of follow-up actions in a definite direction, thus, making the research process fruitful.

What are the disadvantages of true experimental design?

We should also learn about the disadvantages it can pose in research to help you determine when and how you should use this type of research. 

Expensive:

This research design is costly. It takes a lot of investment in recruiting and managing a large number of participants which is necessary for the sample to be representative. 

The high resource investment makes it highly important for the researcher to plan each aspect of the process to its minute details.

Too idealistic:

The research takes place in a completely controlled environment. Such a scenario is not representative of real-world situations and so the results may not be authentic. 

This is one of the main reasons why open-field research is preferred over lab research wherein the study can be influenced by the researcher.

Time-consuming:

The process of setting up and conducting a true experiment is highly time-consuming. This is because of the processes like recruiting a large enough sample, gathering respondent data, random distribution into groups, monitoring the process over a span of time, tracking changes, and making adjustments. 

The amount of processes although essential to the entire model is not a feasible option to go for when the results are required in the near future.

Now that we have learned about the advantages and disadvantages, let’s look at its types.

Get started with your Experimental Research

Send your survey to the right people to receive quality responses.

What are the 3 types of true experimental design?

The research design is categorized into three types based on the way you should conduct the research. Each type has its own procedure and guidelines which you should be aware of in order to achieve reliable data. 

The three types are: 

1) Post-test-only control group design. 

2) Pre-test post-test control group design.

3) Solomon four group control design.

Let’s see how these three types differ. 

1) Post-test-only control group design:

In this type of true experimental research, the control as well as the experimental group that has been formed using random allocation, are not tested before applying the experimental methodology. This is so as to avoid affecting the quality of the study.

The participants are always on the lookout to identify the purpose and criteria for assessment. Pre-test conveys to them the basis on which they are being judged which can allow them to modify their end responses, compromising the quality of the entire research process. 

However, this can hinder your ability to establish a comparison between the pre-experiment and post-experiment conditions which weighs in on the changes that have taken place over the course of the research.

2) Pre-test post-test control group design:

It is a modification of the post-test control group design with an additional test carried out before the implementation of the experimental methodology. 

This two-way testing method can help in noticing significant changes brought in the research groups as a result of the experimental intervention. There is no guarantee that the results present the true picture as post-testing can be affected due to the exposure of the respondents to the pre-test.

3) Solomon four group control design:

As the name suggests, this type of true experimental design involves the random distribution of sample members into 4 groups. These groups consist of 2 control groups that are not subjected to the experiments and changes, and 2 experimental groups that the experimental methodology applies to.

Out of these 4 groups, one control and one experimental group is used for pre-testing while all four groups are subjected to post-tests.

This way researcher gets to establish pre-test post-test contrast while there remains another set of respondents that have not been exposed to pre-tests and so, provide genuine post-test responses, thus, accounting for testing effects.

Customer experience

What is the difference between pre-experimental & true experimental research design?

Pre-experimental research helps determine the researchers’ intervention on a group of people. It is a step where you design the proper experiment to address a research question. 

True experiment defines that you are conducting the research. It helps establish a cause-and-effect relationship between the variables. 

We will discuss the differences between the two based on four categories, which are: 

  1. Observatory Vs. Statistical. 
  2. Absence Vs. Presence of control groups. 
  3. Non-randomization Vs. Randomization. 
  4. Feasibility test Vs. Conclusive test.

Let’s find the differences to better understand the two experiments. 

Observatory vs Statistical:

Pre-experimental research is an observation-based model i.e. it is highly subjective and qualitative in nature. 

The true experimental design offers an accurate analysis of the data collected using statistical data analysis tools.

Absence vs Presence of control groups:

Pre-experimental research designs do not usually employ a control group which makes it difficult to establish contrast. 

While all three types of true experiments employ control groups.

Non-randomization vs Randomization:

Pre-experimental research doesn’t use randomization in certain cases whereas 

True experimental research always adheres to a randomization approach to group distribution.

Feasibility test vs Conclusive test:

Pre-tests are used as a feasibility mechanism to see if the methodology being applied is actually suitable for the research purpose and whether it will have an impact or not.

While true experiments are conclusive in nature.

7 Steps to conduct a true experimental research

It is important to understand the proper steps/guidelines of research in order to maintain research integrity and gather valid and reliable data. 

We have explained 7 steps to conducting this research in detail. The TL;DR version of it is: 

1) Identify the research objective.

2) Identify independent and dependent variables.

3) Define and group the population.

4) Conduct Pre-tests.

5) Conduct the research.

6) Conduct post-tests.

7) Analyse the collected data. 

Now let’s explore these seven steps in true experimental design. 

1) Identify the research objective:

Identify the variables which you need to analyze for a cause-and-effect relationship. Deliberate which particular relationship study will help you make effective decisions and frame this research objective in one of the following manners:

  • Determination of the impact of X on Y
  • Studying how the usage/application of X causes Y

2) Identify independent and dependent variables:

Establish clarity as to what would be your controlling/independent variable and what variable would change and would be observed by the researcher. In the above samples for research purposes, X is the independent variable and Y is the dependent variable.

3) Define and group the population:

Define the targeted audience for the true experimental design. It is out of this target audience that a sample needs to be selected for accurate research to be carried out. It is imperative that the target population gets defined in as much detail as possible.

To narrow the field of view, a random selection of individuals from the population is carried out. These are the selected respondents that help the researcher in answering their research questions. Post their selection, this sample of individuals gets randomly subdivided into control and experimental groups.

4) Conduct Pre-tests:

Before commencing with the actual study, pre-tests are to be carried out wherever necessary. These pre-tests take an assessment of the condition of the respondent so that an effective comparison between the pre and post-tests reveals the change brought about by the research.

5) Conduct the research:

Implement your experimental procedure with the experimental group created in the previous step in the true experimental design. Provide the necessary instructions and solve any doubts or queries that the participants might have. Monitor their practices and track their progress. Ensure that the intervention is being properly complied with, otherwise, the results can be tainted.

6) Conduct post-tests:

Gauge the impact that the intervention has had on the experimental group and compare it with the pre-tests. This is particularly important since the pre-test serves as a starting point from where all the changes that have been measured in the post-test, are the effect of the experimental intervention. 

So for example: If the pre-test in the above example shows that a particular customer service employee was able to solve 10 customer problems in two hours and the post-test conducted after a month of 2-hour workouts every day shows a boost of 5 additional customer problems being solved within those 2 hours, the additional 5 customer service calls that the employee makes is the result of the additional productivity gained by the employee as a result of putting in the requisite time

7) Analyse the collected data:

Use appropriate statistical tools to derive inferences from the data observed and collected. Correlational data analysis tools and tests of significance are highly effective relationship-based studies and so are highly applicable for true experimental research.

This step also includes differentiating between the pre and the post-tests for scoping in on the impact that the independent variable has had on the dependent variable. A contrast between the control group and the experimental groups sheds light on the change brought about within the span of the experiment and how much change is brought intentionally and is not caused by chance.

Explore all the survey question types possible on Voxco

Wrapping up;

This sums up everything about true experimental design. While it is often considered complex and expensive, it is also one of the most accurate research. 

The true experiment uses statistical analysis which ensures that your data is reliable and has a high confidence level. Curious to learn how you can use survey software to conduct your experimental research, book a meeting with us.

Read more

Business Intelligence

Business Intelligence SHARE THE ARTICLE ON Table of Contents Business intelligence (BI) refers to the applications, infrastructure, tools, and best practices that enable information access

Read More »

Satisfaction Survey Questions

Satisfaction Survey Questions SHARE THE ARTICLE ON Share on facebook Share on twitter Share on linkedin Table of Contents What is a Satisfaction Survey? Companies

Read More »