Intraclass Correlation Coefficient (ICC)

SHARE THE ARTICLE ON

Intraclass Correlation Coefficient (ICC) Correlation Coefficient
Table of Contents

What is a Correlation Coefficient?

A correlation coefficient is a measure of the strength of the relationship between two variables. Although there are many different types of correlation coefficients, the most commonly used is Pearson’s correlation coefficient (or Pearson’s R). Pearson’s correlation coefficient can be used to indicate the strength and direction of the linear relationship between two variables. 

In today’s article, we will be specifically exploring the intraclass correlation coefficient to understand what it is, when it is used, and how it is calculated.

Exploratory Research Guide

Conducting exploratory research seems tricky but an effective guide can help.

What is the Intraclass Correlation Coefficient (ICC)?

The intraclass correlation coefficient, or ICC, is a value that acts as a measure of the reliability of the ratings for clusters. Clusters refer to data that exists in groups. ICC is used to find correlations within a single class of data rather than two different classes of data. 

As mentioned above, the most commonly used correlation coefficient is Pearson’s r. Pearson’s correlation coefficient is generally used for inter-rater reliability when there are only one or two meaningful pairs from one or two raters. However, if there are more pairs, the intraclass correlation coefficient should be used. 

Types of Intraclass Correlation Coefficients (ICC)

There are many different versions of ICC that can be calculated. The following factors influence which version is chosen: 

  • Model
  • Type of Relationship 
  • Unit 

Model

There are three key models: 

  • One-way Random Effects Model: Assumes that every individual subject is rated by a different group of randomly selected raters. 
  • Two-way Random Effects Model: Assumes that a group of k raters is randomly chosen from a population and is then used to rate subjects.
  • Two-way Mixed Effects Model: Like the two-way effects model, this model too assumes that a group of k raters is randomly chosen from a population and then used to rate subjects. However, this model assumes that the group of raters selected are the only raters of interest and is hence used when the findings do not need to be generalized to other raters. 

Type of Relationship 

There are two key types of relationships that we are generally measuring when using ICC:

  • Consistency: When we are trying to identify the systematic differences between the ratings of judges.
  • Absolute Agreement: When we are trying to identify the absolute differences between the ratings of judges.

Unit 

There are two key units that we are generally measuring when using ICC: 

    • Single Rater: When we only want to use the ratings from a single rater as the basis of measurement. 
    • Mean of Raters: When we want to use the mean of ratings from all judges as the basis of measurement. 

 

See Voxco survey software in action with a Free demo.

See Voxco survey software in action with a Free demo.

How to Calculate the Intraclass Correlation Coefficient

Formula to Calculate Intraclass Correlation Coefficient

Calculating the ICC for sets of data can be extremely complex. This is partly due to the fact that there are many different formulas that can be used for its calculation. Within this article we will go over a formula that is commonly used to calculate ICC. 

Intraclass correlation coefficient is generally calculated as a ratio, using the following formula: 

(variance of interest) / (total variance) = (variance of interest) / (variance of interest + unwanted variance)

Interpretation of ICC Values

The value of ICC can range from 0 to 1, which is different from Pearson’s correlation coefficient that can take a value between -1 to +1. 

When the ICC value is below 0.5, it indicates that the unwanted variance of interest is equal to or larger than the variance of interest, reflecting that the reliability of the method is poor. When the ICC value is above 0.8, it can be inferred  that the reliability of the method is good or excellent.

FAQs on Intraclass Correlation Coefficient

A correlation coefficient is a value that indicates the strength and direction of the relationship between two variables. There are many different kinds of correlation coefficient, the most common being Pearson’s R that is used in linear regression.

The intraclass correlation coefficient is a value that acts as a measure of the reliability of the ratings for clusters (data that exists in groups or is sorted into groups).

Pearson’s correlation coefficient is generally used for inter-rater reliability when there are only one or two meaningful pairs from one or two raters. However, if there are more pairs, the intraclass correlation coefficient should be used.

Intraclass correlation coefficient is generally calculated as a ratio, using the following formula: 

(variance of interest) / (total variance) = (variance of interest) / (variance of interest + unwanted variance)

Online survey tools 10 1

See why 450+ clients trust Voxco!

By providing this information, you agree that we may process your personal data in accordance with our Privacy Policy.

Read more

Shopping Basket
Great Research
Fast Insights
Best-in-class ROI

Voxco’s platform helps you gather omnichannel feedback, measure sentiment, uncover insights and act on them.

Join 500 + global clients across 40+ countries

Great Research
Fast Insights
Best-in-class ROI

Voxco’s platform helps you gather omnichannel feedback, measure sentiment, uncover insights and act on them.

Join 500 + global clients across 40+ countries