Intraclass Correlation Coefficient (ICC)

There are many different versions of ICC that can be calculated. The following factors influence which version is chosen: 

• Model
• Type of Relationship 
• Unit 

Model

There are three key models: 

• One-way Random Effects Model: Assumes that every individual subject is rated by a different group of randomly selected raters. 
• Two-way Random Effects Model: Assumes that a group of k raters is randomly chosen from a population and is then used to rate subjects.
• Two-way Mixed Effects Model: Like the two-way effects model, this model too assumes that a group of k raters is randomly chosen from a population and then used to rate subjects. However, this model assumes that the group of raters selected are the only raters of interest and is hence used when the findings do not need to be generalized to other raters. 

Type of Relationship 

There are two key types of relationships that we are generally measuring when using ICC:

• Consistency: When we are trying to identify the systematic differences between the ratings of judges.
• Absolute Agreement: When we are trying to identify the absolute differences between the ratings of judges.

Unit 

There are two key units that we are generally measuring when using ICC: 

• • Single Rater: When we only want to use the ratings from a single rater as the basis of measurement. 
  • Mean of Raters: When we want to use the mean of ratings from all judges as the basis of measurement.