From Basics to Brilliance: Insights into Logistic Regression Assumptions

• No extreme outliers

Logistic regression assumes that there are no extreme outliers or any external observations that influence the data that goes into the model. 

Cook’s distance is an effective way to rule out the outliers and external observations from a dataset. You can choose to eradicate those from the data or decide to replace them with a mean or median. You can also let the outliers be, but remember to report those in the regression results. 

• The explanatory variables and the Logit of response variable have a linear relationship between them.

The Logit is stated as:

Logit(p)  = log(p / (1-p)) 

Where p is the probability of an outcome to be positive.

The logistic regression assumes that this Logit of the response variable and the explanatory variables are linearly related.

Box-Tidwell test is used to see if this assumption stands true in your dataset for the regression model. 

• Sufficient sample size 

The logistic regression assumes that the sample size from which the observations are drawn is large enough to give reliable conclusions for the regression model.

There is a rule of thumb to put this assumption in place. You need to have at least 10 cases where the outcome is not very frequent, for each explanatory variable. Let’s say you have 5 explanatory variables and you are expecting the probability of the least frequent outcome turns out to be 0.30, the model demands the sample size of at least (10*5)/0.30 = 166. 

• Both logistic regression and linear regression have common assumptions:

• A linear relationship between the explanatory variables and the response variable.
• Normally distributed residuals. 
• Homoscedasticity between the residuals.