Adventures in Machine Learning

Detecting Outliers in Regression Analysis with Studentized Residuals: A Python Tutorial

When analyzing data, it’s common to use regression analysis to determine the relationship between two variables. Regression analysis can provide meaningful insights into the underlying dynamics between variables, but it’s important to evaluate the residuals to ensure that the model is accurately capturing the data.

Studentized residuals are a useful tool for evaluating residuals and detecting outliers. In this article, we’ll explore what studentized residuals are, how to calculate them in Python, and how to interpret and visualize them to gain valuable insights.

to Studentized Residuals

Definition and Significance

When performing regression analysis, residuals refer to the difference between the observed and predicted values. Studentized residuals take this a step further by standardizing the residuals using the estimated standard deviation.

By doing this, we can determine if a residual is an outlier relative to the other residuals in the data set. This is important because outliers can significantly impact the accuracy of regression analysis.

Studentized residuals are therefore useful in detecting outliers that must be further analyzed for their impact on the analysis and the dataset. When making decisions based on a model or an analysis, its important that outliers are identified and handled.

Calculating Studentized Residuals in Python

Python offers a simple way to calculate studentized residuals. We can start by performing a simple linear regression and then use the outlier_test() function.

This function is present as part of the stats model library in python, a commonly used library for statistical analysis and regression modeling. Before we can perform linear regression, we have to convert our data set into a DataFrame object using pandas, which is a popular open-source library known for manipulating data.

#Importing the necessary Libraries

import pandas as pd

import statsmodels.api as sm

from statsmodels.stats.outliers_influence import outlier_test

dataset = pd.read_csv(“dataset.csv”)



results=sm.OLS(y, sm.add_constant(x)).fit()

#Calculate the studentized residuals

student_resid = pd.Series(outlier_test(results).iloc[:, 0])

The variable results stores the results of the linear regression model and the variable student_resid stores the series of studentized residuals. By calculating studentized residuals, we can identify any potential outliers.

Interpreting Studentized Residuals in Python

Understanding Output Results

After calculating studentized residuals, it’s important to understand their output. The DataFrame object they are stored in contains several columns, with student_resid being the most important.

The student_resid column contains the studentized residuals, and a value greater than 3 suggests an extreme outlier. Another column to consider is the unadj_p column.

In most cases, we use a significance level or p-value of 0.05. If the unadj_p value is less than 0.05, the outlier is considered statistically significant.

We can also adjust for multiple comparisons using the Bonferroni correction. The bonf(p) column shows the adjusted p-value for the Bonferroni correction.

Visualizing Studentized Residuals in Python

Visualizing studentized residuals can be useful in identifying outliers visually. The most commonly used graph to visualize these residuals is a scatterplot of the predictor variable against the studentized residuals, and we use the matplotlib.pyplot library to visualize the data.

import matplotlib.pyplot as plt

plt.scatter(x, student_resid)

plt.axhline(y=0, color=’red’, linestyle=’–‘)

plt.title(‘Scatterplot of Predictor Variable vs Studentized Residuals’)

plt.xlabel(‘Predictor Variable’)

plt.ylabel(‘Studentized Residuals’)

We can now interpret the scatterplot and identify the outliers on the graph. Any points that fall significantly above or below the red dashed line (y=0) are considered outliers, and we should further investigate these data points.


In this article, we’ve explored what studentized residuals are, how to calculate them in python and how to interpret and visualize them. By using studentized residuals, we can evaluate the accuracy of a regression analysis model, identify potential outliers, and gain valuable insights.

These insights can help us make better decisions that are more impactful and more accurate; hence it’s important to use them while working with data. In this article, we’ve discussed studentized residuals in detail, covering what they are, how to calculate them in Python, and how to interpret and visualize them.

Studentized residuals are a helpful tool for detecting outliers and evaluating the accuracy of regression models. By interpreting these residuals, we can make better-informed decisions based on data analytics that are more impactful.

It’s important to use studentized residuals in data analysis to accurately evaluate results and understand potential outliers. In summary, studentized residuals are a valuable tool for data analysis, providing insights that can improve decision-making processes.