Autocorrelation in Linear Regression Models
Autocorrelation is the degree to which a time series data set is dependent on previous measurements. It is an essential concept in time series analysis because it provides information on the data’s behavior over time.
In linear regression, the assumption is that the residuals (the difference between the predicted values and the actual values) should be independent of each other. However, autocorrelation violates this assumption, leading to biased and inconsistent estimates.
This article examines two statistical tests used to detect autocorrelation in linear regression models: the Durbin-Watson test and the Breusch-Godfrey test. We’ll also examine ways to handle autocorrelation, such as adding lags or overdifferencing.
1. Durbin-Watson Test
The Durbin-Watson test is a statistical test used to detect the presence of autocorrelation in the residuals of a linear regression model. The test statistic ranges from 0 to 4, with values closer to 2 indicating no autocorrelation and values closer to 0 or 4 indicating positive or negative autocorrelation, respectively.
The null hypothesis of the test is that there is no autocorrelation. To perform the test, we calculate the test statistic as follows:
DW = (sum of squared differences of adjacent residuals)/(sum of squared residuals)If the calculated DW value is significantly less than 2, then there is evidence of positive autocorrelation in the residuals.
Conversely, if the DW value is significantly greater than 2, there is evidence of negative autocorrelation in the residuals. In such cases, we need to examine our data more closely or undertake other statistical tests to detect any serial correlation.
2. Breusch-Godfrey Test
The Breusch-Godfrey test is a statistical test used to detect autocorrelation in the residuals of a linear regression model with one or more predictor variables. The Breusch-Godfrey test statistic is based on the residuals, where the null hypothesis is that there is no autocorrelation.
The general structure of the Breusch-Godfrey test is similar to that of the Durbin-Watson test, but the Breusch-Godfrey test can handle more complex linear regression models. The test is performed as follows:
- Fit a linear regression model using OLS with predictor variables and a response variable
- Calculate the residuals of the fitted model
- Estimate the residuals by regressing them on the original predictor variables and several lagged values of the residuals
- Calculate the test statistic as [(number of observations – number of parameters) * (R-squared of the residuals regression model)].
- Compare the calculated test statistic with the chi-square distribution with degrees of freedom equal to the number of lagged residuals used in step 3.
If the calculated test statistic exceeds the critical value at the desired significance level, then we reject the null hypothesis that there is no autocorrelation and conclude that the residuals exhibit autocorrelation.
3. Handling Autocorrelation
It is essential to handle autocorrelation in linear regression models because ignoring it leads to biased and inconsistent estimates.
One way to handle autocorrelation is to add lagged values of the residuals as predictor variables in the regression model. We choose the number of lagged residuals by looking at the Durbin-Watson statistic and examining residual plots.
Another way to handle autocorrelation is to overdifference the data, which means differencing the data one or more times until the autocorrelation in the residuals is eliminated or significantly reduced. We can also use weighted least squares regression models, robust regression models, or autoregressive conditional heteroscedasticity (ARCH) models to handle autocorrelation in time series data.
4. Example: Performing Breusch-Godfrey Test in Python
Suppose we want to model the relationship between the number of hours studied and the exam score obtained. We create a dataset containing five observations, where each observation has two columns: the number of hours studied and the exam score obtained.
The dataset is as follows:
| Hours Studied | Exam Score |
|---|---|
| 2 | 70 |
| 3 | 75 |
| 4 | 85 |
| 5 | 90 |
| 6 | 95 |
We fit a multiple linear regression model using OLS with the number of hours studied as the predictor variable and the exam score obtained as the response variable. We then calculate the residuals of the fitted model and perform the Breusch-Godfrey test by regressing the residuals on one lagged residual.
The output of the Breusch-Godfrey test in Python is as follows:
Breusch-Godfrey LM test
chi2(1) = 2.00
Prob > chi2 = 0.1578The test statistic is 2.00, and the p-value is 0.1578, which is greater than the desired significance level of 0.05. Therefore, we accept the null hypothesis that there is no autocorrelation in the residuals.
We conclude that our model is unbiased and consistent and that there is no need to add lagged residuals or overdifference the data.
5. Conclusion
In summary, autocorrelation is a fundamental concept in time series analysis and linear regression models.
The presence of autocorrelation in residuals violates the independence assumption and leads to biased and inconsistent estimates. The Durbin-Watson test and the Breusch-Godfrey test are two statistical tests used to detect autocorrelation in residuals.
We can handle autocorrelation by adding lagged residuals, overdifferencing the data, or using weighted least squares, robust regression, and ARCH models. It is essential to handle autocorrelation in regression models to obtain accurate and reliable estimates and make valid conclusions in real-world applications.
In conclusion, autocorrelation is a significant concept in time series analysis and linear regression models that must be adequately addressed for accurate and reliable estimates. The Durbin-Watson test and the Breusch-Godfrey test are two tests that help in detecting autocorrelation in residuals.
Handling autocorrelation can be accomplished by adding lagged residuals, overdifferencing the data, or using robust regression, ARCH, and weighted least squares models. Linear regression models with autocorrelation neglect may lead to misleading conclusions, and getting conclusions right is essential in making evidence-based decisions that guarantee success.