Multiple Linear Regression in Python: A Comprehensive Guide
As data analysis and machine learning continue to take the world by storm, multiple linear regression has been proven to be an effective tool in predicting the outcome of a dependent variable using several independent variables. Python, being a top preferred programming language for data analysis, makes it possible for users to perform multiple linear regression with ease.
In this article, we will be discussing how multiple linear regression can be performed in Python, checking for linearity, and using sklearn and statsmodels to perform the regression.
Checking for Linearity
Firstly, it is important to identify whether there is a linear relationship between the independent and dependent variables. A scatter diagram can be used to visually determine this relationship.
In Python, the matplotlib library can be used to generate the scatter diagram. Assume we have independent variables – interest_rate, unemployment_rate, and index_price and a dependent variable – stock_price.
We can generate the scatter diagram for the independent variable-interest_rate and the dependent variable-stock_price as follows:
import pandas as pd
import matplotlib.pyplot as plt
data = pd.read_csv('data.csv')
X = data['interest_rate']
Y = data['stock_price']
plt.scatter(X, Y)
plt.show()
If the resulting plot represents a linear pattern, i.e., the plotted points lie close to a straight line, with a positive or negative slope, then the variables have a linear relationship. However, if the plotted points do not follow a specific linear pattern, then the relationship is non-linear.
Multiple Linear Regression in Python using sklearn
With a clear understanding of the linearity of the variables, implementation of the multiple linear regression can be done via the scikit-learn package, commonly known as sklearn. Sklearn is a simple and powerful module for machine learning.
Its one of the primary packages used for multiple linear regression in Python.
In the following example, we will use multiple linear regression to determine how the stock price is affected by the three independent variables mentioned earlier; interest_rate, unemployment_rate, and index_price.
To prepare the data for the regression, we will create a Pandas DataFrame with the variables of interest, create an instance of the linear_model.LinearRegression() class, fit the data into the model and retrieve the results, including the intercept, coefficients, and the multiple linear regression equation:
from sklearn import linear_model
data = pd.read_csv('data.csv')
X = data[['interest_rate', 'unemployment_rate', 'index_price']]
Y = data['stock_price']
# Create linear regression object
reg = linear_model.LinearRegression()
# Train the model using the training sets
reg.fit(X,Y)
# Retrieve the coefficients or parameters of the model
print('Intercept: ', reg.intercept_)
print('Coefficients: ', reg.coef_)
print('Equation: Y = {:.2f} + {:.2f}*interest_rate + {:.2f}*unemployment_rate + {:.2f}*index_price'.format(reg.intercept_, reg.coef_[0], reg.coef_[1], reg.coef_[2]))
The output includes the intercept and weights(regression coefficients) of the multiple regression equation.
In this case, the multiple regression equation is: Y = 1798.4 – 825.03*interest_rate – 192.72*unemployment_rate + 341.43*index_price. From the equation, the intercept value is 1798.4, implying that when all independent variables are zero, the stock price is predicted to be 1798.4. The coefficients, on the other hand, provide an idea of how each independent variable contributes to the dependent variable.
For instance, a one-unit increase in the interest rate causes the stock price to decrease by 825.03 units, while a unit increase in unemployment_rate leads to a decrease in stock_price by 192.72 units. Conversely, a unit increase in index_price leads to an increase in stock_price by 341.43 units.
Multiple Linear Regression in Python using statsmodels
Statsmodels is another library in Python that can be used to perform multiple linear regression. It works by estimating the coefficients of regression using ordinary least squares (OLS), then providing statistical information about the fit of the model and the standard errors of the coefficients.
To use statsmodels for multiple linear regression, we have to pass the independent variables into the sm.add_constant() function to obtain the intercept parameter. We will also specify the dependent variable within the sm.OLS() function, fit the model and return detailed information about the regression model:
import statsmodels.api as sm
data = pd.read_csv('data.csv')
X = sm.add_constant(data[['interest_rate', 'unemployment_rate', 'index_price']])
Y = data['stock_price']
# Create a regression model
reg = sm.OLS(Y,X).fit()
# Print out the summary report
print(reg.summary())
The output generated includes information about the R-squared value, coefficient estimates, p-value, and the confidence interval for each coefficient.
From the R-squared value obtained, we can see how well the regression model fits the data. If R-squared is close to 1, then the model fits the data well.
Conclusion
Multiple linear regression is an important technique in data analysis as it enables the prediction of a dependent variable from several independent variables. Machine learning powered by Python with its robust libraries is an ideal tool for multiple linear regression.
With the use of the right modules, a detailed breakdown and analysis of the regression model can be achieved, and results interpreted accurately. In conclusion, this article highlighted the importance of multiple linear regression as a tool in data analysis and machine learning, and how it can be performed using Python’s powerful libraries, sklearn, and statsmodels.
The article also emphasized the need to first check for linearity between the independent and dependent variables before performing regression. While sklearn and statsmodels were proven to be great tools for performing multiple linear regression, the approach chosen should depend on the user’s specific needs.
This article has provided a comprehensive guide on how to perform multiple linear regression in Python. Knowing how to perform and interpret multiple linear regression will enable the effective prediction of outcomes, and could prove to be essential in solving real-world problems.