Introduction to Principal Components Analysis (PCA)
Principal Components Analysis (PCA) is an unsupervised machine learning technique used to identify patterns in data that are not easily noticeable through visual inspection. It achieves this by transforming the data into a set of new variables known as principal components.
Each principal component represents a linear combination of the original variables. PCA is a powerful tool that can help researchers gain a deeper understanding of the relationships between different variables in the data.
By analyzing the variation explained by each principal component, researchers can identify which variables have the most significant impact on the overall variance of the data. In this article, we will walk you through the process of preparing a dataset for PCA, and explain the importance of understanding variation explained by each principal component.
Preparing the Dataset for PCA
Importing the USArrests dataset and defining columns to use for PCA
To demonstrate the process of preparing a dataset for PCA, we will use the USArrests dataset, which contains data on crime rates in different states of the United States. The first step is to import the dataset and define the columns we want to use for PCA.
In this case, we will use all four variables: Murder, Assault, UrbanPop, and Rape. The code to import the dataset and define the columns would look like this:
“` python
import pandas as pd
usarrests = pd.read_csv(‘USArrests.csv’)
cols_to_use = [‘Murder’, ‘Assault’, ‘UrbanPop’, ‘Rape’]
data = usarrests[cols_to_use]
“`
Creating a scaled version of the dataset using StandardScaler
The next step is to create a scaled version of the dataset using StandardScaler. StandardScaler scales each variable in the dataset to have a mean of 0 and a standard deviation of 1.
Scaling the variables is important because PCA is sensitive to the relative magnitude of the variables. If we do not scale the variables, a variable with a larger magnitude will have a more significant impact on the principal components than a variable with a smaller magnitude, even if the smaller variable is more important in the data.
The code to scale the dataset using StandardScaler would look like this:
“` python
from sklearn.preprocessing import StandardScaler
scaler = StandardScaler()
scaled_data = pd.DataFrame(scaler.fit_transform(data), columns=data.columns)
“`
Conclusion
In this article, we have discussed the basics of Principal Components Analysis (PCA) as an unsupervised machine learning technique. We have also explained the importance of understanding the variation explained by each principal component.
Additionally, we have demonstrated the process of preparing a dataset for PCA by importing the USArrests dataset and creating a scaled version of the dataset using StandardScaler. By using PCA, researchers can reduce the dimensionality of the data without losing too much information.
They can identify the variables that have the most significant impact on the overall variance of the data and gain insights into the relationships between different variables. PCA is a powerful tool that can help researchers make more informed decisions based on data analysis.
3) Performing PCA
Once the dataset is prepared and scaled, we can now perform PCA. In this article, we will use the PCA() function from the sklearn package to perform PCA on the USArrests dataset.
The PCA() function requires us to specify the number of principal components we want to extract. In many cases, we may not know how many principal components we want to extract, but there is a method we can use to determine the number of components.
This method is called the scree plot. To extract the principal components using PCA(), the code would look like this:
“` python
from sklearn.decomposition import PCA
pca = PCA(n_components=4)
principal_components = pca.fit_transform(scaled_data)
“`
In this case, we have specified the number of components as 4.
This means that we want to extract 4 principal components from the data.
4) Creating a Scree Plot
To determine the optimal number of principal components to use, we can create a scree plot. A scree plot is a line plot that shows the percentage of total variance explained by each principal component.
To calculate the percentage of total variance explained by each principal component, we can use the explained_variance_ratio_ attribute of the PCA object. This attribute returns an array of values representing the percentage of variance explained by each component.
“` python
import matplotlib.pyplot as plt
percent_variance = pca.explained_variance_ratio_
scree_plot_data = {‘PC{}’.format(i): [percent_variance[i-1]*100] for i in range(1,len(percent_variance)+1)}
scree_plot_df = pd.DataFrame(data=scree_plot_data)
scree_plot_df.plot(kind=’bar’)
plt.ylabel(‘Percent Variance Explained’)
plt.xlabel(‘Principal Component’)
plt.show()
“`
The resulting scree plot displays the percentage of variance explained by each principal component on the y-axis, and the principal component values on the x-axis. The plot helps us visualize how much variance each principal component contributes relative to the others.
From the scree plot, we can observe the elbow point on the plot, representing the point at which adding more principal components does not result in a significant increase in variance explained. In this case, it appears that the optimal number of principal components is two.
Using just two components instead of all four could improve the performance of models that use this data, as it would reduce the dimensionality of the data while still retaining most of the information.
Conclusion
In this article, we have seen how the sklearn package can be used to perform PCA on a dataset. We have also learned about the scree plot and how it can be used to determine the optimal number of principal components to use.
By using PCA and visualizing the results using a scree plot, researchers can gain valuable insights into the data and make more informed decisions. PCA and scree plots are powerful tools that can help us extract meaningful information from high-dimensional datasets, and are an essential part of any data analysis toolkit.
5) Interpretation of Scree Plot Results
After creating a scree plot, it is essential to understand how to interpret the results. The scree plot helps us determine the appropriate number of principal components to use in the analysis.
Here are some key points to keep in mind when interpreting the scree plot:
Explanation of what the percentage of variance explained by each principal component means
Each principal component derived from PCA explains a specific amount of variation in the original dataset. The sum of the variances explained by all the principal components is equal to the total variance of the data.
The percentage of variance explained by a principal component is obtained by dividing the variance explained by that principal component by the total variance of the data and converting it to a percentage. For example, if a principal component explains 20% of the total variance of the data, it means that 20% of the total variability in the data can be explained by that principal component.
The remaining 80% of the variability is due to other factors not captured by that principal component. The percentage of variance explained by a principal component is a crucial piece of information in interpreting the scree plot.
It helps us determine the relative importance of each principal component in the dataset.
Displaying exact percentage of variance explained by each principal component
To determine the exact percentage of variance explained by each principal component, we can use the explained_variance_ratio_ attribute of the PCA object. As mentioned earlier, this attribute returns an array of values representing the percentage of variance explained by each component.
“` python
percent_variance = pca.explained_variance_ratio_
for i in range(len(percent_variance)):
print(‘PC{} explains {:.2f}% of variance in the data’.format(i+1, percent_variance[i]*100))
“`
This code snippet will display the exact percentage of variance explained by each principal component in the data.
Interpreting the Scree Plot
After creating a scree plot and determining the optimal number of principal components, we need to consider how to interpret each principal component. Each principal component represents a linear combination of the original variables.
Thus, it is essential to look at the loadings of each variable on each principal component to interpret their individual meanings. Loadings represent the correlation between each variable and the principal component.
Loadings can be positive or negative, indicating the direction and strength of the correlation. To calculate the loading of each variable on each principal component, we can use the components_ attribute of the PCA object.
This attribute returns a matrix with dimensions (n_components, n_features) containing the loadings. “` python
loadings = pd.DataFrame(pca.components_.T * np.sqrt(pca.explained_variance_), columns=[‘PC{}’.format(i) for i in range(1,5)], index=data.columns)
print(loadings)
“`
This code will display the loadings of each variable on each principal component. Interpretations of the loadings will be specific to the problem being studied.
Conclusion
In this article, we have seen how to interpret the results of a scree plot by understanding the percentage of variance explained by each principal component. We have also seen how to display the exact percentage of variance explained by each principal component.
By using the loadings, we gain a deeper understanding of the relationship between the original variables and each principal component. This information can be used to make more informed decisions, develop hypotheses for further analysis, and support the development of predictive models.
The scree plot, combined with the loadings, is a powerful tool for interpreting the results of PCA and gaining insights into high-dimensional datasets. In conclusion, Principal Component Analysis (PCA) is a powerful unsupervised machine learning technique that helps to identify patterns in high-dimensional datasets.
To prepare a dataset for PCA, it’s important to import the dataset, define the columns to use, and scale the variables. Creating a scree plot helps to determine the optimal number of principal components to use and interpret each principal component’s loadings.
Proper interpretation and use of PCA can lead to more informed decision-making, hypothesis development, and improved predictive models. By understanding the percentage of variance explained by each principal component and the loadings of each variable, researchers gain deeper insights into relationships within their data.
For any data analyst or researcher, PCA is a powerful tool and understanding its use can improve the quality of their results.