Dropping All Columns Except Specific Ones from a Pandas DataFrame
Data is essential for making informed decisions in various fields like finance, healthcare, and education. It’s one thing to have a lot of data, but it’s another thing to extract actionable insights from that data.
Filtering out specific columns from large datasets is one way to make data analysis more manageable and simplify decision-making. In this article, we’ll be discussing two methods for dropping all columns except specific ones from a Pandas DataFrame.
Method 1: Using Double Brackets
The first method involves using double brackets to select specific columns from a DataFrame. This method is a quick fix for selecting specific columns from a DataFrame and for updating the same DataFrame with only the intended columns.
Here is a simple example;
import pandas as pd
# Creating a sample dataset
data = {'Name': ['John', 'Mary', 'Mark', 'Sara'],
'Age': [20, 30, 25, 26],
'Country': ['USA', 'Canada', 'Mexico', 'Nigeria'],
'Salary': [50000, 60000, 55000, 45000]}
df = pd.DataFrame(data)
# Printing the original DataFrame
print(df)
# Selecting specific columns and updating the DataFrame
df = df[['Name', 'Country']]
# Printing the updated DataFrame
print(df)
In the example above, we created a DataFrame with four columns: ‘Name’, ‘Age’, ‘Country’, and ‘Salary.’ We then selected only the ‘Name’ and ‘Country’ columns using double brackets to create a new DataFrame with these columns. The updated DataFrame now includes only the desired columns.
Method 2: Using .loc
The second method involves using the .loc method to select specific columns from a DataFrame. The .loc method allows for accessing specified rows and columns by label(s) or a Boolean array.
To select specific columns, we pass a list of column names as the second argument of the .loc method as seen in the following example:
import pandas as pd
# Creating a sample dataset
data = {'Name': ['John', 'Mary', 'Mark', 'Sara'],
'Age': [20, 30, 25, 26],
'Country': ['USA', 'Canada', 'Mexico', 'Nigeria'],
'Salary': [50000, 60000, 55000, 45000]}
df = pd.DataFrame(data)
# Printing the original DataFrame
print(df)
# Selecting specific columns and updating the DataFrame
df = df.loc[:, ['Name', 'Country']]
# Printing the updated DataFrame
print(df)
The output of the above code is the same as with Method 1. We selected only the ‘Name’ and ‘Country’ columns, and the updated DataFrame includes only these columns.
Example 1: Dropping All Columns Except Specific Ones Using Double Brackets
To further illustrate the first method, let’s look at an example where we have a large dataset, and we want to extract a few columns from it. Suppose we have a dataset with columns like Name, Age, Country, Email, Education, and Occupation.
We want to create a new dataset with only the Name, Email, and Occupation column. Here’s how we can do it using double brackets:
import pandas as pd
# Reading the dataset
df = pd.read_csv("dataset.csv")
# Dropping all columns except Name, Email, and Occupation
df = df[['Name', 'Email', 'Occupation']]
# Writing the updated dataset to a new file
df.to_csv("updated_dataset.csv", index=False)
In this example, we read the dataset from a CSV file. Then, we used double brackets to select the ‘Name,’ ‘Email,’ and ‘Occupation’ columns and created a new dataset.
Finally, we saved the new dataset to another CSV file.
Example 2: Dropping All Columns Except Specific Ones Using .loc
Now let’s look at another example that illustrates the second method of dropping all columns except specific ones using the .loc method. Suppose we have a dataset with columns like Name, Age, Country, Email, Education and Occupation.
We want to create a new dataset with only the Name, Country, and Education columns. Here’s how we can do it using the .loc method:
import pandas as pd
# Reading the dataset
df = pd.read_csv("dataset.csv")
# Dropping all columns except Name, Country, and Education
df = df.loc[:, ['Name', 'Country', 'Education']]
# Writing the updated dataset to a new file
df.to_csv("updated_dataset.csv", index=False)
In this example, we again read the dataset from a CSV file and selected only the ‘Name,’ ‘Country,’ and ‘Education’ columns using the .loc method. Finally, we saved the new dataset to another CSV file.
Conclusion
Dropping all columns except specific ones from a Pandas DataFrame is essential when dealing with large datasets. We discussed two methods for extracting specific columns from a DataFrame: using double brackets and using the .loc method.
Both methods are simple and efficient for selecting specific columns from a DataFrame. By using these methods, we can easily create updated datasets with the specific pieces of information we need to conduct further analysis.
Additional Resources
Pandas is a powerful library for data manipulation and analysis, and there are many useful resources available online to help you get started. Here are some recommended tutorials for common tasks associated with analyzing DataFrames using Pandas:
-
Python Pandas Tutorial: A Complete for Beginners (https://www.dataquest.io/blog/pandas-python-tutorial/)
This is a comprehensive tutorial that covers all the basics of using Pandas, including loading data, manipulating data, and visualizing data using charts.
-
Pandas Cookbook (https://github.com/jvns/pandas-cookbook)
This is a great resource for intermediate and advanced Pandas users. It provides recipes for common data manipulation tasks using Pandas, along with detailed explanations of the code.
-
Real Python Pandas Tutorials (https://realpython.com/learning-paths/pandas-data-science/)
This is a series of tutorials that cover a range of topics related to using Pandas for data analysis.
The tutorials include step-by-step instructions and code examples for each topic. In conclusion, dropping all columns except specific ones from a Pandas DataFrame is a common task in data analysis.
The two methods we discussed, using double brackets and using the .loc method, are simple and efficient ways to accomplish this task. In addition, there are many resources available online that can help you to learn more about using Pandas for data manipulation and analysis.
Whether you’re a beginner or an experienced user, these resources can help you to improve your skills and make more informed decisions based on your data. In conclusion, extracting specific columns from large datasets is a crucial task for making informed decisions based on data analysis.
In this article, we discussed two methods for dropping all columns except specific ones from a Pandas DataFrame: using double brackets and using the .loc method. Both methods are simple and efficient and can be used to create updated datasets with only the relevant information needed for further analysis.
Moreover, we recommended some tutorials and resources to help readers learn more about using Pandas for data manipulation. Overall, being able to extract specific columns from a dataset is an essential skill to have in data analysis, and the tips and resources provided in this article can be useful for anyone looking to improve their knowledge in this area.