Pandas DataFrame Selection: How to Access Top and Bottom Rows
As data analysis continues to grow in popularity, using pandas dataframes has become a standard for many data scientists and analysts worldwide. To maximize the potential of our data, we need to know how to access specific parts of our dataframe.
This article outlines essential techniques for selecting top and bottom rows from a pandas dataframe.
How to use DataFrame.head() function
In cases where we are working with massive datasets, DataFrame.head() is an excellent function that allows us to take a sneak peek at the first few rows.
The DataFrame.head() function selects the first few rows of the dataframe and is particularly useful when working with dataframes containing numerous rows. It facilitates viewing a manageable section of the data for easy analysis.
To use DataFrame.head(), we need to call this function on the dataframe. The function can also take a parameter n indicating the number of rows to select from the top.
Here’s an example of how to call the function, assuming we want to select the first five rows of our dataframe:
import pandas as pd
df = pd.read_csv("data.csv")
# Select top 5 rows
df.head(5)
Select top n rows in pandas DataFrame
If we prefer to select a specific number of rows from the dataframe, we need to pass the integer value n in the head() function. For instance, if we want only the first ten rows from our dataframe, we’ll do this:
import pandas as pd
df = pd.read_csv("data.csv")
# Select top 10 rows
df.head(10)
Select top rows except for last n rows
If we want to select all the top rows except the last n rows, we can make use of the negative parameter. For example, if we intend to select all the rows from the top except the last three rows:
import pandas as pd
df = pd.read_csv("data.csv")
# Select all rows except last 3 rows
df.head(-3)
Select top rows from multi-index DataFrames
When dealing with dataframes with multiple indices, we can use the head() function to select the top rows. For example:
import pandas as pd
df = pd.read_csv("data.csv")
multi_indexed_df = df.set_index(['Name','Age'])
# Select top 2 rows from multi-indexed dataframe
multi_indexed_df.head(2)
How to use DataFrame.tail() function
This function allows us to access the last few rows of our dataframe. Similar to DataFrame.head(), DataFrame.tail() helps us avoid viewing lots of data simultaneously.
Viewing only a smaller section of the data makes it easier to analyse. To use the DataFrame.tail() function, we need to call it as follows:
import pandas as pd
df = pd.read_csv("data.csv")
# Select bottom 5 rows
df.tail(5)
Select bottom n rows in pandas DataFrame
If we want to select the last n rows from our dataframe, we can apply the tail() function. For instance:
import pandas as pd
df = pd.read_csv("data.csv")
# Select bottom 10 rows
df.tail(10)
Select bottom rows except for first n rows
Like with the head() function, we can also use the negative parameter, in this case, to exclude the first n rows. For instance:
import pandas as pd
df = pd.read_csv("data.csv")
# Select all bottom rows except the first 3 rows
df.tail(-3)
Select bottom rows from the multi-index DataFrame
When working with dataframes with multiple indices, we can use the tail() function to select the last few rows. Let’s see an example below:
import pandas as pd
df = pd.read_csv("data.csv")
multi_indexed_df = df.set_index(['Name','Age'])
# Select bottom 2 rows from multi-indexed dataframe
multi_indexed_df.tail(2)
DataFrame Value Selection
The pandas DataFrame.at[] and DataFrame.iat[] selectors help retrieve data using row labels and column names and indexes, respectively.
Select value using row and column labels using DataFrame.at
In cases where we want to select any particular value using specific row and column labels, DataFrame.at[] function is the best approach.
Let’s see an example of how this can be achieved:
import pandas as pd
df = pd.read_csv("data.csv")
value = df.at[1, 'Name']
print(value)
Set specific value in pandas DataFrame
We use the same at[] selector to update any value in the dataframe. Let’s see an example:
import pandas as pd
df = pd.read_csv("data.csv")
# update value at specific row and column index
df.at[1, 'Name'] = 'John'
# Output updated DataFrame
print(df)
Select value using row and column position using DataFrame.iat
The iat[] selector helps us select data at specific positions defined by row and column integers. Here’s an example of how to use this function:
import pandas as pd
df = pd.read_csv("data.csv")
value = df.iat[1, 2]
print(value)
Set specific value in pandas DataFrame
Like with the at[] selector, the iat[] selector helps update any selected value in a dataframe.
import pandas as pd
df = pd.read_csv("data.csv")
# Updates value in the first row and second column
df.iat[0,1] = 'new_value'
# Print DataFrame after updating a value
print(df)
Conclusion
This tutorial explored different techniques for selecting top and bottom rows of data from pandas dataframes and accessing specific values using row and column labels and integers.
All these techniques play an essential role in preparing and cleaning our data before we proceed with visualisations, analysis, and machine learning models.
Practising these techniques regularly can help streamline our data preparation process. In summary, this article discussed essential techniques for selecting top and bottom rows from a Pandas dataframe, as well as accessing specific values using row and column labels and integers.
It is crucial to understand these techniques to clean and prepare our data effectively before proceeding with analysis or machine learning models. Remembering to implement these steps regularly will help streamline data preparation processes and maximize the potential of our data.
With these techniques, data scientists and analysts can dive deep into the details of their massive datasets and access specific values easily. Implementing these techniques can make data analysis and management a breeze.