Setting Column Widths in Jupyter Notebooks with Pandas
As a data analyst or scientist, one of the most common data manipulation tasks you’ll undertake is to format and display data in a tabular format. Pandas, a popular data manipulation library in Python, provides powerful tools for formatting and visualizing data.
However, by default, Pandas can be restrictive, especially when it comes to displaying data columns. By default, Pandas sets a maximum column width, which can be limiting when dealing with long strings or data that requires wider columns.
Fortunately, Pandas provides a range of options for customizing column widths in Jupyter Notebooks. In this article, we’ll explore some of these methods in-depth.
Default Column Width in Pandas
The default column width in Pandas is set to display only a subset of the data in each column. Pandas does this by cutting off the rest of the data and replacing it with an ellipsis (…).
While this is acceptable for most data, it can be frustrating when dealing with long strings or data that requires wider columns. To change the maximum width for columns, you can use the ‘set_option’ method in Pandas by setting the ‘display.max_colwidth’ parameter.
This parameter takes an integer value that represents the number of characters to be displayed in the column. For example, to set the maximum column width to 100, we can run the following code in our Jupyter Notebook:
import pandas as pd
pd.set_option('display.max_colwidth', 100)
The above code sets the maximum column width to 100 for the entire Jupyter notebook session.
Forcing Maximum Column Width in Pandas DataFrame
While setting the maximum column width for the entire session is useful, it can be limiting when working with large datasets that require different column widths. In such cases, you can force Pandas to display the entire column width for specific columns in a DataFrame by using the ‘set_option’ method.
This method overrides the maximum column width setting for specified columns. The syntax to display the entire column width in Pandas is as follows:
pd.set_option('display.max_colwidth', None)
Using the ‘None’ argument sets the maximum column width to ‘None’ and forces Pandas to display the entire column width for specified columns.
Temporarily Displaying an Entire Column Width
Another way to display the entire column width is to use the ‘style.format’ method in Pandas. This method allows you to format specific columns temporarily and display the entire width.
The syntax to display the entire column width using the ‘style.format’ method is:
df.style.set_properties(subset=[COLUMN_NAME], **{'width': '300px'})
The above code sets the column width for COLUMN_NAME to 300 pixels.
Resetting Default Column Width Settings in a Jupyter Notebook
If you have changed the maximum column width setting in your Pandas session and need to reset it to the default, you can use the ‘reset_option’ method. This method resets all Pandas settings to their default state.
The syntax to reset the default column width settings in Jupyter Notebook is:
pd.reset_option('display.max_colwidth')
Example: Set Column Widths in Pandas
To demonstrate how to set column widths in Pandas, let’s look at an example. Suppose we have a Pandas DataFrame called ‘example_df’ with a column ‘long_string’ that contains long strings.
# create example dataframe
example_df = pd.DataFrame({'id': [1, 2, 3, 4], 'long_string': ['Vestibulum dolor nulla, eleifend nec velit eget, blandit facilisis sapien. Phasellus et elementum odio, et imperdiet ante.', 'Curabitur eu elit luctus, vehicula justo a, dapibus lorem. Ut ornare sed quam quis accumsan.', 'Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed pretium leo non fringilla porttitor.', 'In id aliquam neque. Praesent id bibendum mi. Duis vel blandit nisi.']})
# view dataframe
example_df
The above code creates a DataFrame with a column ‘long_string’ that contains long strings. We can view the DataFrame to see how Pandas cuts off the string data:
id long_string
0 1 Vestibulum dolor nulla, eleifend nec velit eget, blandit facilisis sapien. Phasellus et elementum ...
1 2 Curabitur eu elit luctus, vehicula justo a, dapibus lorem. Ut ornare sed quam quis acc...
2 3 Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed pretium leo non fringilla porttitor.
3 4 In id aliquam neque. Praesent id bibendum mi. Duis vel blandit nisi.
As you can see, Pandas cuts off the long strings and replaces the rest of the data with an ellipsis.
To display the entire column width for ‘long_string,’ we can use the ‘style.format’ method:
example_df.style.set_properties(subset=['long_string'], **{'width': '500px'})
The above code sets the column width for ‘long_string’ to 500 pixels. In conclusion, setting column widths in Jupyter Notebooks with Pandas is an essential skill for data analysts and scientists.
Understanding how to change the maximum column width for the entire session, force Pandas to display the entire width for specified columns, and temporarily display the entire column width is crucial for manipulating and presenting data effectively. With the methods outlined in this article, you’ll be well-equipped to manipulate, format, and visualize data in your Jupyter Notebooks like a pro.
In addition to managing column widths, Pandas provides several powerful tools for data manipulation and analysis. To help you get started, we’ve put together a list of some of the most common operations in Pandas and links to tutorials covering these topics.
Performing Other Common Operations in Pandas
1. Merging DataFrames:
Merging datasets in Pandas is a common operation when working with data from multiple sources. The Pandas ‘merge’ function allows you to merge datasets based on one or more key columns. To learn more about merging datasets in Pandas, check out this tutorial: https://www.analyticsvidhya.com/blog/2020/02/joins-in-pandas-master-the-different-types-of-joins-in-python/
2. Grouping Data:
Grouping data in Pandas involves splitting data into groups, applying a function on each group, and combining the results. The ‘groupby’ function in Pandas allows you to group data based on one or more columns. To learn more about grouping data in Pandas, check out this tutorial: https://www.datacamp.com/community/tutorials/pandas-split-apply-combine-groupby
3. Aggregating Data:
Aggregating data in Pandas involves performing a function on a dataset that returns a summary statistic or a single value. The Pandas ‘agg’ function allows you to apply one or more aggregation functions on a dataset. To learn more about aggregating data in Pandas, check out this tutorial: https://realpython.com/pandas-aggregate-agg/
4. Reshaping Data:
Reshaping data in Pandas involves transforming data from one shape to another. The Pandas ‘pivot_table’ and ‘melt’ functions allow you to reshape data into different formats. To learn more about reshaping data in Pandas, check out this tutorial: https://www.datacamp.com/community/tutorials/pandas-pivot-tables
5. Cleaning Data:
Cleaning data in Pandas involves identifying and correcting or removing errors and anomalies in a dataset. The Pandas ‘fillna’ and ‘dropna’ functions allow you to fill in missing data or drop rows or columns containing missing data. To learn more about cleaning data in Pandas, check out this tutorial: https://www.datacamp.com/community/tutorials/cleaning-data-python
6. Visualizing Data:
Visualizing data in Pandas involves creating visual representations of data, such as scatter plots, line charts, and histograms. The ‘plot’ function in Pandas allows you to create many types of visualizations. To learn more about visualizing data in Pandas, check out this tutorial: https://pandas.pydata.org/pandas-docs/stable/user_guide/visualization.html
7. Feature Engineering:
Feature engineering involves creating new features from existing data that can improve the performance of machine learning models. The Pandas ‘apply’ function allows you to apply a function to each row or column of a dataset. To learn more about feature engineering in Pandas, check out this tutorial: https://towardsdatascience.com/feature-engineering-with-pandas-24b26cd8b988
In conclusion, Pandas provides several powerful tools for manipulating and analyzing data. Whether you’re merging datasets, grouping data, aggregating data, reshaping data, cleaning data, visualizing data, or carrying out feature engineering, Pandas has got you covered. The tutorials listed above will give you a deeper understanding of these common operations in Pandas.
In this article, we explored setting column widths in Jupyter Notebooks with Pandas and discussed additional common operations. We learned that Pandas sets a default column width and how to force Pandas to display the entire column width for specific columns.
Additionally, we covered the significance of merging datasets, grouping data, aggregating data, reshaping data, cleaning data, visualizing data, and feature engineering with Pandas, and provided links to tutorials covering these topics. These tools help data analysts and scientists manipulate and present data effectively.
By mastering these techniques, you will be well-equipped to analyze and visualize data efficiently and effectively.