pandas: Check if DataFrame/Series is empty
You can use the empty attribute to check if a pandas.DataFrame or pandas.Series is empty.
- pandas.DataFrame.empty — pandas 2.0.3 documentation
- pandas.Series.empty — pandas 2.0.3 documentation
The sample code in this article uses pandas version 2.0.3.
import pandas as pd
print(pd.__version__)
# 2.0.3
How to use the empty attribute in pandas.DataFrame and pandas.Series
pandas.DataFrame.empty
The empty attribute returns True for an empty DataFrame and False otherwise.
df_empty = pd.DataFrame()
print(df_empty)
# Empty DataFrame
# Columns: []
# Index: []
print(df_empty.empty)
# True
df_full = pd.DataFrame({'A': [0, 1, 2]})
print(df_full)
# A
# 0 0
# 1 1
# 2 2
print(df_full.empty)
# False
Note that NaN is treated as a valid value. Therefore, even if all elements are NaN, empty returns False.
df_nan = pd.DataFrame({'A': [float('NaN'), float('NaN'), float('NaN')]})
print(df_nan)
# A
# 0 NaN
# 1 NaN
# 2 NaN
print(df_nan.empty)
# False
To determine whether all elements are NaN, first apply the dropna() method to remove NaN and then use empty.
print(df_nan.dropna())
# Empty DataFrame
# Columns: [A]
# Index: []
print(df_nan.dropna().empty)
# True
By default, dropna() removes rows with at least one NaN. To remove rows where all elements are NaN, use how='all'.
df_mix = pd.DataFrame({'A': [float('NaN'), float('NaN'), float('NaN')], 'B': [0, 1, 2]})
print(df_mix)
# A B
# 0 NaN 0
# 1 NaN 1
# 2 NaN 2
print(df_mix.dropna())
# Empty DataFrame
# Columns: [A, B]
# Index: []
print(df_mix.dropna(how='all'))
# A B
# 0 NaN 0
# 1 NaN 1
# 2 NaN 2
pandas.Series.empty
The empty attribute of Series works the same way as DataFrame.
s_empty = pd.Series()
print(s_empty)
# Series([], dtype: object)
print(s_empty.empty)
# True
s_full = pd.Series([0, 1, 2])
print(s_full)
# 0 0
# 1 1
# 2 2
# dtype: int64
print(s_full.empty)
# False
s_nan = pd.Series([float('NaN'), float('NaN'), float('NaN')])
print(s_nan)
# 0 NaN
# 1 NaN
# 2 NaN
# dtype: float64
print(s_nan.empty)
# False
print(s_nan.dropna())
# Series([], dtype: float64)
print(s_nan.dropna().empty)
# True
Note: An empty pandas.DataFrame or pandas.Series is not considered False
In Python, empty lists, tuples, dictionaries, and similar structures are evaluated as False.
l_empty = []
print(bool(l_empty))
# False
For example, in if statements, you can directly use the object to check if a list or similar data structure is empty.
if not l_empty:
print('Empty!')
# Empty!
However, an empty DataFrame or Series is not considered False. Therefore, you must explicitly use the empty attribute to check for emptiness.
df_empty = pd.DataFrame()
print(df_empty)
# Empty DataFrame
# Columns: []
# Index: []
# print(bool(df_empty))
# ValueError: The truth value of a DataFrame is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all().
print(df_empty.empty)
# True
# if df_empty:
# print('Empty!')
# ValueError: The truth value of a DataFrame is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all().
if df_empty.empty:
print('Empty!')
# Empty!
For more details on the error The truth value of a DataFrame is ambiguous, refer to the following article.
Example: Check if any rows or elements meet a condition
You can extract rows from a DataFrame that satisfy a certain condition using boolean indexing with comparison operators or the query() method.
If no row meets the condition, an empty DataFrame is returned. You can then check this result with the empty attribute to confirm whether any rows meet the condition.
df = pd.DataFrame({'A': [0, 1, 2], 'B': ['aaa', 'bbb', 'ccc']})
print(df)
# A B
# 0 0 aaa
# 1 1 bbb
# 2 2 ccc
print(df[df['A'] < 0])
# Empty DataFrame
# Columns: [A, B]
# Index: []
print(df[df['A'] < 0].empty)
# True
print(df.query('A < 0'))
# Empty DataFrame
# Columns: [A, B]
# Index: []
print(df.query('A < 0').empty)
# True