note.nkmk.me

How to fix "ValueError: The truth value ... is ambiguous" in NumPy, pandas

Modified: | Tags: Python, NumPy, pandas, Error handling

In NumPy and pandas, using numpy.ndarray, pandas.DataFrame, or pandas.Series in conditions or with and or or operations may raise an error.

ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
ValueError: The truth value of a DataFrame is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all().
ValueError: The truth value of a Series is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all().

This article explains the causes of this error and how to fix it.

Contents

To avoid errors, remember these two key points:

  1. Use &, |, ~ instead of and, or, not.
    • and, or, not check if the object itself is True or False.
    • &, |, ~ perform element-wise Boolean/bitwise operations.
  2. When combining multiple conditions, enclose each condition in parentheses ().
    • & and | have higher precedence than comparison operators (such as <).

These concepts are the same for numpy.ndarray, pandas.DataFrame, and pandas.Series.

The sample code used in this article is based on NumPy version 1.25.1 and pandas version 2.0.3. Please note that behavior may vary with different versions.

import numpy as np

print(np.__version__)
# 1.25.1
source: numpy_error_ambiguous.py
import pandas as pd

print(pd.__version__)
# 2.0.3
source: pandas_error_ambiguous.py

Examples of "ValueError: The truth value of ... is ambiguous"

When using a numpy.ndarray of bool in conditions or with and, or, and not operations, you might encounter an error.

a_bool = np.array([True, True, True])
b_bool = np.array([True, False, False])

# if a_bool:
#     pass
# ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

# a_bool and b_bool
# ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

# a_bool or b_bool
# ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

# not b_bool
# ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
source: numpy_error_ambiguous.py

The cases of pandas.DataFrame and pandas.Series will be discussed in later sections.

Root cause of "ValueError: The truth value of ... is ambiguous"

In Python, objects and expressions are evaluated as bool values (True, False) in conditions and and, or, not operations.

For example, an empty list is evaluated as False, while a non-empty list is evaluated as True.

print(bool([0, 1, 2]))
# True

print(bool([]))
# False

print(not [0, 1, 2])
# False

print(not [])
# True
source: numpy_error_ambiguous.py

Evaluating numpy.ndarray as a bool value raises an error.

a_bool = np.array([True, True, True])

# bool(a_bool)
# ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
source: numpy_error_ambiguous.py

As the word "ambiguous" suggests, it is unclear whether the goal is to check if the object itself is True or False, or to evaluate each element individually as True or False.

Use all(), any(), size

If you want to check if the object itself is True or False, use all() or any() as suggested in the error message.

all() returns True if all elements are True, while any() returns True if at least one element is True.

a_bool = np.array([True, True, True])
b_bool = np.array([True, False, False])

print(a_bool.all())
# True

print(a_bool.any())
# True

print(b_bool.all())
# False

print(b_bool.any())
# True
source: numpy_error_ambiguous.py

Additionally, you can use the size attribute to get the total number of elements and check if the numpy.ndarray is empty.

print(a_bool.size)
# 3

print(a_bool.size == 0)
# False
source: numpy_error_ambiguous.py

Use &, |, ~ instead of and, or, not

Use &, |, ~ for element-wise Boolean/bitwise operations

If you want to perform element-wise AND, OR, NOT operations, use &, |, ~ instead of and, or, not. ^ (XOR) is also available.

For numpy.ndarray containing bool values, the &, |, ~, and ^ operators perform element-wise AND, OR, NOT, and XOR operations, respectively.

a_bool = np.array([True, True, True])
b_bool = np.array([True, False, False])

print(a_bool & b_bool)
# [ True False False]

print(a_bool | b_bool)
# [ True  True  True]

print(~b_bool)
# [False  True  True]

print(a_bool ^ b_bool)
# [False  True  True]
source: numpy_error_ambiguous.py

Note that &, |, ~, and ^ are used for bitwise operations on integer values in Python.

For numpy.ndarray containing integer (int) values, the &, |, ~, and ^ operators perform element-wise bitwise operations.

a_int = np.array([0, 1, 3])  # [0b00 0b01 0b11]
b_int = np.array([1, 0, 2])  # [0b01 0b00 0b10]

print(a_int & b_int)
# [0 0 2]

print(a_int | b_int)
# [1 1 3]

print(a_int ^ b_int)
# [1 1 1]

print(~a_int)
# [-1 -2 -4]
source: numpy_error_ambiguous.py

It is easy to confuse and, or, not with &, |, ~.

and, or, not check if the object itself is True or False. On the other hand, & and | are used for bitwise operations for integer values and element-wise operations for numpy.ndarray as described above, and set operations for set.

Remember that the English terms and and or are typically used in conditions like if A and B:, while the symbols & and | are used in other mathematical operations.

Parentheses are required for multiple conditions

A comparison operation on numpy.ndarray returns a numpy.ndarray of bool values, which can be combined using & and | for element-wise AND and OR operations.

a = np.arange(12).reshape(3, 4)
print(a)
# [[ 0  1  2  3]
#  [ 4  5  6  7]
#  [ 8  9 10 11]]

print(a > 3)
# [[False False False False]
#  [ True  True  True  True]
#  [ True  True  True  True]]

print(a % 2 == 0)
# [[ True False  True False]
#  [ True False  True False]
#  [ True False  True False]]
source: numpy_error_ambiguous.py

As mentioned above, to calculate AND or OR for each element of these numpy.ndarray, use & or | instead of and or or. When combining multiple conditions with & or |, it is necessary to enclose each condition in parentheses ().

# print(a > 3 & a % 2 == 0)
# ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
source: numpy_error_ambiguous.py

This is because & and | have higher precedence than comparison operators (such as <).

The above example would be operated as follows.

# print(a > (3 & (a % 2)) == 0)
# ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
source: numpy_error_ambiguous.py

Each condition must be enclosed in parentheses ().

print((a > 3) & (a % 2 == 0))
# [[False False False False]
#  [ True False  True False]
#  [ True False  True False]]
source: numpy_error_ambiguous.py

Note that comparison operations on many objects other than numpy.ndarray return True or False.

x = 10

print(x > 3)
# True

print(x % 2 == 1)
# False
source: numpy_error_ambiguous.py

and and or are used for Boolean operations involving True and False values. Since and and or have lower precedence than comparison operators (such as <), there is no error without parentheses in this case. Of course, parentheses are also acceptable.

print(x > 3 or x % 2 == 1)
# True

print((x > 3) or (x % 2 == 1))
# True
source: numpy_error_ambiguous.py

For arrays with 1 or 0 elements

If the number of elements is one or zero, no error is raised, as indicated by the error message "more than one element".

ValueError: The truth value of an array with more than one element is ambiguous.

If the number of elements is one, its value is evaluated as a bool value. For example, if the element is an integer int, it is False if it is 0 and True otherwise.

a_single = np.array([0])
b_single = np.array([1])
c_single = np.array([2])

print(bool(a_single))
# False

print(bool(b_single))
# True

print(bool(c_single))
# True
source: numpy_error_ambiguous.py

and and or return either the left or right side objects instead of True or False.

print(b_single and c_single)
# [2]

print(c_single and b_single)
# [1]

print(b_single or c_single)
# [1]

print(c_single or b_single)
# [2]
source: numpy_error_ambiguous.py

& and | return element-wise AND and OR.

print(b_single & c_single)
# [0]

print(b_single | c_single)
# [3]
source: numpy_error_ambiguous.py

not returns element-wise NOT. ~ returns element-wise ~ (for signed integers, ~x returns -(x + 1)).

print(not a_single)
# True

print(not b_single)
# False

print(not c_single)
# False

print(~a_single)
# [-1]

print(~b_single)
# [-2]

print(~c_single)
# [-3]
source: numpy_error_ambiguous.py

If there are no elements, a DeprecationWarning is issued.

a_empty = np.array([])
print(a_empty)
# []

print(bool(a_empty))
# False
# 
# /var/folders/rf/b7l8_vgj5mdgvghn_326rn_c0000gn/T/ipykernel_40648/256863747.py:1: DeprecationWarning: The truth value of an empty array is ambiguous. Returning False, but in future this will result in an error. Use `array.size > 0` to check that an array is not empty.
#   print(bool(a_empty))
source: numpy_error_ambiguous.py

The warning indicates that an error will be raised in future versions (the example above uses version 1.25.1). Therefore, it is advisable to use the size attribute as suggested in the message.

Fix "ValueError: The truth value of ... is ambiguous" for pandas.DataFrame

For pandas.DataFrame, similar to numpy.ndarray, use & or | for element-wise operations and enclose multiple conditions in parentheses ().

df = pd.DataFrame(np.arange(12).reshape(3, 4), columns=['a', 'b', 'c', 'd'], index=['x', 'y', 'z'])
print(df)
#    a  b   c   d
# x  0  1   2   3
# y  4  5   6   7
# z  8  9  10  11

print((df > 3) & (df % 2 == 0))
#        a      b      c      d
# x  False  False  False  False
# y   True  False   True  False
# z   True  False   True  False
source: pandas_error_ambiguous.py

Errors are raised if you use and/or or omit parentheses ().

# print((df > 3) and (df % 2 == 0))
# ValueError: The truth value of a DataFrame is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all().

# print(df > 3 & df % 2 == 0)
# ValueError: The truth value of a DataFrame is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all().
source: pandas_error_ambiguous.py

Note that the all() and any() methods of pandas.DataFrame default to axis=0 (column-wise), unlike numpy.ndarray. If you want to cover all elements, use axis=None.

print(df > 3)
#        a      b      c      d
# x  False  False  False  False
# y   True   True   True   True
# z   True   True   True   True

print((df > 3).all())
# a    False
# b    False
# c    False
# d    False
# dtype: bool

print((df > 3).all(axis=None))
# False
source: pandas_error_ambiguous.py

The empty and size attributes are also provided.

print(df)
#    a  b   c   d
# x  0  1   2   3
# y  4  5   6   7
# z  8  9  10  11

print(df.empty)
# False

print(df.size)
# 12
source: pandas_error_ambiguous.py
df_empty = pd.DataFrame()
print(df_empty)
# Empty DataFrame
# Columns: []
# Index: []

print(df_empty.empty)
# True

print(df_empty.size)
# 0
source: pandas_error_ambiguous.py

Fix "ValueError: The truth value of ... is ambiguous" for pandas.Series

For pandas.Series, similar to numpy.ndarray and pandas.DataFrame, use & or | for element-wise operations and enclose multiple conditions in parentheses ().

pandas.Series of bool is used to select rows according to conditions.

df = pd.read_csv('data/src/sample_pandas_normal.csv')
print(df)
#       name  age state  point
# 0    Alice   24    NY     64
# 1      Bob   42    CA     92
# 2  Charlie   18    CA     70
# 3     Dave   68    TX     70
# 4    Ellen   24    CA     88
# 5    Frank   30    NY     57
source: pandas_multiple_conditions.py
print(df['age'] < 35)
# 0     True
# 1    False
# 2     True
# 3    False
# 4     True
# 5     True
# Name: age, dtype: bool

print(~(df['state'] == 'NY'))
# 0    False
# 1     True
# 2     True
# 3     True
# 4     True
# 5    False
# Name: state, dtype: bool

print((df['age'] < 35) & ~(df['state'] == 'NY'))
# 0    False
# 1    False
# 2     True
# 3    False
# 4     True
# 5    False
# dtype: bool

df_and = df[(df['age'] < 35) & ~(df['state'] == 'NY')]
print(df_and)
#       name  age state  point
# 2  Charlie   18    CA     70
# 4    Ellen   24    CA     88
source: pandas_multiple_conditions.py

Related Categories

Related Articles