note.nkmk.me

Filter (extract / remove) items of a list with filter() in Python

Posted: 2021-09-10 / Tags: Python, List

In Python, you can use filter() to filter (extract / remove) items of a list, tuple, or other iterable that satisfy the conditions.

This article describes the following contents.

  • Basic usage of filter()
    • filter() returns an iterator in Python3
    • Convert to a list
    • Extract and remove elements according to conditions
  • Apply lambda expressions (lambda) and functions defined with def
  • Apply multiple conditions
  • Specify None as the first argument
  • Extract False elements: itertools.filterfalse()
  • Use list comprehensions and generator expressions instead

Note that filter() can be substituted by list comprehensions or generator expressions, as described later, and in many cases it is preferable to use them.

For more information on extracting elements of a list using list comprehensions, see the following article.

Sponsored Link

Basic usage of filter()

The first argument of filter() is a callable object such as a function to be applied, and the second argument is an iterable object such as a list. Apply the function to the iterable elements and extract the ones whose result is determined to be True.

filter() returns an iterator in Python3

For example, use the lambda expression that returns True if the value is an even number (= the remainder of the value divided by 2 is 0).

In Python 3, filter() returns an object of type filter which is an iterator, and print() does not output items.

l = [-2, -1, 0, 1, 2]
print(filter(lambda x: x % 2 == 0, l))
# <filter object at 0x10bb38580>

print(type(filter(lambda x: x % 2 == 0, l)))
# <class 'filter'>

The value of the iterator can be retrieved with a for statement.

for i in filter(lambda x: x % 2 == 0, l):
    print(i)
# -2
# 0
# 2

Note that filter() in Python 2 returns a list, so be careful when running Python 2 code in Python 3.

Convert to a list

If you want to convert the result of filter() to a list, use list().

print(list(filter(lambda x: x % 2 == 0, l)))
# [-2, 0, 2]

Extract and remove elements according to conditions

filter() extracts elements whose result is determined to be True by applying the function of the first argument. If you want to remove an element, specify a function whose result is the opposite.

For example, "remove even elements" is equivalent to "extract odd elements".

print(list(filter(lambda x: x % 2 == 0, l)))
# [-2, 0, 2]

print(list(filter(lambda x: x % 2 != 0, l)))
# [-1, 1]

As in the example above, you can replace the comparison operators with ones that produce opposite results (== and ! =, > and <=, etc.), or you can use not for negation.

For example, extract and remove strings ending with 'e'.

l_s = ['apple', 'orange', 'strawberry']
print(list(filter(lambda x: x.endswith('e'), l_s)))
# ['apple', 'orange']

print(list(filter(lambda x: not x.endswith('e'), l_s)))
# ['strawberry']

Contrary to filter(), a function itertools.filterfalse() is also provided to keep elements that are False. It is described later.

Apply lambda expressions (lambda) and functions defined with def

The first argument of filter() is a callable object.

As in the previous examples, lambda expressions (lambda) are often used, but of course it is also possible to specify a function defined with def.

def is_even(x):
    return x % 2 == 0

l = [-2, -1, 0, 1, 2]
print(list(filter(is_even, l)))
# [-2, 0, 2]

Apply multiple conditions

If you want to apply multiple conditions, you can specify a lambda expression / function that connects multiple conditions with and or or.

l = [-2, -1, 0, 1, 2]
print(list(filter(lambda x: x % 2 == 0 and x > 0, l)))
# [2]

print(list(filter(lambda x: x % 2 == 0 or x > 0, l)))
# [-2, 0, 1, 2]
Sponsored Link

Specify None as the first argument

If None is specified as the first argument of filter(), elements which are determined to be True (= truthy elements) are extracted.

l_b = [True, False]
print(list(filter(None, l_b)))
# [True]

Not only True and False, but also numbers, lists, and strings are determined.

For example, in the case of numbers, 0 is regarded as False and the others are regarded as True, in the case of lists and strings, if it is empty, it is regarded as False and the others are regarded as True.

See the following article for details.

Therefore, if you specify None as the first argument of filter(), 0 and empty list / string is removed.

l = [-2, -1, 0, 1, 2]
print(list(filter(None, l)))
# [-2, -1, 1, 2]

l_2d = [[0, 1, 2], [], [3, 4, 5]]
print(list(filter(None, l_2d)))
# [[0, 1, 2], [3, 4, 5]]

l_s = ['apple', '', 'orange', 'strawberry']
print(list(filter(None, l_s)))
# ['apple', 'orange', 'strawberry']

Extract False elements: itertools.filterfalse()

Contrary to filter(), a function itertools.filterfalse() is also provided to keep elements that are False.

Usage is the same as filter(). You need to import itertools.

import itertools

l = [-2, -1, 0, 1, 2]
print(list(itertools.filterfalse(lambda x: x % 2 == 0, l)))
# [-1, 1]

print(list(itertools.filterfalse(lambda x: x % 2 != 0, l)))
# [-2, 0, 2]

l_s = ['apple', 'orange', 'strawberry']
print(list(itertools.filterfalse(lambda x: x.endswith('e'), l_s)))
# ['strawberry']

As mentioned above, the same result can be obtained with filter() by changing the function of the first argument, but in some cases, itertools.filterfalse() can be used to write code whose intention is clearer than using filter() and not.

If the first argument of itertools.filterfalse() is set to None, elements which are determined to be False (= falsy elements) are extracted.

l = [-2, -1, 0, 1, 2]
print(list(itertools.filterfalse(None, l)))
# [0]

Use list comprehensions and generator expressions instead

The equivalent of filter() can also be achieved with list comprehensions and generator expressions.

Note that filter(function, iterable) is equivalent to the generator expression (item for item in iterable if function(item)) if function is not None and (item for item in iterable if item) if function is None. Built-in Functions - filter() — Python 3.9.7 documentation

l = [-2, -1, 0, 1, 2]
print([x for x in l if x % 2 == 0])
# [-2, 0, 2]

print([x for x in l if x % 2 != 0])
# [-1, 1]

l_s = ['apple', 'orange', 'strawberry']
print([x for x in l_s if x.endswith('e')])
# ['apple', 'orange']

print([x for x in l_s if not x.endswith('e')])
# ['strawberry']
l = [-2, -1, 0, 1, 2]
print([x for x in l if x])
# [-2, -1, 1, 2]

l_2d = [[0, 1, 2], [], [3, 4, 5]]
print([x for x in l_2d if x])
# [[0, 1, 2], [3, 4, 5]]

If you want to get a list like list(filter()), use the list comprehensions, and if you want to get an iterator like filter(), use a generator expression.

As shown in the following Stack Overflow question, in most cases, using list comprehensions and generator expressions is preferable to filter() because the code is more concise and clear.

Note that, since processing speed may vary due to various factors, if speed is important to you, it is recommended to measure it under conditions as close to your assumptions as possible.

Sponsored Link
Share

Related Categories

Related Articles