note.nkmk.me

Get/Set the Recursion Limit in Python: sys.getrecursionlimit, sys.setrecursionlimit

Modified: | Tags: Python

In Python, the sys.getrecursionlimit() and sys.setrecursionlimit() functions are used to get and set the recursion limit, which is the maximum recursion depth allowed by the interpreter.

The number of recursions is also limited by the stack size. In some environments, the maximum stack size can be modified using the resource module. In my case, this was successful on Ubuntu but not on Windows or macOS.

Contents

The sample code below was tested on Ubuntu.

Get the recursion limit: sys.getrecursionlimit()

You can get the current value of the recursion limit with sys.getrecursionlimit().

import sys

print(sys.getrecursionlimit())
# 1000
source: sys_recursionlimit.py

The default limit is 1000 in this example, but it may vary depending on the environment.

Consider a simple recursive function that takes a positive integer n as its argument, where the number of recursions equals n.

def recu_test(n):
    if n == 1:
        print('Finish')
        return
    recu_test(n - 1)
source: sys_recursionlimit.py

If n exceeds the limit, a RecursionError occurs.

recu_test(950)
# Finish

# recu_test(1500)
# RecursionError: maximum recursion depth exceeded in comparison
source: sys_recursionlimit.py

It is important to note that the value returned by sys.getrecursionlimit() represents the maximum depth of the Python interpreter stack rather than the absolute maximum number of recursions. As a result, a RecursionError may be raised even if the actual number of recursions is slightly below this limit.

The recursion limit is not the limit on recursion but the maximum depth of the python interpreter stack.
python - Max recursion is not exactly what sys.getrecursionlimit() claims. How come? - Stack Overflow

# recu_test(995)
# RecursionError: maximum recursion depth exceeded while calling a Python object
source: sys_recursionlimit.py

Set the recursion limit: sys.setrecursionlimit()

You can set the recursion limit with sys.setrecursionlimit().

import sys

sys.setrecursionlimit(2000)

print(sys.getrecursionlimit())
# 2000
source: sys_recursionlimit.py

Larger values enable deeper recursion.

recu_test(1500)
# Finish
source: sys_recursionlimit.py

Setting the limit too high or too low may cause errors.

The highest possible limit is platform-dependent. A user may need to set the limit higher when they have a program that requires deep recursion and a platform that supports a higher limit. This should be done with care, because a too-high limit can lead to a crash. If the new limit is too low at the current recursion depth, a RecursionError exception is raised. sys.setrecursionlimit() — Python 3.12.1 documentation

sys.setrecursionlimit(4)
print(sys.getrecursionlimit())
# 4

# sys.setrecursionlimit(3)
# RecursionError: cannot set the recursion limit to 3 at the recursion depth 1: the limit is too low

sys.setrecursionlimit(10 ** 9)
print(sys.getrecursionlimit())
# 1000000000

# sys.setrecursionlimit(10 ** 10)
# OverflowError: signed integer is greater than maximum
source: sys_recursionlimit.py

Additionally, as explained next, the stack size also constrains the maximum recursive limit.

Change the maximum stack size: resource.setrlimit()

Even after increasing the recursion limit, executing a large number of recursions can lead to a segmentation fault.

sys.setrecursionlimit(10 ** 9)
print(sys.getrecursionlimit())
# 1000000000

recu_test(10 ** 4)
# Finish

# recu_test(10 ** 5)
# Segmentation fault
source: sys_recursionlimit.py

In Python, you can change the maximum stack size with the resource module. Note that the resource module is specific to Unix-based systems and is unavailable on Windows.

You can get the limits of the specified resource as (soft_limit, hard_limit) with resource.getrlimit(). resource.RLIMIT_STACK, which represents the maximum size of the call stack of the current process, is specified in the following example.

import resource

print(resource.getrlimit(resource.RLIMIT_STACK))
# (8388608, -1)
source: sys_recursionlimit.py

In this example, the soft limit is 8388608 (8388608 B = 8192 KB = 8 MB), and the hard limit is -1 (unlimited).

You can change the limit of the resource with resource.setrlimit(). The soft limit is set to -1 (unlimited) in the following example. Alternatively, the constant resource.RLIM_INFINITY can be used to represent an unlimited limit.

Setting the soft limit to -1 enables deeper recursion previously limited by segmentation faults.

resource.setrlimit(resource.RLIMIT_STACK, (-1, -1))

print(resource.getrlimit(resource.RLIMIT_STACK))
# (-1, -1)

recu_test(10 ** 5)
# Finish
source: sys_recursionlimit.py

In this example, the soft limit is set to -1 for a simple experiment, but in practice, it is safer to set an appropriate limit.

On macOS, attempting to set an unlimited soft limit results in a ValueError: not allowed to raise maximum limit. Running the script with sudo did not help. It might be a system-imposed limitation.

A process with the effective UID of super-user can request any valid limit value, including unlimited, but ValueError will still be raised if the requested limit exceeds the system imposed limit. resource.setrlimit() — Python 3.12.1 documentation

Related Categories

Related Articles