You can retrieve and change the thread stack size via the threading.stack_size() function.
In this tutorial you will discover how to configure the thread stack size in Python.
Let’s get started.
What is a Thread Stack Size
Python code is compiled at runtime into Python bytecode and executed in the Python virtual machine.
Python source code is compiled into bytecode, the internal representation of a Python program in the CPython interpreter. […] This “intermediate language” is said to run on a virtual machine that executes the machine code corresponding to each bytecode.
— bytecode, Python Glossary.
The Python bytecodes are executed using a stack data structure that adds instructions as functions are called and pops instructions as they are executed, e.g. a so-called stack machine.
Each Python thread will have its own stack of instructions that are maintained and executed.
The size of the stack is fixed and is pre-allocated by the Python interpreter for each thread. This is so that the management and execution of instructions is fast.
The size of the stack may limit the number of instructions that can be maintained by a thread, such as the depth of a tree of function calls and the local variables maintained in those function calls. This is particularly an issue when a thread executes a single function recursively.
It is hard to nail down exact details of the Python thread stack and stack size details in the CPython interpreter, but a good place to start is here:
— Python Data model
Now that we know what the Python thread stack size is, let’s look at why we may change it.
Run loops using all CPUs, download your FREE book to learn how.
Why Change Thread Stack Size
Python will use a default thread stack size that is specific to your underlying operating system.
Note that some platforms may have particular restrictions on values for the stack size, such as requiring a minimum stack size > 32 KiB …
— threading — Thread-based parallelism
We may want to change the Python thread stack size in our program.
For example, we may have a large number of very simple worker threads that execute a single function with few local variables. In that case, we may want to set the thread stack size to the minimum value.
More likely, we may have one or more worker threads executing complex function call graphs. This might involve many small functions with many local variables and perhaps some functions called recursively.
In that case, we may want to increase the thread stack size. This may result in better performance for the threads given the stack size is pre-allocated, and may avoid stability problems if the interpreter imposes a hard limit on the size of a thread stack.
A limitation of a small thread stack size is that a program that requires a large thread stack size may result in a memory error resulting running out of space on the thread stack, called a stack overflow.
Now that we have some ideas on why we might adjust the stack size, let’s look at how we might do this.
How To Get the Thread Stack Size
The thread stack size can be retrieved using the threading.stack_size() function.
For example:
|
1 2 3 |
... # get the stack size size = threading.sstack_size() |
Note, the threading.stack_size() function is a synonym or alias for the _thread.stack_size() function.
Calling the threading.stack_size() function will return the stack size used by the Python interpreter when creating new threads.
Return the thread stack size used when creating new threads.
— threading — Thread-based parallelism
The function takes an argument named “size” which is optional. If not specified, it will be set to 0, which will cause the function to return the default thread stack size.
If size is not specified, 0 is used.
— threading — Thread-based parallelism
The example below shows how to retrieve the default thread stack size.
|
1 2 3 4 5 6 |
# SuperFastPython.com # example of getting the thread stack size from threading import stack_size # get the default stack size size = stack_size() print(size) |
Running the example retrieves and reports the default thread stack size.
In this case, we can see that the value of zero is returned and reported, which indicates that the default for the operating system is being used.
|
1 |
Free Python Threading Course
Download your FREE threading PDF cheat sheet and get BONUS access to my free 7-day crash course on the threading API.
Discover how to use the Python threading module including how to create and start new threads and how to use a mutex locks and semaphores
How To Set Thread Stack Size
The thread stack size can be set via the threading.stack_size() function, the same function for getting the thread stack size.
The optional size argument specifies the stack size to be used for subsequently created threads, and must be 0 (use platform or configured default) or a positive integer value of at least 32,768 (32 KiB).
— threading — Thread-based parallelism
The function takes a single argument named “size” that specifies the new thread stack size in bytes.
For example:
|
1 2 3 |
... # set the stack size old_size = threading.stack_size(32768) |
Note, the threading.stack_size() function is a synonym or alias for the _thread.stack_size() function.
When setting a new thread stack size, the function will return the old (current) thread stack size.
A subsequent call to threading.stack_size() with no argument will return the current value of the thread stack, reporting the new value.
The thread stack size must be specified before new threads are created for those threads to take on the new value.
The minimum value for the “size” argument is 32 kilobytes (kb) which is 32,768 bytes.
32 KiB is currently the minimum supported stack size value to guarantee sufficient stack space for the interpreter itself.
— threading — Thread-based parallelism
Most operating systems impose a constraint on the value of the “size” argument that it must be a multiple of 4,096 bytes.
If an unsupported size value is specified, a ValueError may be raised.
If the specified stack size is invalid, a ValueError is raised and the stack size is unmodified. […] using multiples of 4096 for the stack size is the suggested approach in the absence of more specific information
— threading — Thread-based parallelism
Some operating systems do not permit the thread stack size to be modified, in which case a RuntimeError may be raised.
If changing the thread stack size is unsupported, a RuntimeError is raised.
— threading — Thread-based parallelism
Now that we know how to set the thread stack size, let’s look at some worked examples.
Overwhelmed by the python concurrency APIs?
Find relief, download my FREE Python Concurrency Mind Maps
Examples of Setting the Thread Stack Size
In this section we will look at some examples of setting the thread stack size.
Set the Minimum Thread Stack Size
The minimum stack size is 32 kilobytes (kb) which is 32,768 bytes.
We can set the minimum thread stack size as follows:
|
1 2 3 |
... # set the minimum thread stack size stack_size(32768) |
The example below demonstrates how to set the minimum thread stack size.
Once set, the current thread stack size is retrieved and reported, confirming that the change took effect.
|
1 2 3 4 5 6 7 8 |
# SuperFastPython.com # example of setting the minimum thread stack size from threading import stack_size # set the minimum thread stack size stack_size(32768) # report the changed stack size size = stack_size() print(size) |
Running the example first sets the stack size to 32 kilobytes.
The stack size is then retrieved, confirming the new value took effect.
|
1 |
32768 |
Increase the Thread Stack Size
The thread stack size can be increased by specifying a value that is a multiple of 4,096 bytes and larger than 32 kilobytes (kb) which is 32,768 bytes.
Common values might include
- 4,096 * 8 = 32,768 bytes
- 4,096 * 16 = 65,536 bytes
- 4,096 * 32 = 131,072 bytes
- 4,096 * 64 = 262,144 bytes
- 4,096 * 128 = 524,288 bytes
For example:
|
1 2 3 |
... # increase the stack size stack_size(4096 * 16) |
Choosing a suitable value for your specific application might require some trial and error with careful benchmarking to confirm the change had the desired improvement in performance and/or stability.
The example below shows how to increase the thread stack size to 64 kilobytes, or 65,536 bytes.
|
1 2 3 4 5 6 7 8 |
# SuperFastPython.com # example of increasing the thread stack size from threading import stack_size # increase the stack size stack_size(4096 * 16) # report the changed stack size size = stack_size() print(size) |
Running the example first sets the increased thread size to 64 kilobytes.
The stack size is then retrieved, confirming the new value took effect.
|
1 |
65536 |
Set Invalid Thread Stack Size
Changing the thread stack size to an unsupported value will result in a ValueError.
Valid thread stack size values are multiples of 4,096 bytes and above 32,768 bytes.
For example, changing the thread stack size to 100,000 bytes will result in a ValueError on most platforms.
The example below demonstrates the effect of setting an invalid thread stack size.
|
1 2 3 4 5 |
# SuperFastPython.com # example of setting an invalid thread stack size from threading import stack_size # set an invalid stack size stack_size(100000) # will raise ValueError |
Running the example results in a ValueError.
This is expected, given that the specified thread stack size of 100,000 is not a multiple of 4,096 bytes.
|
1 2 3 4 |
Traceback (most recent call last): ... stack_size(100000) # will raise ValueError ValueError: size not valid: 100000 bytes |
Further Reading
This section provides additional resources that you may find helpful.
Python Threading Books
- Python Threading Jump-Start, Jason Brownlee (my book!)
- Threading API Interview Questions
- Threading Module API Cheat Sheet
I also recommend specific chapters in the following books:
- Python Cookbook, David Beazley and Brian Jones, 2013.
- See: Chapter 12: Concurrency
- Effective Python, Brett Slatkin, 2019.
- See: Chapter 7: Concurrency and Parallelism
- Python in a Nutshell, Alex Martelli, et al., 2017.
- See: Chapter: 14: Threads and Processes
Guides
- Python Threading: The Complete Guide
- Python ThreadPoolExecutor: The Complete Guide
- Python ThreadPool: The Complete Guide
APIs
References
Takeaways
You now know how to retrieve and configure the thread stack size in Python.
Do you have any questions?
Ask your questions in the comments below and I will do my best to answer.
Photo by Loïc Lassence on Unsplash
Leave a Reply