You can configure thread names for ThreadPoolExecutor in Python by setting the “thread_name_prefix” argument.
In this tutorial, you will discover how to configure thread names when using the Python thread pool.
Let’s get started.
Need to Configure Thread Names With the ThreadPoolExecutor
The ThreadPoolExecutor in Python provides a pool of reusable threads for executing ad hoc tasks.
You can submit tasks to the thread pool by calling the submit() function and passing in the name of the function you wish to execute on another thread.
Calling the submit() function will return a Future object that allows you to check on the status of the task and get the result from the task once it completes.
You can also submit tasks by calling the map() function and specifying the name of the function to execute and the iterable of items to which your function will be applied.
Each thread in Python has a name.
Thread names are unique by default.
This is by design to be helpful when debugging a program with multiple threads. Log messages can report the thread that is performing a specific step or a debugging can be used to trace a thread with a specific name.
All threads used in the thread pool have unique thread names, which we may want to configure in our application aid in logging and debugging.
How can we set the name of threads in the ThreadPoolExecutor?
Run loops using all CPUs, download your FREE book to learn how.
How to Set Thread Names in the ThreadPoolExecutor
You can configure the name of threads when configuring the ThreadPoolExecutor when calling the constructor.
When creating threads in the thread pool, each thread has the name "ThreadPoolExecutor-%d_%d" where the first %d indicates the thread pool number and the second %d indicates the thread number, both in the order that thread pools and threads are created.
Both thread pools and threads in each pool start their numbering at zero.
For example, if we had a thread pool with five threads in one thread pool, the names of the threads would be:
- ThreadPoolExecutor-0_0
- ThreadPoolExecutor-0_1
- ThreadPoolExecutor-0_2
- ThreadPoolExecutor-0_3
- ThreadPoolExecutor-0_4
The "ThreadPoolExecutor-%d" is a prefix for all threads in the thread pool and we can customize it with a name that may be meaningful in the application for the types of tasks executed by the pool.
The thread name prefix can be set via the thread_name_prefix argument when constructing the thread pool.
For example, we can set the thread name prefix to be 'WorkerThreadPool' as follows:
|
1 2 3 |
... # create the thread pool executor = ThreadPoolExecutor(num_workers=5, thread_name_prefix='WorkerThreadPool') |
The five threads in the pool would then have the names:
- WorkerThreadPool_0
- WorkerThreadPool_1
- WorkerThreadPool_2
- WorkerThreadPool_3
- WorkerThreadPool_4
It is a good idea to set a thread name prefix that is meaningful to the application and the types of tasks that will be executed by the thread pool.
It is also a good idea to name each thread pool used in your application so that you can tell the threads apart clearly when debugging and/or when logging using custom thread names.
Now that we know how to set the thread name prefix for threads in the ThreadPoolExecutor, let’s look at a worked example.
Example of Configuring Thread Names in the ThreadPoolExecutor
Let’s look at a worked example of setting the thread name prefix for the ThreadPoolExecutor.
We can start off by simply accessing and reporting the names of threads.
The main thread in Python has the name "MainThread". You can access the main thread via a call to the threading.main_thread() function and then access the name member.
For example:
|
1 2 3 4 5 6 7 |
# SuperFastPython.com # access the name of the main thread from threading import main_thread # access the main thread thread = main_thread() # report the thread name print(thread.name) |
Running the example accesses the main thread and reports its name,
|
1 |
MainThread |
We can also access and report the names of the threads within the ThreadPoolExecutor.
This can be achieved by first creating a thread pool and submitting some tasks, then enumerating all threads in a Python program (process) via the threading.enumerate() function, then reporting the name for each.
The example below creates a ThreadPoolExecutor with the default number of threads, allocates work to the pool to ensure the threads are created, then reports the names of all threads in the program.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
# SuperFastPython.com # report the default name of threads in the thread pool import threading from concurrent.futures import ThreadPoolExecutor # a mock task that does nothing def task(name): pass # create a thread pool executor = ThreadPoolExecutor() # execute asks executor.map(task, range(10)) # report all thread names for thread in threading.enumerate(): print(thread.name) # shutdown the thread pool executor.shutdown() |
Running the example reports the names of all threads in the system, showing first the name of the main thread and the name of four threads in the pool.
In this case, only four threads were created as the tasks were executed so quickly.
Recall that worker threads are used after they finish executing their tasks. This ability to reuse workers is a major benefit of using thread pools.
|
1 2 3 4 5 |
MainThread ThreadPoolExecutor-0_0 ThreadPoolExecutor-0_1 ThreadPoolExecutor-0_2 ThreadPoolExecutor-0_3 |
Next, we can configure the thread name prefix, which will customize the names of all threads in the ThreadPoolExecutor.
The example below sets the thread_name_prefix to be "TaskPool", which is prepended to the name of each thread created in the pool.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
# SuperFastPython.com # set a custom thread name prefix for all threads in the pool import threading from concurrent.futures import ThreadPoolExecutor # a mock task that does nothing def task(name): pass # create a thread pool with a custom name prefix executor = ThreadPoolExecutor(thread_name_prefix='TaskPool') # execute asks executor.map(task, range(10)) # report all thread names for thread in threading.enumerate(): print(thread.name) # shutdown the thread pool executor.shutdown() |
Running the example reports the name of the main thread as before but in this case, it includes the names of threads in the thread pool with the custom thread name prefix.
|
1 2 3 4 5 |
MainThread TaskPool_0 TaskPool_1 TaskPool_2 TaskPool_3 |
Free Python ThreadPoolExecutor Course
Download your FREE ThreadPoolExecutor PDF cheat sheet and get BONUS access to my free 7-day crash course on the ThreadPoolExecutor API.
Discover how to use the ThreadPoolExecutor class including how to configure the number of workers and how to execute tasks asynchronously.
Further Reading
This section provides additional resources that you may find helpful.
Books
- ThreadPoolExecutor Jump-Start, Jason Brownlee, (my book!)
- Concurrent Futures API Interview Questions
- ThreadPoolExecutor Class API Cheat Sheet
I also recommend specific chapters from the following books:
- Effective Python, Brett Slatkin, 2019.
- See Chapter 7: Concurrency and Parallelism
- Python in a Nutshell, Alex Martelli, et al., 2017.
- See: Chapter: 14: Threads and Processes
Guides
- Python ThreadPoolExecutor: The Complete Guide
- Python ProcessPoolExecutor: The Complete Guide
- Python Threading: The Complete Guide
- Python ThreadPool: The Complete Guide
APIs
References
Overwhelmed by the python concurrency APIs?
Find relief, download my FREE Python Concurrency Mind Maps
Takeaways
You now know how to set thread names when using the ThreadPoolExecutor in python.
Do you have any questions about how to set thread names?
Ask your question in the comments below and I will do my best to answer.
Photo by Razvan Chisu on Unsplash
Leave a Reply