The ThreadPoolExecutor in Python creates internal worker threads.
In this tutorial, you will discover how to check the names of threads created in Python thread pools.
Let’s get started.
Main Thread Name
When you run a Python program, you will create one Python process with one thread.
Each Python program is created with one thread to execute the instructions, called the “main thread.”
We can access the current thread context via the current_thread() function in the threading module.
You can learn more about the threading module here:
The current_thread() function returns a Thread instance for the current working thread, on which we can access details like the (typically unique) name of the thread via the name property.
For example, we can report the name of the current thread in our Python program (the main thread) as follows:
|
1 2 3 |
... # report the thread print(f'Main thread: {current_thread().name}') |
Each thread also has an identifier.
Python will assign each thread an integer identifier number. This can be retrieved via the get_ident() function. The operating system will also assign each thread a unique integer that can be retrieved by the get_native_id() function.
Threads may have similar or identical names, but the Python ID and the native ID will (or should) be unique.
We can report a threads name, ID, and native ID in one line, as follows:
|
1 2 3 4 |
... # report the thread details thread = current_thread() print(f'Main thread: name={thread.name}, idnet={get_ident()}, id={get_native_id()}') |
Tying this together, the complete example of checking the details of the main thread in a Python program is as follows.
|
1 2 3 4 5 6 7 8 9 10 11 |
# SuperFastPython.com # check the name and id of the main thread from threading import current_thread from threading import get_ident from threading import get_native_id # entry point if __name__ == '__main__': # report the thread details thread = current_thread() print(f'Main thread: name={thread.name}, idnet={get_ident()}, id={get_native_id()}') |
Running the example reports the process ID and thread name for our program.
In this case, we can see that the name of the main thread was MainThread. It has a Python identifier of 7195014 and a native identifier of 4723641856.
Note: you will have a different identifier number than the one listed below, and likely a different number each time the program is run. This is because the numbers are allocated by your operating system and Python respectively at runtime.
|
1 |
Main thread: name=MainThread, idnet=4723641856, id=7195014 |
Run loops using all CPUs, download your FREE book to learn how.
Worker Thread Names
The ThreadPoolExecutor will create worker threads, by design.
We can define a target task function that will report the name and identifier of the thread it is running in.
We would expect each worker thread to have a separate native and Python ID from the main thread and from each other.
|
1 2 3 4 5 6 |
# target task function def work(value): sleep(0.1) # worker thread details thread = current_thread() print(f'Worker thread: name={thread.name}, idnet={get_ident()}, id={get_native_id()}') |
We can then create a thread pool with two worker threads and use map() on the thread pool to submit two tasks to report their details.
|
1 2 3 4 5 |
... # create a process pool with ThreadPoolExecutor(2) as executor: # submit some tasks _ = executor.map(work, range(2)) |
Tying this together, the complete example of reporting the details of worker processes in the ThreadPoolExecutor is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 |
# SuperFastPython.com # report details of all worker threads in the thread pool from time import sleep from threading import current_thread from threading import get_ident from threading import get_native_id from concurrent.futures import ThreadPoolExecutor # target task function def work(value): sleep(0.1) # worker thread details thread = current_thread() print(f'Worker thread: name={thread.name}, idnet={get_ident()}, id={get_native_id()}') # entry point if __name__ == '__main__': # main thread details thread = current_thread() print(f'Main thread: name={thread.name}, idnet={get_ident()}, id={get_native_id()}') # create a thread pool with ThreadPoolExecutor(2) as executor: # submit some tasks _ = executor.map(work, range(2)) |
Running the example first reports the details of the main thread.
We then report the details for each worker thread.
We can see that each worker has a different native ID and Python, as we expected, and that both have a unique thread name.
The name uses a format of “ThreadPoolExecutor-%d-%d” by default, where the first number indicates the thread number within the process and the second number indicates the worker thread number within the thread pool.
This can be helpful for debugging and diagnostic processes; for example, each worker thread can log messages that include its name.
|
1 2 3 |
Main thread: name=MainThread, idnet=4556213760, id=7196592 Worker thread: name=ThreadPoolExecutor-0_1, idnet=123145337270272, id=7196597 Worker thread: name=ThreadPoolExecutor-0_0, idnet=123145320480768, id=7196596 |
Worker Threads Run Callbacks
One question we might consider is what thread runs callback functions.
This is important to know if we need to access a resource shared between threads from within the callback function. It will determine any locks and cross-process data sharing mechanisms that may be needed.
When we submit tasks to the thread pool via the submit() function, we receive a Future object in return.
The Future object can be used to check the status of the task, get the result, and to add a callback function to call once the task is done. This can be achieved by calling the add_done_callback() function in the future and specifying the callback function to call.
The callback function must take a single argument which is the Future object for the task that is done.
We can define a callback function that reports the thread details to learn about the thread that is executing the callback.
|
1 2 3 4 5 |
# callback function to call when a task is completed def custom_callback(future): # callback thread details thread = current_thread() print(f'Callback thread: name={thread.name}, idnet={get_ident()}, id={get_native_id()}') |
We can define a target task function that reports the same details; for example:
|
1 2 3 4 5 6 |
# target task function def work(): sleep(0.1) # worker thread details thread = current_thread() print(f'Worker thread: name={thread.name}, idnet={get_ident()}, id={get_native_id()}') |
In our main thread, we can first report the details of the thread.
|
1 2 3 4 |
... # main thread details thread = current_thread() print(f'Main thread: name={thread.name}, idnet={get_ident()}, id={get_native_id()}') |
Then, we can create the thread pool, submit a single task in order to get a Future object, then register the callback function.
|
1 2 3 4 5 6 7 |
... # create a thread pool with ThreadPoolExecutor(2) as executor: # submit a task future = executor.submit(work) # add a callback function future.add_done_callback(custom_callback) |
Tying this together, the complete example of checking the details of the thread that executes callbacks for the ThreadPoolExecutor is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 |
# SuperFastPython.com # check the thread used to execute future callbacks from time import sleep from threading import current_thread from threading import get_ident from threading import get_native_id from concurrent.futures import ThreadPoolExecutor # target task function def work(): sleep(0.1) # worker thread details thread = current_thread() print(f'Worker thread: name={thread.name}, idnet={get_ident()}, id={get_native_id()}') # callback function to call when a task is completed def custom_callback(future): # callback thread details thread = current_thread() print(f'Callback thread: name={thread.name}, idnet={get_ident()}, id={get_native_id()}') # entry point if __name__ == '__main__': # main thread details thread = current_thread() print(f'Main thread: name={thread.name}, idnet={get_ident()}, id={get_native_id()}') # create a thread pool with ThreadPoolExecutor(2) as executor: # submit a task future = executor.submit(work) # add a callback function future.add_done_callback(custom_callback) |
Running the example, we can first see the details of the main thread as before.
We can then see that the worker thread has a different name and identifiers, again, as we expected from the previous example.
Finally, the details of the thread that executed the callback function are reported.
In this case, we can see that the worker thread in the thread pool that executed the task is the same thread that executes the callback function.
|
1 2 3 |
Main thread: name=MainThread, idnet=4391812608, id=7198790 Worker thread: name=ThreadPoolExecutor-0_0, idnet=123145467486208, id=7198794 Callback thread: name=ThreadPoolExecutor-0_0, idnet=123145467486208, id=7198794 |
This means that if the callback function modifies shared data, e.g. a counter or data structure accessed by call callbacks, then it must be protected; it is a critical section.
This is because worker threads may execute concurrently and, in turn, call callback functions concurrently, potentially creating a race condition for any shared data modified in the callback function.
This can be achieved using an instance of the threading.Lock class.
First, it can be defined in the main program.
|
1 2 3 |
... # a lock for protecting critical sections in callbacks lock = Lock() |
This lock can then be accessed and used via the context manager within the callback function.
For example:
|
1 2 3 4 5 6 |
# custom callback function def custom_callback(future): global lock # protect critical section from race conditions with lock: # ... |
Free Python ThreadPoolExecutor Course
Download your FREE ThreadPoolExecutor PDF cheat sheet and get BONUS access to my free 7-day crash course on the ThreadPoolExecutor API.
Discover how to use the ThreadPoolExecutor class including how to configure the number of workers and how to execute tasks asynchronously.
Further Reading
This section provides additional resources that you may find helpful.
Books
- ThreadPoolExecutor Jump-Start, Jason Brownlee, (my book!)
- Concurrent Futures API Interview Questions
- ThreadPoolExecutor Class API Cheat Sheet
I also recommend specific chapters from the following books:
- Effective Python, Brett Slatkin, 2019.
- See Chapter 7: Concurrency and Parallelism
- Python in a Nutshell, Alex Martelli, et al., 2017.
- See: Chapter: 14: Threads and Processes
Guides
- Python ThreadPoolExecutor: The Complete Guide
- Python ProcessPoolExecutor: The Complete Guide
- Python Threading: The Complete Guide
- Python ThreadPool: The Complete Guide
APIs
References
Overwhelmed by the python concurrency APIs?
Find relief, download my FREE Python Concurrency Mind Maps
Takeaways
You now know how to check the thread names in the ThreadPoolExecutor.
Do you have any questions?
Ask your questions in the comments below and I will do my best to answer.
Photo by Matt Saling on Unsplash
Leave a Reply