Last Updated on September 12, 2022
You can get thread deadlocks when using the ThreadPoolExecutor in Python.
In this tutorial, you will discover how to identify deadlocks when using thread pools in Python.
Let’s get started.
ThreadPoolExecutor Deadlocks
The ThreadPoolExecutor is a flexible and powerful thread pool for executing ad hoc tasks in an asynchronous manner.
Nevertheless, it is possible to get deadlocks when using the ThreadPoolExecutor.
A deadlock refers to a failure condition in multithreaded programs when one thread is waiting for a result from another thread, and the other thread is waiting on a result from the first thread.
The result is that neither thread can progress and the program will halt until it is externally terminated.
If threads in a ThreadPoolExecutor are stuck in a deadlock, it means that the thread pool itself cannot be shutdown and will impact the main thread. This is especially the case if the context manager for the ThreadPoolExecutor is used, meaning that the shutdown() function of the thread pool will be called automatically and will wait forever for the deadlocked threads to complete.
In this tutorial, we will look at three examples of deadlocks when using the ThreadPoolExecutor. They are:
- Deadlock 1: Submit and Wait for a Task Within a Task
- Deadlock 2: Task A Waits on Task B; Task B Waits on Task A
- Deadlock 3: Task Waits on Its Own Result
Deadlocks are caused in the ThreadPoolExecutor by one task waiting on one or more other tasks.
Reviewing how deadlocks are possible when using the ThreadPoolExecutor will help you both identify and avoid deadlocks in your own programs.
Note: these examples are inspired by the examples provided in the ThreadPoolExecutor API.
Can you think of any other examples of deadlocks with the threadpoolexecutor?
Let me know in the comments; I’d love to see what you discover.
Next, let’s take a look at the first case of a deadlock.
Run loops using all CPUs, download your FREE book to learn how.
Deadlock 1: Submit and Wait for a Task Within a Task
Submitting tasks to the ThreadPoolExecutor from target task functions can lead to a deadlock.
Consider the situation where we have very few worker threads or all threads are fully occupied.
Submitting a new task in this circumstance will mean the task will wait on the queue for execution, and will not execute until a worker thread becomes available. Any threads waiting on the result of the task will in turn wait.
If a worker thread is the thread that submitted the new task and is waiting for the result, then this will occupy a worker thread. The worker thread will never free-up to take the new task off the queue and we will be in a deadlock.
We can make this clear with an extreme version of this situation where the thread pool has a single worker thread; the worker thread enqueues a task in the thread pool and waits for the result. The worker thread will wait forever because it is occupying the single thread in the pool that is required to execute the tasks that it has submitted.
We can demonstrate this case with a worked example.
First, we can define a target task function named task1() that takes the ThreadPoolExecutor instance, submits a task to the pool, and waits for the result.
|
1 2 3 4 5 6 7 8 |
# define a task def task1(executor): sleep(0.5) # submit a task future = executor.submit(task2) # wait for the result print('Worker in task1 for task2...') return future.result() # deadlock |
We can then define a second task named task2() that is called from the first task. This second task doesn’t do anything special.
|
1 2 3 |
# define a task def task2(): return 'Hi here' |
Finally, we can define our thread pool and submit the first task.
Key to this example is that the thread pool only has a single worker thread, forcing the deadlock situation.
This simulates the situation where worker threads are fully occupied with long running tasks.
|
1 2 3 4 5 6 7 8 |
... # create the thread pool with ThreadPoolExecutor(1) as executor: # submit a task future = executor.submit(task1, executor) print('Main thread waiting for task1...') result = future.result() print(result) |
Tying this together, the complete example of a thread deadlock in the ThreadPoolExecutor caused by a task submitting and waiting on a task is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 |
# SuperFastPython.com # example of a deadlock caused by waiting for a task within a task from time import sleep from concurrent.futures import ThreadPoolExecutor # define a task def task1(executor): sleep(0.5) # submit a task future = executor.submit(task2) # wait for the result print('Worker in task1 for task2...') return future.result() # deadlock # define a task def task2(): return 'Hi here' # create the thread pool with ThreadPoolExecutor(1) as executor: # submit a task future = executor.submit(task1, executor) print('Main thread waiting for task1...') result = future.result() print(result) |
Running the example first submits and waits for the results from task1(), which in turn submits and waits for the results of task2().
The limited resources available in the thread pool results in a deadlock and task1 will wait forever, in turn causing the main thread to wait forever.
The program will need to be forcibly killed.
|
1 2 |
Main thread waiting for task1... Worker in task1 for task2... |
Increasing the thread pool so that there is additional capacity will avoid the deadlock in this specific situation, but not in the general case.
You can avoid this deadlock by never waiting on the results of tasks executed by the thread pool within tasks executed by the thread pool.
Further, it may be a good practice to only submit tasks from one thread, e.g. the main thread, and to not expose the thread pool itself to target task functions.
Deadlock 2: Task A Waits on Task B; Task B Waits on Task A
You can get a deadlock if two tasks wait on each other.
For example, two tasks are submitted to the thread pool for asynchronous execution. The first task has a reference to the second task and waits on a result, and the second task has a reference to the first task and waits on a result.
In this case, neither task can ever progress and both are stuck in a deadlock.
This may seem obvious, in the simplest case, but may be less obvious if tasks executed by the thread pool directly interact and wait on the results from other tasks.
Let’s make this case concrete with a worked example.
First, we can define task1() that will sleep for a moment, then attempt to access the result from a Future object, which happens to be for task1().
|
1 2 3 4 5 6 |
# define a task def task1(): sleep(0.1) print('Task1 waiting on Task2...') result = future2.result() # deadlock return 2 + result |
Next, we can define task2() that will also sleep for a moment, then attempt to access the result from a Future object; in this case, it happens to be for task1().
|
1 2 3 4 5 6 |
# define a task def task2(): sleep(0.2) print('Task2 waiting on Task1...') result = future1.result() # deadlock return 10 + result |
Finally, we can create the thread pool and submit the two tasks.
|
1 2 3 4 5 6 |
# create the thread pool with ThreadPoolExecutor(2) as executor: # submit a task future1 = executor.submit(task1) future2 = executor.submit(task2) print('Main thread waiting for tasks 1 and 2...') |
As the tasks wait on each other, neither will ever progress and will be stuck in a deadlock.
Additionally, the context manager for the ThreadPoolExecutor will wait for all running threads to complete before exiting. As the two threads in the deadlock will never complete, the thread pool will wait forever to close, and the block for the context manager will never exit.
Tying this together, the complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 |
# SuperFastPython.com # example of a deadlock caused by waiting tasks waiting on each other from time import sleep from concurrent.futures import ThreadPoolExecutor # define a task def task1(): sleep(0.1) print('Task1 waiting on Task2...') result = future2.result() # deadlock return 2 + result # define a task def task2(): sleep(0.2) print('Task2 waiting on Task1...') result = future1.result() # deadlock return 10 + result # create the thread pool with ThreadPoolExecutor(2) as executor: # submit a task future1 = executor.submit(task1) future2 = executor.submit(task2) print('Main thread waiting for tasks 1 and 2...') |
Running the example creates the thread pool and submits task1() and task2().
As the tasks wait for the result from each other, they wait forever and the program is stuck in a deadlock.
|
1 2 3 |
Main thread waiting for tasks 1 and 2... Task1 waiting on Task2... Task2 waiting on Task1... |
You can avoid this deadlock by not accessing the results from tasks within tasks executed by the ThreadPoolExecutor. That is, do not pass around or access Future objects within target task functions.
Free Python ThreadPoolExecutor Course
Download your FREE ThreadPoolExecutor PDF cheat sheet and get BONUS access to my free 7-day crash course on the ThreadPoolExecutor API.
Discover how to use the ThreadPoolExecutor class including how to configure the number of workers and how to execute tasks asynchronously.
Deadlock 3: Task Waits on Its Own Result
You can get a deadlock if a target task function waits on its own result.
That is, a target task function can attempt to retrieve a result from a Future object, which just so happens to be the Future object that represents the task that is executing.
This might seem obvious and easily preventable, but can happen in situations where target task functions access Future objects for tasks in the same thread pool.
Let’s explore an example.
First, we can define a task that attempts to retrieve a result from a Future object.
|
1 2 3 4 5 6 7 |
# define a task def task1(): sleep(0.5) # attempt to get a result from itself print('Worker in task1 for task1...') value = future.result() # deadlock return 99 |
Next, we can define the ThreadPoolExecutor and submit the task.
This defines a Future object that the task will attempt to access and, in turn, cause the deadlock.
|
1 2 3 4 5 |
# create the thread pool with ThreadPoolExecutor(1) as executor: # submit a task future = executor.submit(task1) print('Main thread waiting for task1...') |
Tying this together, the complete example of a deadlock caused by a task waiting on its own result in the ThreadPoolExecutor is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
# SuperFastPython.com # example of a deadlock caused by a task waiting on itself from time import sleep from concurrent.futures import ThreadPoolExecutor # define a task def task1(): sleep(0.5) # attempt to get a result from itself print('Worker in task1 for task1...') value = future.result() # deadlock return 99 # create the thread pool with ThreadPoolExecutor(1) as executor: # submit a task future = executor.submit(task1) print('Main thread waiting for task1...') |
Running the example first creates the thread pool, then submits the task for execution.
The task accesses a Future object, which happens to be for itself in the thread pool, and waits for a result.
This triggers a deadlock, meaning the thread will wait forever.
In turn, the context manager will close the thread pool and wait for all running threads to complete. As one thread is stuck in a deadlock, the thread pool will never shut down.
|
1 2 |
Main thread waiting for task1... Worker in task1 for task1... |
This deadlock could be avoided by rearranging the code so that tasks in the thread pool do not access the Future objects directly, and instead receive the results from future objects as arguments.
Overwhelmed by the python concurrency APIs?
Find relief, download my FREE Python Concurrency Mind Maps
Further Reading
This section provides additional resources that you may find helpful.
Books
- ThreadPoolExecutor Jump-Start, Jason Brownlee, (my book!)
- Concurrent Futures API Interview Questions
- ThreadPoolExecutor Class API Cheat Sheet
I also recommend specific chapters from the following books:
- Effective Python, Brett Slatkin, 2019.
- See Chapter 7: Concurrency and Parallelism
- Python in a Nutshell, Alex Martelli, et al., 2017.
- See: Chapter: 14: Threads and Processes
Guides
- Python ThreadPoolExecutor: The Complete Guide
- Python ProcessPoolExecutor: The Complete Guide
- Python Threading: The Complete Guide
- Python ThreadPool: The Complete Guide
APIs
References
Takeaways
You now know how to identify deadlocks when using the ThreadPoolExecutor in Python.
Do you have any questions about how to identify deadlocks when using thread pools?
Ask your questions in the comments below and I will do my best to answer.
Photo by Patrick Hendry on Unsplash
Leave a Reply