Last Updated on September 12, 2022
You can check the status of tasks in the ProcessPoolExecutor in Python by calling running(), done(), and cancelled() on the Future object.
In this tutorial, you will discover how to check the status of tasks submitted to a process pool.
Let’s get started.
Need to Check the Status of Submitted Tasks
The ProcessPoolExecutor in Python provides a pool of reusable processes for executing ad hoc tasks.
You can submit tasks to the process pool by calling the submit() function and passing in the name of the function you wish to execute on another process.
Calling the submit() function will return a Future object that allows you to check on the status of the task and get the result from the task once it completes.
After you have submitted tasks, you may want to check on their status.
For example, you may want to check if a task is still running or has been done. If a task is done, you may want to check if it was cancelled.
Run loops using all CPUs, download your FREE book to learn how.
How to Check the Status of Tasks
You can check the status of tasks submitted to the ProcessPoolExecutor via the task’s Future object.
Recall that you will receive a Future object when you submit your task to the process pool when you call the submit() function.
|
1 2 3 |
... # submit a task to the process pool future = executor.submit(work) |
The Future object has a number of functions that you can use to check the status of the task; they are as follows:
- running(): Returns True if the task is running, False otherwise.
- done(): Returns True if the task is done, False otherwise.
- cancelled(): Returns True if the task was cancelled, False otherwise.
A running task will not be done or cancelled.
|
1 2 3 |
... # check if a task is running running = future.running() |
A done task will not be running but may have been cancelled.
|
1 2 3 |
... # check if a task is done done = future.done() |
A cancelled task will not be running and will be done.
|
1 2 3 |
... # check if a task was cancelled cancelled = future.cancelled() |
A task can be cancelled via its future object by calling the cancel() function that returns True if the task was cancelled and False otherwise. Only a task that is not running and not done can be cancelled.
- cancel(): Attempts to cancel a task, returns True if cancelled, False otherwise.
Now that we know how to check the status of a task, let’s look at a worked example.
Example of Checking the Status of Tasks
Let’s explore how to check the status of a task submitted to the ProcessPoolExecutor.
First, we can define a simple task that will block for a given number of seconds.
|
1 2 3 |
# task that will block for a moment def work(sleep_time): sleep(sleep_time) |
We can then create a process pool using the context manager.
|
1 2 3 4 |
... # create a process pool with ProcessPoolExecutor() as executor: # ... |
Next, we can check the status of our task, which we expect to be running and not yet done.
|
1 2 3 4 5 |
... # start one task future = executor.submit(work, 1) # confirm that the task is running print(f'running={future.running()}, done={future.done()}') |
We can then wait for the task to complete by calling the wait() module function and passing it a list that contains the single Future object.
|
1 2 3 |
... # wait for the task to complete wait([future]) |
Now that the task is complete, we can check the status again. We would expect the task to no longer be running and to be done.
|
1 2 |
# confirm that the task is done print(f'Future running={future.running()}, done={future.done()}') |
Tying this together, the complete example of checking the status of a task is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 |
# SuperFastPython.com # check the status of a Future object for task executed by a process pool from time import sleep from concurrent.futures import ProcessPoolExecutor from concurrent.futures import wait # task that will block for a moment def work(sleep_time): sleep(sleep_time) # entry point if __name__ == '__main__': # create a process pool with ProcessPoolExecutor() as executor: # start one process future = executor.submit(work, 1) # confirm that the task is running print(f'Future running={future.running()}, done={future.done()}') # wait for the task to complete wait([future]) # confirm that the task is done print(f'Future running={future.running()}, done={future.done()}') |
Running the example, we can see that immediately after starting the task that it is marked running and not yet done.
After it has completed, the task is marked as not running and done, as we expected.
|
1 2 |
Future running=True, done=False Future running=False, done=True |
Free Python ProcessPoolExecutor Course
Download your FREE ProcessPoolExecutor PDF cheat sheet and get BONUS access to my free 7-day crash course on the ProcessPoolExecutor API.
Discover how to use the ProcessPoolExecutor class including how to configure the number of workers and how to execute tasks asynchronously.
Example of Checking the Status of a Cancelled Task
We can also demonstrate the status of a task that is cancelled.
First, we can update the example so that the process pool only creates a single worker process.
|
1 2 3 4 |
... # create a process pool with ProcessPoolExecutor(1) as executor: # ... |
We can then submit a long-running task that will occupy the single worker process.
This is required so that we can submit a second task that can be cancelled because it is not yet running.
|
1 2 3 |
... # start a long running task _ = executor.submit(work, 1) |
Next, we can submit a second shorter duration task and check its status.
|
1 2 3 4 |
... # start a second task future = executor.submit(work, 0.1) print(f'running={future.running()}, cancelled={future.cancelled()}, done={future.done()}') |
We can then cancel the second task and check its status again.
Note that we can only cancel the second task because it is not yet running.
|
1 2 3 4 5 6 7 |
... # cancel the second task was_cancelled = future.cancel() print(f'Cancelled: {was_cancelled}') # wait for the second task to complete wait([future]) print(f'running={future.running()}, cancelled={future.cancelled()}, done={future.done()}') |
Tying this together, the complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 |
# SuperFastPython.com # example of checking the status of a cancelled task from time import sleep from concurrent.futures import ProcessPoolExecutor from concurrent.futures import wait # mock task that will sleep for a moment def work(sleep_time): sleep(sleep_time) # entry point if __name__ == '__main__': # create a process pool with ProcessPoolExecutor(1) as executor: # start a long running task _ = executor.submit(work, 1) # start a second task future = executor.submit(work, 0.1) print(f'running={future.running()}, cancelled={future.cancelled()}, done={future.done()}') # cancel the second task was_cancelled = future.cancel() print(f'Cancelled: {was_cancelled}') # wait for the second task to complete wait([future]) print(f'running={future.running()}, cancelled={future.cancelled()}, done={future.done()}') |
Running the example first creates the process pool with a single worker process, then occupies the worker process with a long running task.
We then submit our second task and check its status. We can see that it is not yet running and is not cancelled or done.
We then cancel the second task, which is reported successfully.
The status of the second task is checked again and we confirm that the task is still not running and was indeed cancelled and is now marked done.
|
1 2 3 |
running=False, cancelled=False, done=False Cancelled: True running=False, cancelled=True, done=True |
Overwhelmed by the python concurrency APIs?
Find relief, download my FREE Python Concurrency Mind Maps
Further Reading
This section provides additional resources that you may find helpful.
Books
- ProcessPoolExecutor Jump-Start, Jason Brownlee (my book!)
- Concurrent Futures API Interview Questions
- ProcessPoolExecutor PDF Cheat Sheet
I also recommend specific chapters from the following books:
- Effective Python, Brett Slatkin, 2019.
- See Chapter 7: Concurrency and Parallelism
- Python in a Nutshell, Alex Martelli, et al., 2017.
- See: Chapter: 14: Threads and Processes
Guides
- Python ProcessPoolExecutor: The Complete Guide
- Python ThreadPoolExecutor: The Complete Guide
- Python Multiprocessing: The Complete Guide
- Python Pool: The Complete Guide
APIs
References
- Thread (computing), Wikipedia.
- Process (computing), Wikipedia.
- Thread Pool, Wikipedia.
- Futures and promises, Wikipedia.
Takeaways
You now know how to check the status of tasks submitted to a ProcessPoolExecutor.
Do you have any questions?
Ask your question in the comments below and I will do my best to answer.
Photo by S O C I A L . C U T on Unsplash
Leave a Reply