You can force an asyncio task to cancel by repeatedly calling the cancel() in a loop.
In this tutorial, you will discover how to force an asyncio task to cancel.
Let’s get started.
What is Asyncio Task Cancellation?
Asyncio tasks can be canceled.
This can be achieved by calling the cancel() method on the asyncio.Task. This will request that the task be canceled as soon as possible and return True if the request was successful or False if it was not, e.g. the task is already done.
Request the Task to be cancelled. This arranges for a CancelledError exception to be thrown into the wrapped coroutine on the next cycle of the event loop.
— Coroutines and Tasks
For example:
|
1 2 3 4 |
... # cancel a task if task.cancel(): # request successful |
The next time that the target task that was canceled resumes, a CancelledError exception will be raised.
This exception will bubble up to the top level of the task and cause it to stop running.
You can learn more about the mechanics of cancellation a task in the tutorial:
Run loops using all CPUs, download your FREE book to learn how.
Need to Force a Task to Cancel
Sometimes tasks don’t cancel when requested.
This can happen for many reasons:
- The task consumes the CancelledError.
- The task has a finally block that does lots of awaiting.
- The task is shielded.
We can’t do a lot if the task is shielded.
Nevertheless, if the task attempts to consume the CancelledError or does too much work in a finally block, we may want to force it to cancel by requesting a cancellation a second or third time.
How can we force a task to be canceled?
How to Force a Task to Cancel
We cannot force an asyncio task to cancel.
This is because the task can forever resist the request to cancel if it so chooses. That is, it can continue to consume the CancelledError raised when the task resumes after suspension. But this is unlikely.
Instead, it is more likely that a task will attempt to clean up when canceled, and perhaps spend too much time doing so.
In this case, we can force a task to cancel by repeatedly issuing requests to cancel until the task is indeed done.
- Force a task to cancel by calling cancel() until the task is done.
We can develop a coroutine that will do this.
The coroutine will loop until the task is done. Each iteration, will call the cancel() method on the task and wait a moment for the task to cancel.
We will set the wait period at one second, but it could be longer or shorter, depending on the application.
We will also add a limited number of retries, to ensure the forced cancellation does not go on forever. In this case, if more than 10 attempts to cancel the target task fail, a RuntimeError is raised.
The force_cancel() coroutine implements this.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
# helper function to force a task to cancel async def force_cancel(task, max_tries=10): # keep track of the number of times tried tries = 0 # keep trying to cancel the task while not task.done(): # check if we tried too many times if tries >= max_tries: raise RuntimeError(f'Failed to cancel task {task} after {max_tries} attempts.') # request the task cancel task.cancel() # update attempt count tries += 1 # give the task time to cancel await asyncio.sleep(1) |
We can use this as a helper in our programs when we need to force a task to cancel.
Note that the force_cancel() coroutine is only concerned with calling cancel() and checking if the task is done. It does not await the target task and therefore does not propagate any CancelledError that may be raised by the target task.
Now that we know how we might force a task to be canceled, let’s look at a worked example.
Free Python Asyncio Course
Download your FREE Asyncio PDF cheat sheet and get BONUS access to my free 7-day crash course on the Asyncio API.
Discover how to use the Python asyncio module including how to define, create, and run new coroutines and how to use non-blocking I/O.
Example of Forcing a Task to Cancel
We can explore an example of forcing an asyncio task to cancel.
In this case, we can develop a task that takes a long time. Within the body of the task, it consumes the asyncio.CancelledError (which is not a good practice), and then takes more time doing clean-up. We will then use our force_cancel() helper developed in the previous section to force this task to cancel.
For more on the best practices when canceling tasks, see the tutorial:
Firstly, we can develop our long-running task that also does long-running clean-up when canceled.
The work() coroutine below implements this.
|
1 2 3 4 5 6 7 8 9 10 11 12 |
# task that takes a long time async def work(): try: # sleep a long time await asyncio.sleep(5) # report a message print('Task sleep completed normally') except asyncio.CancelledError: # report a message print('Task as cancelled, cleaning up') # sleep a long time await asyncio.sleep(5) |
We can then define our force_cancel() helper coroutine, developed in the previous section.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
# helper function to force a task to cancel async def force_cancel(task, max_tries=10): # keep track of the number of times tried tries = 0 # keep trying to cancel the task while not task.done(): # check if we tried too many times if tries >= max_tries: raise RuntimeError(f'Failed to cancel task {task} after {max_tries} attempts.') # request the task cancel task.cancel() # update attempt count tries += 1 # give the task time to cancel await asyncio.sleep(1) |
In the main() coroutine, we will first create and issue the work() coroutine, and then sleep, allowing it to run.
|
1 2 3 4 5 |
... # create and schedule the task task = asyncio.create_task(work()) # allow the task to run a moment await asyncio.sleep(1) |
Next, we will force the work task to be canceled.
|
1 2 3 |
... # force the task to cancel await force_cancel(task) |
Finally, we can report the details of the work task, to confirm it was canceled.
|
1 2 3 4 5 |
... # report details of the task print(f'Task Cancelling: {task.cancelling()}') print(f'Task Cancelled: {task.cancelled()}') print(f'Task Done: {task.done()}') |
Tying this together, the main() coroutine is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 |
# main coroutine async def main(): # create and schedule the task task = asyncio.create_task(work()) # allow the task to run a moment await asyncio.sleep(1) # force the task to cancel await force_cancel(task) # report details of the task print(f'Task Cancelling: {task.cancelling()}') print(f'Task Cancelled: {task.cancelled()}') print(f'Task Done: {task.done()}') |
Finally, we can start the asyncio event loop.
|
1 2 3 |
... # start the event loop asyncio.run(main()) |
Tying all of this together, the complete example of forcing an asyncio task to cancel is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 |
# SuperFastPython.com # example of forcing a task to cancel import asyncio # task that takes a long time async def work(): try: # sleep a long time await asyncio.sleep(5) # report a message print('Task sleep completed normally') except asyncio.CancelledError: # report a message print('Task as cancelled, cleaning up') # sleep a long time await asyncio.sleep(5) # helper function to force a task to cancel async def force_cancel(task, max_tries=10): # keep track of the number of times tried tries = 0 # keep trying to cancel the task while not task.done(): # check if we tried too many times if tries >= max_tries: raise RuntimeError(f'Failed to cancel task {task} after {max_tries} attempts.') # request the task cancel task.cancel() # update attempt count tries += 1 # give the task time to cancel await asyncio.sleep(1) # main coroutine async def main(): # create and schedule the task task = asyncio.create_task(work()) # allow the task to run a moment await asyncio.sleep(1) # force the task to cancel await force_cancel(task) # report details of the task print(f'Task Cancelling: {task.cancelling()}') print(f'Task Cancelled: {task.cancelled()}') print(f'Task Done: {task.done()}') # start the event loop asyncio.run(main()) |
Running the example first starts the event loop and runs the main() coroutine.
The main() coroutine runs and creates and schedules the work() coroutine, then the main() coroutine suspends.
The work() coroutine runs and begins a long sleep.
The main() coroutine resumes and then calls the force_cancel() coroutine to cancel the work() coroutine, and suspends until it is complete.
The force_cancel() coroutine runs and starts the main loop. It checks if the target task is done, which it is not, so continues. It checks if the maximum number of attempts has been reached, which it has not so it continues.
It then cancels the task, increments the count, and waits a moment.
The work() coroutine resumes and a CancelledError is raised. This is handled by the task and it begins its cleanup, reporting a message and suspending again with another long sleep.
The force_cancel() coroutine resumes and performs another iteration of the loop and again calls cancel on the work() task and suspends for a moment.
The work() task resumes and another CancelledError is raised. This one is not handled by the task, which bubbles up and causes the work() task to terminate.
The force_cancel() coroutine resumes and checks if the work() task is done, which it now is. The loop exits and the force_cancel() coroutine terminates.
The main() coroutine results and reports the details of the work() task.
We can see that the target task received two requests to cancel and is indeed done with a canceled state.
This highlights how we can aggressively force a task to cancel by repeatedly calling its cancel() method in a loop.
|
1 2 3 4 |
Task as cancelled, cleaning up Task Cancelling: 2 Task Cancelled: True Task Done: True |
Overwhelmed by the python concurrency APIs?
Find relief, download my FREE Python Concurrency Mind Maps
Further Reading
This section provides additional resources that you may find helpful.
Python Asyncio Books
- Python Asyncio Mastery, Jason Brownlee (my book!)
- Python Asyncio Jump-Start, Jason Brownlee.
- Python Asyncio Interview Questions, Jason Brownlee.
- Asyncio Module API Cheat Sheet
I also recommend the following books:
- Python Concurrency with asyncio, Matthew Fowler, 2022.
- Using Asyncio in Python, Caleb Hattingh, 2020.
- asyncio Recipes, Mohamed Mustapha Tahrioui, 2019.
Guides
APIs
- asyncio — Asynchronous I/O
- Asyncio Coroutines and Tasks
- Asyncio Streams
- Asyncio Subprocesses
- Asyncio Queues
- Asyncio Synchronization Primitives
References
Takeaways
You now know how to force an asyncio task to cancel.
Did I make a mistake? See a typo?
I’m a simple humble human. Correct me, please!
Do you have any additional tips?
I’d love to hear about them!
Do you have any questions?
Ask your questions in the comments below and I will do my best to answer.
Photo by Louis Hansel on Unsplash
Leave a Reply