We often need to execute long-running tasks in asyncio.
For example, we may need to wait for a response from a remote server, for something to change, or for input from another system.
It is a best practice to use a timeout when waiting for a long-running task. If the task does not complete within a given time limit, can then be canceled and perhaps tried again, or an error raised.
The asyncio module provides the timeout() asynchronous context manager to address exactly this problem.
In this tutorial, you will discover how to wait for coroutines and tasks using the timeout() asynchronous context manager.
After completing this tutorial, you will know:
- What is timeout() and how can we use it to wait for tasks to complete with a timeout.
- How to manage the case when the timeout is exceeded before the required tasks are complete.
- How to configure the timeout and how to extend the timeout if more time is needed.
Let’s get started.
Need to Wait For Long-Running Task With a Timeout
We often need to execute long-running tasks in asyncio.
For example:
- We may need to wait for a response from a remote server.
- We may need to wait for something to change.
- We may need to wait for input from a user or system.
It is a best practice to use a timeout when waiting for a long-running task.
If the task does not complete within a given time limit, can then be canceled and perhaps tried again, or an error raised.
The asyncio module provides the asyncio.timeout() asynchronous context manager to address exactly this problem.
How can we use asyncio.timeout() to add timeouts to long-running tasks?
Run loops using all CPUs, download your FREE book to learn how.
How to Use asyncio.timeout()
The asyncio.timeout() is an asynchronous context manager.
As such, the entry and exit methods are coroutines that can be awaited and it must be used via the “async with” expression.
You can learn more about asynchronous context managers in the tutorial:
The asyncio.timeout() context manager and associated asyncio.Timeout class were added to Python in version 3.11.
Added timeout(), an asynchronous context manager for setting a timeout on asynchronous operations. For new code this is recommended over using wait_for() directly. (Contributed by Andrew Svetlov in gh-90927.)
— What’s New In Python 3.11
Entering the asyncio.timeout() context manager will set a time in the future when the context manager will give up waiting.
The timeout is provided in seconds, e.g. 5 seconds in the future, as recorded by the clock within the asyncio event loop.
For example:
|
1 2 3 4 |
... # wait for 5 seconds async with asyncio.timeout(5): # ... |
We can then explicitly await long-running coroutines within the block of the asyncio.timeout() context manager.
For example:
|
1 2 3 4 5 |
... # set a timeout async with asyncio.timeout(5): # execute long running task result = await task() |
The benefit of asyncio.timeout() being a context manager, means that we can await many coroutines, e.g. many sub-tasks, within the body and only cancel the one task that takes too long.
For example:
|
1 2 3 4 5 6 7 8 9 |
... # set a timeout async with asyncio.timeout(5): # execute many tasks await task1() await task2() await task3() await task4() await task5() |
If the wait time elapsed before the context manager is exited, any awaitables (e.g. coroutines and tasks) awaited within the block will be canceled and an asyncio.TimeoutError will be raised.
We can handle the asyncio.TimeoutError and report and perform an action, like report a message of the timeout and cancellation of the task.
For example:
|
1 2 3 4 5 6 7 8 9 10 11 |
... # handle timeout try: # set a timeout async with asyncio.timeout(5): # execute long running task result = await task(1) # report the result print(result) except asyncio.TimeoutError: print(f'Timeout waiting') |
The asyncio.timeout() function creates an instance of the asyncio.Timeout class has three methods:
- asyncio.Timeout.when(): Return the current deadline
- asyncio.Timeout.reschedule(): Change the current deadline to a new time.
- asyncio.Timeout.expired(): Return True if the timeout has expired, False otherwise.
It is possible to extend the timeout further into the future.
This can be used to either set no timeout initially, and then set a timeout later while running, or to extend a predefined timeout further into the future.
For example, we can set a timeout by passing None for the delay, then calling the reschedule() method on the asynchronous context manager with a time in the future.
|
1 2 3 4 5 6 7 8 9 10 11 |
... # set no timeout async with asyncio.timeout(None) as timeout: # do something else # ... # calculate a deadline 5 seconds in the future deadline = asyncio.get_running_loop().time() + 5 # set the new deadline timeout.reschedule(deadline) # execute long running task result = await task() |
We can also set an initial timeout delay, then extend it into the future using the same approach.
For example:
|
1 2 3 4 5 6 7 8 9 10 11 |
... # set a 5 second timeout async with asyncio.timeout(5) as timeout: # do something else # ... # calculate a deadline 10 seconds in the future deadline = asyncio.get_running_loop().time() + 10 # set the new deadline timeout.reschedule(deadline) # execute long running task result = await task() |
Now that we know how to use asyncio.timeout(), let’s look at some worked examples.
Example of Timeout with Long Running Coroutine
We can explore an example of waiting for a long-running coroutine with a timeout.
In this example, we will define a long-running coroutine that takes an argument and returns a result. We will then execute this coroutine within an asyncio.timeout() context manager and handle any asyncio.TimeoutError raised.
Firstly, we can define the long-running coroutine. In this case, it sleeps for 10 seconds.
|
1 2 3 4 5 6 |
# long running task async def task(value): # sleep to simulate waiting await asyncio.sleep(10) # return value return value * 100 |
Next, we can define the main coroutine.
It first handles the asyncio.TimeoutError, then opens the asyncio.timeout() context manager with a timeout of 5 seconds in the future.
Within the context manager, the task() coroutine is called, passing an argument and retrieving the return value which is then printed.
|
1 2 3 4 5 6 7 8 9 10 11 12 |
# asyncio entry point async def main(): # handle timeout try: # set a timeout async with asyncio.timeout(5): # execute long running task result = await task(1) # report the result print(result) except asyncio.TimeoutError: print(f'Timeout waiting') |
We can then start the asyncio event loop and execute the main() coroutine.
|
1 2 3 |
... # start the event loop asyncio.run(main()) |
Tying this together, the complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 |
# SuperFastPython.com # example of waiting for a coroutine with asyncio.timeout() import asyncio # long running task async def task(value): # sleep to simulate waiting await asyncio.sleep(10) # return value return value * 100 # asyncio entry point async def main(): # handle timeout try: # set a timeout async with asyncio.timeout(5): # execute long running task result = await task(1) # report the result print(result) except asyncio.TimeoutError: print(f'Timeout waiting') # start the event loop asyncio.run(main()) |
Running the example opens the asyncio.timeout() with a timeout 5 seconds in the future.
The task() coroutine is started and runs, sleeping for 5 seconds.
After 5 seconds, the asyncio.timeout() context manager resumes and cancels the task() coroutine and raises an asyncio.TimeoutError.
This exception is handled and a failure message is reported.
This example highlights how we can execute a long-running coroutine with a timeout, then cancel the coroutine after the timeout expires and handle the termination exception.
|
1 |
Timeout waiting |
Next, let’s explore how we might perform the same operation using a task.
Free Python Asyncio Course
Download your FREE Asyncio PDF cheat sheet and get BONUS access to my free 7-day crash course on the Asyncio API.
Discover how to use the Python asyncio module including how to define, create, and run new coroutines and how to use non-blocking I/O.
Example of Timeout with Long Running asyncio.Task
We can explore an example of waiting for a long-running asyncio.Task with a timeout.
In this example, we will update the above example to instead use an asyncio.Task.
We can create and schedule the task() coroutine as an asyncio.Task before the asyncio.timeout() block.
For example:
|
1 2 3 |
... # schedule the task running_task = asyncio.create_task(task(1)) |
We can also allow the task to start running.
|
1 2 3 |
... # allow the task to run await asyncio.sleep(0) |
We can then open the context manager and await this already scheduled task, with a timeout.
|
1 2 3 4 5 6 7 8 9 10 11 |
... # handle timeout try: # set a timeout async with asyncio.timeout(5): # wait for the task to complete result = await running_task # report the result print(result) except asyncio.TimeoutError: print(f'Timeout waiting') |
Tying this together, the complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 |
# SuperFastPython.com # example of waiting for a task with asyncio.timeout() import asyncio # long running task async def task(value): # sleep to simulate waiting await asyncio.sleep(10) # return value return value * 100 # asyncio entry point async def main(): # schedule the task running_task = asyncio.create_task(task(1)) # allow the task to run await asyncio.sleep(0) # handle timeout try: # set a timeout async with asyncio.timeout(5): # wait for the task to complete result = await running_task # report the result print(result) except asyncio.TimeoutError: print(f'Timeout waiting') # start the event loop asyncio.run(main()) |
Running the example first creates the task() coroutine and schedules it as a task.
It then suspends the main() coroutine and allows the task to start running, beginning its sleep.
The main() coroutine resumes and opens the timeout context manager and awaits the task.
After 5 seconds, the asyncio.timeout() context manager resumes and cancels the asyncio.Task and raises an asyncio.TimeoutError.
This exception is handled and a failure message is reported.
This example highlights how we can wait for an already executing long-running asyncio.Task with a timeout, then cancel it after the timeout expires and handle the termination exception.
|
1 |
Timeout waiting |
Overwhelmed by the python concurrency APIs?
Find relief, download my FREE Python Concurrency Mind Maps
Example of Timeout without Handling asyncio.TimeouError
We can explore an example of waiting for a long-running coroutine with a timeout without handling the asyncio.TimeoutError.
In this case, we will update the above example to wait for a coroutine with a timeout, except we will not handle the exception.
The coroutine will be canceled raising an asyncio.exceptions.CancelledError, which will be caught and re-raised as an asyncio.TimeoutError.
This can be achieved by simply removing the try-except.
The complete example of this change is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 |
# SuperFastPython.com # example of waiting for a coroutine with asyncio.timeout() import asyncio # long running task async def task(value): # sleep to simulate waiting await asyncio.sleep(10) # return value return value * 100 # asyncio entry point async def main(): # set a timeout async with asyncio.timeout(5): # execute long running task result = await task(1) # report the result print(result) # start the event loop asyncio.run(main()) |
Running the example opens the asyncio.timeout() with a timeout of 5 seconds in the future.
The task() coroutine is started and runs, sleeping for 5 seconds.
After 5 seconds, the asyncio.timeout() context manager resumes and cancels the task() coroutine and raises an asyncio.TimeoutError.
In this case, the exception is not handled and bubbles up, causing the program to terminate.
We can see both the source asyncio.exceptions.CancelledError exception and the wrapping TimeoutError exception.
This example highlights how we can execute a long-running coroutine with a timeout, then cancel the coroutine after the timeout expires and not handle the termination exception.
|
1 2 3 4 5 6 7 8 9 |
Traceback (most recent call last): ... asyncio.exceptions.CancelledError The above exception was the direct cause of the following exception: Traceback (most recent call last): ... TimeoutError |
Example of Timeout With A Delay Time Set Later
We can explore an example of waiting for a long-running coroutine without an initial timeout, then adding a timeout later.
In this case, we can set no initial timeout.
|
1 2 3 4 |
... # set a timeout async with asyncio.timeout(None) as timeout: # ... |
We can then perform some work and determine a timeout in the future and set this into the asyncio.timeout() context manager.
|
1 2 3 4 5 6 7 8 9 |
... # set a timeout async with asyncio.timeout(None) as timeout: # wait a moment await asyncio.sleep(1) # calculate a deadline 5 seconds in the future deadline = asyncio.get_running_loop().time() + 5 # set the new deadline timeout.reschedule(deadline) |
This allows the timeout to be set later, on demand.
Tying this together, the complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 |
# SuperFastPython.com # example of setting a delay later with asyncio.timeout() import asyncio # long running task async def task(value): # sleep to simulate waiting await asyncio.sleep(10) # return value return value * 100 # asyncio entry point async def main(): # handle timeout try: # set a timeout async with asyncio.timeout(None) as timeout: # wait a moment await asyncio.sleep(1) # calculate a deadline 5 seconds in the future deadline = asyncio.get_running_loop().time() + 5 # set the new deadline timeout.reschedule(deadline) # execute long running task result = await task(1) # report the result print(result) except asyncio.TimeoutError: print(f'Timeout waiting') # start the event loop asyncio.run(main()) |
Running the example opens the asyncio.timeout() without a timeout.
The main() coroutine sleeps to simulate other work.
A timeout of 5 seconds in the future is then determined and the timeout is set into the asyncio.timeout() context manager.
The task() coroutine is then issued and awaited.
After 5 seconds, the asyncio.timeout() context manager resumes and cancels the task() coroutine and raises an asyncio.TimeoutError.
The exception is handled and a message is reported.
This example highlights how we can set a timeout after we have entered the block and that this timeout has the normal effect of canceling the blocked task and raising an asyncio.TimeoutError exception.
|
1 |
Timeout waiting |
Example of Extending a Timeout Further Into The Future
We can explore an example of waiting for a long-running coroutine with a timeout and then extending the timeout further into the future later.
In this case, we will update the example with the long-running coroutine. A first timeout will be set to 5 seconds on the context manager, not enough time for the task to complete.
The program will perform other tasks, and then increase the timeout by another 11 seconds into the future.
It will then execute the long-running task, which will have enough time to complete.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
... # set a timeout async with asyncio.timeout(5) as timeout: # wait a moment await asyncio.sleep(4) # calculate a deadline 5 seconds in the future deadline = asyncio.get_running_loop().time() + 11 # set the new deadline timeout.reschedule(deadline) # execute long running task result = await task(1) # report the result print(result) |
Tying this together, the complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 |
# SuperFastPython.com # example of extending a timeout delay with asyncio.timeout() import asyncio # long running task async def task(value): # sleep to simulate waiting await asyncio.sleep(10) # return value return value * 100 # asyncio entry point async def main(): # handle timeout try: # set a timeout async with asyncio.timeout(5) as timeout: # wait a moment await asyncio.sleep(4) # calculate a deadline 5 seconds in the future deadline = asyncio.get_running_loop().time() + 11 # set the new deadline timeout.reschedule(deadline) # execute long running task result = await task(1) # report the result print(result) except asyncio.TimeoutError: print(f'Timeout waiting') # start the event loop asyncio.run(main()) |
Running the example opens the asyncio.timeout() with a timeout of 5 seconds, not long enough for our task.
The main() coroutine sleeps for 4 seconds to simulate other work.
The main() coroutine resumes and updates the timeout to 11 seconds into the future. This is enough time to complete the task.
The task() coroutine is then issued and awaited.
The task completes normally and the return value is reported.
This example highlights how we can set a timeout on the context manager, then update that time out later as more information becomes available.
|
1 |
100 |
Further Reading
This section provides additional resources that you may find helpful.
Python Asyncio Books
- Python Asyncio Mastery, Jason Brownlee (my book!)
- Python Asyncio Jump-Start, Jason Brownlee.
- Python Asyncio Interview Questions, Jason Brownlee.
- Asyncio Module API Cheat Sheet
I also recommend the following books:
- Python Concurrency with asyncio, Matthew Fowler, 2022.
- Using Asyncio in Python, Caleb Hattingh, 2020.
- asyncio Recipes, Mohamed Mustapha Tahrioui, 2019.
Guides
APIs
- asyncio — Asynchronous I/O
- Asyncio Coroutines and Tasks
- Asyncio Streams
- Asyncio Subprocesses
- Asyncio Queues
- Asyncio Synchronization Primitives
References
Takeaways
You now know how to wait for coroutines and tasks using the asyncio.timeout() asynchronous context manager.
Did I make a mistake? See a typo?
I’m a simple humble human. Correct me, please!
Do you have any questions?
Ask your questions in the comments below and I will do my best to answer.
Leave a Reply