You can execute a coroutine from another thread via the run_coroutine_threadsafe() function.
In this tutorial, you will discover how to execute coroutines from another thread in Python.
Let’s get started.
Need to Run a Coroutine From Thread
We may need to execute a coroutine from outside of a coroutine program.
Recall that asyncio provides coroutine-based concurrency.
We may be executing tasks in other threads, separate from a coroutine event loop.
Tasks executed in other threads may need to perform a non-blocking IO or to interact with an asyncio program via a coroutine.
How can a thread execute a coroutine?
Run loops using all CPUs, download your FREE book to learn how.
Run Coroutine From Another Thread
The run_coroutine_threadsafe() function allows a coroutine to be run in an asyncio program from another thread.
Submit a coroutine to the given event loop. Thread-safe.
— Coroutines and Tasks
Here, “another thread“, refers to a thread that is not currently executing the asyncio event loop.
This function is meant to be called from a different OS thread than the one where the event loop is running.
— Coroutines and Tasks
This function takes a coroutine object and an event loop and returns a concurrent.futures.Future which is a handle on the task.
For example:
|
1 2 3 |
... # execute a coroutine from another thread future = asyncio.run_coroutine_threadsafe(coro, loop) |
Note, that the return value is not an asyncio.Future object, it is an instance of the Future object used in the executor framework, such as with the ThreadPoolExecutor and ProcessPoolExecutor classes.
The returned futures can be used to access the return value from the task, such as via the result() method. This method will block until the result is available.
For example:
|
1 2 3 |
... # get the return value from the coroutine value = future.result() |
You can learn more about Future objects in the tutorial:
The thread that is executing the asyncio program can get a reference to the current event loop via the asyncio.get_running_loop() or asyncio.get_event_loop() functions.
For example:
|
1 2 3 |
... # get the event loop loop = asyncio.get_event_loop() |
This loop object can then be shared with another thread that needs to execute coroutines.
Because the run_coroutine_threadsafe() requires a loop, it means that a coroutine or task first retrieved the loop and made it available to the new thread. This suggests that new threads that need to execute coroutines are likely created and managed by the asyncio program itself, e.g. to execute a background task that blocks.
Now that we know how to use the run_coroutine_threadsafe() function, let’s look at some worked examples.
Example of Running a Coroutine From Another Thread
We can explore how to execute a coroutine from another thread.
In this example, we will run an asyncio program that reports some messages and runs a coroutine in the background. While the coroutine is running a new thread will run and will execute another coroutine in the asyncio program and wait for it to finish.
There are a few steps to this example, let’s work through them.
Firstly, we can define the coroutine that will be executed by the new thread.
This coroutine will report a message, sleep for two seconds, then report a final message.
The task_coro2() coroutine function below defines this.
|
1 2 3 4 5 6 7 8 |
# another task coroutine async def task_coro2(): # report a message print(f'>>task2 running') # block for a moment await asyncio.sleep(2) # report a message print(f'>>task2 done') |
Next, we can define a coroutine that is executed by the asyncio program, normally.
This coroutine is a long-running task. It loops five times, reports a message, and sleeps every iteration.
The task_coro() coroutine function below defines this.
|
1 2 3 4 5 6 7 8 |
# task coroutine async def task_coro(): # loop a few times for i in range(5): # report a message print(f'>task at {i}') # block a moment await asyncio.sleep(1) |
Next, we can define a function that will execute in a new thread.
The function will take the asyncio event loop object as an argument. This is needed so that the task in the new thread can execute a coroutine in the asyncio program.
The thread function will report a message, sleep a moment, then execute the task_coro2() coroutine in the asyncio program.
This returns a concurrent.futures.Future object that the thread then blocks on via the result() method, within for the coroutine to complete.
The task_thread() function below implements this.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
# function executed in another thread def task_thread(loop): # report a message print('thread running') # wait a moment time.sleep(1) # create a coroutine coro = task_coro2() # execute a coroutine future = asyncio.run_coroutine_threadsafe(coro, loop) # wait for the task to finish future.result() # report a message print('thread done') |
We can then define the main coroutine that is the entry point to the asyncio program.
This coroutine reports a message, then gets the event loop object and creates a new thread to execute our task_thread() thread function, passing the event loop object as an argument.
It then executes and awaits the task_coro() coroutine.
The main() coroutine function below implements this.
|
1 2 3 4 5 6 7 8 9 10 11 12 |
# entry point to the asyncio program async def main(): # report a message print('asyncio running') # get the event loop loop = asyncio.get_running_loop() # start a new thread threading.Thread(target=task_thread, args=(loop,)).start() # execute a task await task_coro() # report a message print('asyncio done') |
Finally, the entry point of the program runs the main() coroutine.
|
1 2 3 |
... # start the asyncio program asyncio.run(main()) |
The complete example is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 |
# SuperFastPython.com # example of executing a coroutine from another thread import threading import time import asyncio # another task coroutine async def task_coro2(): # report a message print(f'>>task2 running') # block for a moment await asyncio.sleep(2) # report a message print(f'>>task2 done') # task coroutine async def task_coro(): # loop a few times for i in range(5): # report a message print(f'>task at {i}') # block a moment await asyncio.sleep(1) # function executed in another thread def task_thread(loop): # report a message print('thread running') # wait a moment time.sleep(1) # create a coroutine coro = task_coro2() # execute a coroutine future = asyncio.run_coroutine_threadsafe(coro, loop) # wait for the task to finish future.result() # report a message print('thread done') # entry point to the asyncio program async def main(): # report a message print('asyncio running') # get the event loop loop = asyncio.get_running_loop() # start a new thread threading.Thread(target=task_thread, args=(loop,)).start() # execute a task await task_coro() # report a message print('asyncio done') # start the asyncio program asyncio.run(main()) |
Running the example first creates the main() coroutine and uses it to start the asyncio program.
The main() coroutine then reports a message. It then gets a reference to the current asyncio event loop.
A new thread is created and started, configured to execute the task_thread() function, and passed the event loop object as an argument.
This begins executing the task_thread() function in a new thread, reporting a message and blocking.
The main() coroutine then creates the task_coro() coroutine and awaits it. This suspends the main() coroutine and executes the task_coro() coroutine.
The task_coro() loops, reporting a message and sleeping each iteration.
The new thread resumes from its sleep and creates a task_coro2() coroutine and executes it in the asyncio event loop. It then awaits it to execute.
The second task coroutine is scheduled in the event loop and executes as soon as it is able, e.g. when the first task coroutine is suspended with a sleep. The second task reports a message, sleeps, and reports a second message.
The second task coroutine completes and the new thread resumes and terminates.
The first task coroutine eventually completes.
Finally, the main() coroutine resumes and terminates the program.
This highlights how an asyncio program can create and run a new separate thread and how a separate thread from the asyncio program can execute a coroutine and wait for it to finish.
|
1 2 3 4 5 6 7 8 9 10 11 |
asyncio running thread running >task at 0 >>task2 running >task at 1 >task at 2 >>task2 done >task at 3 thread done >task at 4 asyncio done |
Free Python Asyncio Course
Download your FREE Asyncio PDF cheat sheet and get BONUS access to my free 7-day crash course on the Asyncio API.
Discover how to use the Python asyncio module including how to define, create, and run new coroutines and how to use non-blocking I/O.
Further Reading
This section provides additional resources that you may find helpful.
Python Asyncio Books
- Python Asyncio Mastery, Jason Brownlee (my book!)
- Python Asyncio Jump-Start, Jason Brownlee.
- Python Asyncio Interview Questions, Jason Brownlee.
- Asyncio Module API Cheat Sheet
I also recommend the following books:
- Python Concurrency with asyncio, Matthew Fowler, 2022.
- Using Asyncio in Python, Caleb Hattingh, 2020.
- asyncio Recipes, Mohamed Mustapha Tahrioui, 2019.
Guides
APIs
- asyncio — Asynchronous I/O
- Asyncio Coroutines and Tasks
- Asyncio Streams
- Asyncio Subprocesses
- Asyncio Queues
- Asyncio Synchronization Primitives
References
Overwhelmed by the python concurrency APIs?
Find relief, download my FREE Python Concurrency Mind Maps
Takeaways
You now know how to execute coroutines from another thread with the run_coroutine_threadsafe() function.
Do you have any questions?
Ask your questions in the comments below and I will do my best to answer.
Photo by iStrfry , Marcus on Unsplash
Leave a Reply