Coroutine-safe is the concept of thread-safety for concurrency with coroutines, such as with asyncio.

In this tutorial, you will discover the concept of coroutine safety in Python.

Let’s get started.

What is Thread-Safe

Thread-safe refers to program code that can be executed free of concurrency errors by multiple threads concurrently.

Primarily, it refers to the fact that the code is free of race conditions.

A race condition is a bug in concurrency programming.

It is a failure case where the behavior of the program is dependent upon the order of execution by two or more threads. This means the behavior of the program will not be predictable, possibly changing each time it is run.

There are many types of race conditions, although a common type of race condition is when two or more threads attempt to change the same data variable.

You can see examples of race conditions in threads and how to fix them in these tutorials:

• Race Condition With a Shared Variable in Python
• How to Fix a Race Condition With Timing in Python

We can develop code to be thread-safe using concurrency primitives like mutex locks.

For example:

• Thread-Safe Counter in Python

Next, let’s take a look at process-safe.

What is Process-Safe

Process-safe refers to program code that can be executed free of concurrency errors by multiple processes concurrently.

It is the concept of “thread-safe” applied to processes, where processes are the unit of concurrency instead of threads.

Thread safety is a major concern of concurrent programming using threads. This is because threads have shared memory within the process, meaning that concurrent access of the same data or variables can lead to race conditions.

Processes do not have direct shared memory and therefore are not subject to the same concerns of race conditions.

Nevertheless, processes do simulate shared memory using socket connections and files. They may need to protect simulated shared program state or data from race conditions due to timing and concurrent modification.

Happily, some tools used for inter-process communication provide some process safety, such as queues.

You can learn more about process safety in the tutorial:

• Process-Safe in Python

Let’s take a closer look at how we can ensure process safety in our multiprocessing programs.

What is Coroutine-Safe

Coroutine-safe is the idea of “thread-safe” and “process-safe” applied to coroutines.

Recall that a coroutine is like a function (routine) except that it can be suspended and resumed.

It is a unit of concurrency lower than the level of an operating system thread and is used in the asyncio module.

Although two or more coroutines cannot execute at the same time within a thread, it is possible for program state and resources to be corrupted or made inconsistent via concurrent execution.

It is possible to have race conditions with coroutines in asyncio.

It is also possible to have deadlocks with coroutines in asyncio.

Coroutine-safe means that code or a program can be executed concurrently with multiple coroutines and will not result in concurrency failure modes such as race conditions and deadlocks.

Coroutine safety is achieved using common concurrency primitives, such as queues, locks, and more.

Next, let’s take a closer look at how we might implement coroutine safety in our asyncio programs.

How to Ensure Coroutine-Safety

The asyncio module provides a number of concurrency primitives that are coroutine-safe.

Nevertheless, they are not thread-safe (or process-safe), and this is clearly stated in the API documentation.

Additionally, if we attempt to use thread-safe concurrency primitives to protect coroutines from race conditions, it can result in unexpected behavior, such as deadlocks.

The asyncio module provides a suite of concurrency primitives to help us make our programs coroutine-safe.

asyncio synchronization primitives are designed to be similar to those of the threading module […] asyncio primitives are not thread-safe, therefore they should not be used for OS thread synchronization

— Asyncio Synchronization Primitives

These primitives mirror the primitives provided in the threading and multiprocessing modules, including the classes:

• asyncio.Lock
• asyncio.Event
• asyncio.Condition
• asyncio.Semaphore
• asyncio.BoundedSemaphore

We can use the primitives in our asyncio programs to ensure that our code is coronet-safe. That is, to protect critical sections, synchronize and coordinate behavior across multiple coroutines within a single thread (event loop).

In addition to these primitives, the asyncio module provides a number of coroutine-safe queue classes.

These too mirror the thread-safe queues in the “queue” module, but unlike those classes, these classes are not thread-safe.

asyncio queues are designed to be similar to classes of the queue module. Although asyncio queues are not thread-safe, they are designed to be used specifically in async/await code.

— Asyncio Queues

They include:

• asyncio.Queue
• asyncio.PriorityQueue
• asyncio.LifoQueue

We can use these queues to send and receive messages safely between coroutines within an asyncio program.

Further Reading

This section provides additional resources that you may find helpful.

Python Asyncio Books

• Python Asyncio Mastery, Jason Brownlee (my book!)
• Python Asyncio Jump-Start, Jason Brownlee.
• Python Asyncio Interview Questions, Jason Brownlee.
• Asyncio Module API Cheat Sheet

I also recommend the following books:

• Python Concurrency with asyncio, Matthew Fowler, 2022.
• Using Asyncio in Python, Caleb Hattingh, 2020.
• asyncio Recipes, Mohamed Mustapha Tahrioui, 2019.

Guides

• Python Asyncio: The Complete Guide
• Python Asynchronous Programming

APIs

• asyncio — Asynchronous I/O
• Asyncio Coroutines and Tasks
• Asyncio Streams
• Asyncio Subprocesses
• Asyncio Queues
• Asyncio Synchronization Primitives

References

• Asynchronous I/O, Wikipedia.
• Coroutine, Wikipedia.

Takeaways

You now know about coroutine safety in Python.

Do you have any questions?
Ask your questions in the comments below and I will do my best to answer.

Photo by Dan Gold on Unsplash