Last Updated on September 12, 2022
You can fix a race condition based on timing using a threading.Event.
In this tutorial you will discover how to identify and fix a timing-based race condition in Python.
Let’s get started.
Race Condition with Timing
A race condition is a bug in concurrency programming.
It is a failure case where the behavior of the program is dependent upon the order of execution by two or more threads. This means, the behavior of the program will not be predictable, possibly changing each time it is run.
There are many types of race conditions, although a common type of race condition is when two threads attempt to coordinate their behavior.
Note: Race conditions are a real problem in Python when using threads, even in the presence of the global interpreter lock (GIL). The refrain that there are no race conditions in Python because of the GIL is dangerously wrong.
For example, consider the case of the use of a threading.Condition used by two threads to coordinate their behavior.
If you are new to a condition object, you can learn more here:
One thread may wait on the threading.Condition to be notified of some state change within the application via the wait function.
|
1 2 3 4 |
... # wait for a state change with condition: condition.wait() |
Recall that when using a threading.Condition, you must acquire the condition before you can call wait() or notify(), then release it once you are done. This is easily achieved using the context manager.
Another thread may perform some change within the application and alert the waiting thread via the condition with the notify() function.
|
1 2 3 4 |
... # alert the waiting thread with condition: condition.notify() |
This is an example of coordinated behavior between two threads where one thread signals another thread.
For the behavior to work as intended, the notification from the second thread must be sent after the first thread has started waiting. If the first thread calls wait() after the second thread calls notify(), then it will not be notified and will wait forever.
This may happen if there is a context switch by the operating system that allows the second thread that calls notify() to run before the first thread that calls wait() to run.
You may recall that the operating system controls what threads execute and when, and that a context switch refers to pausing the execution of a thread and storing its state, while unpausing another thread and restoring its state.
Now that we are familiar with a race condition due to timing, let’s look at how we might fix it.
Run loops using all CPUs, download your FREE book to learn how.
Fix a Race Condition with Timing
There are many ways to fix a race condition between two or more threads based on timing.
The approach taken may depend on the specifics of the application.
Nevertheless, one common approach is to have the waiting threads signal that they are ready before the notifying thread starts its work and calls notify.
This can be achieved with a threading.Event, which is like a thread-safe boolean flag variable.
If you are not familiar with the threading.Event, you can learn more here:
A shared threading.Event object can be created.
|
1 2 3 |
... # create a shared event event = threading.Event() |
When the waiting thread (or threads) are ready they can signal this fact via setting the event flag to true.
|
1 2 3 |
... # signal that the waiting thread is ready event.set() |
Importantly, this should be performed while the waiting thread has acquired the threading.Condition. For example:
|
1 2 3 4 5 6 7 |
... # wait to be notified with condition: # indicate we are ready to be notified event.set() # wait to be notified condition.wait() |
This will be clearer in a moment, but it ensures we avoid fixing the timing race condition by introducing a second timing race condition. Specifically between one thread setting the flag and another waiting on the flag to be set.
The notifying thread can then wait for the waiting thread to signal that it is ready before doing its work, such as changing state, and notifying the waiting thread.
This can be achieved using busy waiting. That is, a type of waiting where we check for the event flag to be set in a loop.
If you are new to busy waiting, you can learn more about it here:
To lessen the computational effort of the busy wait loop, the main thread can sleep for a fraction of a second each iteration.
For example:
|
1 2 3 4 |
... # busy wait for the new thread to get ready while not event.is_set(): sleep(0.1) |
When the threading.Event flag is set by the new thread, the main thread will exit its busy wait loop.
At this time the new thread has already acquired the threading.Condition and will call wait().
After exiting the busy wait loop, the main thread will then acquire the threading.Condition and notify the waiting thread.
|
1 2 3 4 |
... # notify the new thread with condition: condition.notify() |
Importantly, if there was a context switch or a delay of any kind between the new thread calling event.set() and condition.wait(), it will not introduce a second race condition.
For example:
|
1 2 3 4 5 6 7 8 9 |
... # wait to be notified with condition: # indicate we are ready to be notified event.set() # add an artificial delay sleep(5) # wait to be notified condition.wait() |
The reason is because the new thread has acquired the threading.Condition and no other thread can acquire it until it releases it or waits on it.
Therefore, even if the new thread performs a long calculation or a sleep between setting the event and waiting on the condition, the main thread will not be able to acquire the threading.Condition (and call notify()) until the new thread calls wait().
Now that we know how to fix the race condition due to timing using an event, let’s look at a worked example.
Example of a Race Condition With Timing
We can explore an example of a race condition due to timing.
In this example we will create a threading.Condition, then start a new thread that waits on the condition to be notified and the main thread that notifies the new thread.
To force the race condition, we will add a delay between the new thread starting and waiting on the condition. This will cause the new thread to always miss the notification from the main thread and wait forever, requiring the program to be killed manually rather than terminating normally.
Firstly, we can define a function named task() to be executed by a new thread.
The function will first sleep for a fraction of a second to force the timing race condition, then acquire the condition and wait to be notified.
The complete function is listed below.
|
1 2 3 4 5 6 7 8 9 |
# thread waiting to be notified def task(condition): # insert a delay sleep(0.5) # wait to be notified print('Thread: Waiting to be notified...') with condition: condition.wait() print('Thread: Notified') |
Next, in the main thread we can create the shared threading.Condition object.
|
1 2 3 |
... # create the shared condition condition = Condition() |
Next, we can create and start a new threading.Thread configured to execute our task() function and pass in the condition as an argument to the function.
|
1 2 3 4 5 |
... # create the new thread thread = Thread(target=task, args=(condition,)) # start the new thread thread.start() |
Finally, the main thread can acquire the threading.Condition and notify the new thread.
|
1 2 3 4 5 6 |
... # notify the new thread print('Main: Notifying the thread') with condition: condition.notify() print('Main: Done') |
Tying this together, the complete example of a race condition based on timing is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 |
# SuperFastPython.com # example of a race condition with timing from time import sleep from threading import Thread from threading import Condition # thread waiting to be notified def task(condition): # insert a delay sleep(0.5) # wait to be notified print('Thread: Waiting to be notified...') with condition: condition.wait() print('Thread: Notified') # create the shared condition condition = Condition() # create the new thread thread = Thread(target=task, args=(condition,)) # start the new thread thread.start() # notify the new thread print('Main: Notifying the thread') with condition: condition.notify() print('Main: Done') |
Running the example first creates and starts the new thread.
The new thread starts running and then sleeps for a fraction of a second.
Meanwhile, the main thread acquires the condition and calls notify().
The new thread wakes up, acquires the condition and waits to be notified. Because the notification has already occurred, it is missed and the new thread waits forever.
This demonstrates a race condition based on timing.
|
1 2 3 |
Main: Notifying the thread Main: Done Thread: Waiting to be notified... |
Next, let’s look at how we might fix this race condition using an event.
Free Python Threading Course
Download your FREE threading PDF cheat sheet and get BONUS access to my free 7-day crash course on the threading API.
Discover how to use the Python threading module including how to create and start new threads and how to use a mutex locks and semaphores
Example of Fixing a Race Condition With Timing
A race condition based on timing seen in the previous section can be fixed by allowing the notify thread to wait for the waiting thread to be ready before doing its work and calling notify().
One way to achieve this is by using a threading.Event, which is a thread safe boolean flag variable.
The shared threading.Event can be passed to the task() function as an argument and then set by the new thread while holding the threading.Condition, right before waiting on the condition.
|
1 2 3 4 5 6 7 8 9 |
... # wait to be notified with condition: # indicate we are ready to be notified print('Thread: Ready') event.set() # wait to be notified print('Thread: Waiting to be notified...') condition.wait() |
It is important in this change that the event is set while the condition is held as it blocks the main thread from acquiring the condition and calling notify until the new thread releases the condition when calling wait().
The updated version of the task() function with this change is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
# thread waiting to be notified def task(condition, event): # insert a delay sleep(0.5) # wait to be notified with condition: # indicate we are ready to be notified print('Thread: Ready') event.set() # wait to be notified print('Thread: Waiting to be notified...') condition.wait() print('Thread: Notified') |
The main thread can then create the shared threading.Event object.
|
1 2 3 |
... # create the shared event event = Event() |
It can then be passed to a new thread instance.
|
1 2 3 4 5 |
... # create the new thread thread = Thread(target=task, args=(condition,event)) # start the new thread thread.start() |
Finally, the main thread can wait for the threading.Event in a busy loop.
In order to limit the amount of computation performed within the busy loop, we can add a sleep each iteration.
|
1 2 3 4 5 |
... # busy wait for the new thread to get ready print('Main: Waiting for threads to get ready...') while not event.is_set(): sleep(0.1) |
Tying this together, the complete example of fixing a race condition based on timing is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 |
# SuperFastPython.com # example of a race condition with timing from time import sleep from threading import Thread from threading import Condition from threading import Event # thread waiting to be notified def task(condition, event): # insert a delay sleep(0.5) # wait to be notified with condition: # indicate we are ready to be notified print('Thread: Ready') event.set() # wait to be notified print('Thread: Waiting to be notified...') condition.wait() print('Thread: Notified') # create the shared condition condition = Condition() # create the shared event event = Event() # create the new thread thread = Thread(target=task, args=(condition,event)) # start the new thread thread.start() # busy wait for the new thread to get ready print('Main: Waiting for threads to get ready...') while not event.is_set(): sleep(0.1) # notify the new thread print('Main: Notifying the thread') with condition: condition.notify() print('Main: Done') |
Running the example first creates the shared condition and event.
The new thread is then created and started, which immediately blocks for a moment.
Me while the main thread enters its busy wait loop and checks the event ten times per second.
The new thread wakes up, acquires the condition, sets the event and then waits on the condition.
The main thread notices that the event has been set, then acquires the condition and notifies the new thread.
The program works as expected and the race condition no longer exists.
|
1 2 3 4 5 6 |
Main: Waiting for threads to get ready... Thread: Ready Thread: Waiting to be notified... Main: Notifying the thread Main: Done Thread: Notified |
Overwhelmed by the python concurrency APIs?
Find relief, download my FREE Python Concurrency Mind Maps
Further Reading
This section provides additional resources that you may find helpful.
Python Threading Books
- Python Threading Jump-Start, Jason Brownlee (my book!)
- Threading API Interview Questions
- Threading Module API Cheat Sheet
I also recommend specific chapters in the following books:
- Python Cookbook, David Beazley and Brian Jones, 2013.
- See: Chapter 12: Concurrency
- Effective Python, Brett Slatkin, 2019.
- See: Chapter 7: Concurrency and Parallelism
- Python in a Nutshell, Alex Martelli, et al., 2017.
- See: Chapter: 14: Threads and Processes
Guides
- Python Threading: The Complete Guide
- Python ThreadPoolExecutor: The Complete Guide
- Python ThreadPool: The Complete Guide
APIs
References
Takeaways
You now know how to identify and fix a timing-based race condition in Python.
Do you have any questions?
Ask your questions in the comments below and I will do my best to answer.
Leave a Reply