You can gracefully stop a daemon thread using a threading.Event.
In this tutorial you will discover how to stop daemon thread background tasks in Python.
Let’s get started.
Need to Stop a Daemon Thread
A thread is a thread of execution in a computer program.
Every Python program has at least one thread of execution called the main thread. Both processes and threads are created and managed by the underlying operating system.
Sometimes we may need to create additional threads in our program in order to execute code concurrently.
Python provides the ability to create and manage new threads via the threading module and the threading.Thread class.
You can learn more about Python threads in the guide:
In concurrent programming, we may need to run a task in the background using a daemon thread.
The problem with daemon threads is that they will be terminated abruptly by the Python process once all other non-daemon threads are finished. This can cause problems for some background tasks that may need to close and release resources before being closed.
How can we gracefully stop a daemon thread in Python?
Run loops using all CPUs, download your FREE book to learn how.
How to Stop a Daemon Thread
A daemon thread can be stopped gracefully using a threading.Event.
A daemon thread is a background thread. Daemon is pronounced “dee-mon“, like the alternate spelling “demon“.
A thread may be configured to be a daemon or not, and most threads in concurrent programming, including the main thread, are non-daemon threads (not background threads) by default. The difference between daemon threads and non-daemon threads is that the process will exit if only daemon threads are running, whereas it cannot exit if at least one non-daemon thread is running.
As such, daemon threads are helpful for executing tasks in the background to support the non-daemon threads in an application.
If you are new to daemon threads, you can learn more about them here:
Importantly, daemon threads are terminated abruptly once all non daemon threads are finished. This means that any background task running in a daemon thread will not have a chance to close resources and stop gracefully.
A Python threading.Thread instance can be configured to be a daemon thread.
We can configure a new thread to be a daemon thread by specifying the “daemon” argument to True in the constructor of the threading.Thread class.
For example:
|
1 2 3 |
... # create a new daemon thread thread = Thread(daemon=True, ...) |
Typically a background task is a long-running task that may run for the duration of the main program.
This may be implemented using a loop that performs the task periodically based on a time interval or sporadically based on the arrival of events.
For example, we can define a simple background task as a while loop and a sleep interval.
|
1 2 3 4 5 6 7 |
# background task def task(): # run forever while True: # run every 2 seconds sleep(2) # perform task... |
A background task of this type that executes in a loop can be adjusted to check program state as to whether it should shutdown each iteration.
For example, a threading.Event can be used to trigger the background task to shutdown if it is set true.
You may recall that a threading.Event is a thread-safe boolean flag that can be shared between threads.
You can learn more about thread events here:
The main loop of the background task can be configured to check the status of the event each iteration.
For example:
|
1 2 3 4 |
... # check of the task should exit if event.is_set(): return |
Alternatively, the main loop of the background task could be configured to check the status of the event each iteration.
For example:
|
1 2 3 4 5 6 |
... # run until event is set while not event.is_set(): # run every 2 seconds sleep(2) # perform task... |
The main thread can then set the event once the main task of the program has completed, then join (block) on the threading.Thread object until the background task has terminated.
For example:
|
1 2 3 4 5 |
... # request the background thread stop stop_event.set() # wait for the background thread to stop thread.join() |
If you are new to joining thread instances, you can learn more here:
This process of stopping the daemon thread can be automated using the atexit module.
The atexit module allows a function to be registered that will be called by the Python interpreter right before the process is exited.
A new function can be defined that will perform this action for us, setting the event and joining the background thread instance.
The function can take the threading.Event and threading.Thread instances as arguments.
For example:
|
1 2 3 4 5 6 |
# stop the background task gracefully before exit def stop_background(stop_event, thread): # request the background thread stop stop_event.set() # wait for the background thread to stop thread.join() |
Our stop_background() function can then be registered with the atexit module, for example:
|
1 2 3 |
... # register the at exit atexit.register(stop_background, stop_event, thread) |
This will call the function right before the Python process exists, terminating the background daemon thread gracefully.
Now that we know how to stop a background task in a daemon thread, let’s look at some worked examples.
Example of Daemon Thread Stopping Abruptly
Before we explore how to stop a background task in a daemon thread, let’s look at an example where a background task is terminated abruptly.
We will define a background task that will run for the duration of the program and execute every two seconds. In this case, just reporting a message. The main thread will perform its own task for a while to give the background thread a chance to run. Then the main thread will terminate and abruptly stop the background thread.
First, we can define a function named task() that we will execute in the background via a daemon thread.
The function will be composed of a while-loop that loops forever. Each iteration, the function will sleep for 2 seconds, then wake-up and perform the task, which in this case is to simply print a message.
The complete function for the background task is listed below.
|
1 2 3 4 5 6 7 8 |
# background task def task(): # run forever while True: # run every 2 seconds sleep(2) # perform task print('Background performing task') |
Back in the main thread, we can then create a new threading.Thread instance and configure it to execute our task() function and to do so as a daemon thread. Once created, we can start this new thread immediately.
|
1 2 3 4 |
... # create and start the background thread thread = Thread(target=task, daemon=True, name="Background") thread.start() |
We can then simulate work in the main thread for a while to allow the background task a chance to run. This can be achieved with a sleep for ten seconds after which the main thread and the entire Python process will terminate.
|
1 2 3 4 5 |
... # run the main thread for a while print('Main thread running...') sleep(10) print('Main thread stopping') |
Tying this together, the complete example of a daemon thread stopping abruptly is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
# SuperFastPython.com # example of a daemon thread background task that runs in a loop from time import sleep from threading import Thread # background task def task(): # run forever while True: # run every 2 seconds sleep(2) # perform task print('Background performing task') # create and start the background thread thread = Thread(target=task, daemon=True, name="Background") thread.start() # run the main thread for a while print('Main thread running...') sleep(10) print('Main thread stopping') |
Running the example first creates and runs the background thread.
The background threads start running, entering the loop and sleeping for two seconds.
The main thread carries on and then sleeps for ten seconds.
This gives the background daemon thread a chance to run a few times, in this case four times
Finally, the main thread wakes and terminates, which abruptly terminates the background thread.
This highlights that by default, we don’t have any control over how or when the background thread is terminated.
|
1 2 3 4 5 6 |
Main thread running... Background performing task Background performing task Background performing task Background performing task Main thread stopping |
Next, let’s look at how we might add some control over stopping the daemon thread.
Free Python Threading Course
Download your FREE threading PDF cheat sheet and get BONUS access to my free 7-day crash course on the threading API.
Discover how to use the Python threading module including how to create and start new threads and how to use a mutex locks and semaphores
Example of Stopping a Daemon Thread
The example from the previous section can be updated to give control over stopping the daemon thread.
This can be achieved by having the main loop in the background task check a shared threading.Event each iteration and have the main thread set the event at the point that we wish to exit the program. This will allow the background thread to have a controlled termination before the main thread and the entire Python process terminates.
First, we can update the task() function to take a threading.Thread instance as an argument and then to check that the event is not set each iteration of the while-loop.
Once the event is set, the loop will exit and the background task will report a message that it is terminating gracefully.
The updated task() function with these changes is listed below.
|
1 2 3 4 5 6 7 8 9 |
# background task def task(event): # run until the event is set while not event.is_set(): # run every 2 seconds sleep(2) # perform task print('Background performing task') print('Background done') |
Next, in the main thread can create the shared threading.Event object and pass it to the task() function as an argument when configuring our new daemon threading.Thread instance.
|
1 2 3 4 5 6 |
... # prepare state for stopping the background stop_event = Event() # create and start the background thread thread = Thread(target=task, args=(stop_event,), daemon=True, name="Background") thread.start() |
Then, once the application is finished in the main thread and ready to terminate, we can set the event and join the daemon thread, blocking until it has terminated.
|
1 2 3 4 5 6 |
... # request the background thread stop stop_event.set() # wait for the background thread to stop thread.join() print('Main done') |
Tying this together, the complete example of stopping a daemon thread is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 |
# SuperFastPython.com # example of stopping daemon thread background task from time import sleep from threading import Thread from threading import Event # background task def task(event): # run until the event is set while not event.is_set(): # run every 2 seconds sleep(2) # perform task print('Background performing task') print('Background done') # prepare state for stopping the background stop_event = Event() # create and start the background thread thread = Thread(target=task, args=(stop_event,), daemon=True, name="Background") thread.start() # run the main thread for a while print('Main thread running...') sleep(10) print('Main thread stopping') # request the background thread stop stop_event.set() # wait for the background thread to stop thread.join() print('Main done') |
Running the example first creates and starts the daemon thread and runs the background task.
The background task runs in a while-loop, checking if the event is set each iteration, sleeping and printing a message.
The main thread simulates its task and sleeps for ten seconds. It then wakes and sets the event and waits for the background thread to terminate.
The background thread notices the event is set on the next iteration. It exits the loop in the background thread, reports a final message and the thread terminates.
The main thread then terminates and the entire Python process is terminated.
|
1 2 3 4 5 6 7 8 9 |
Main thread running... Background performing task Background performing task Background performing task Background performing task Main thread stopping Background performing task Background done Main done |
Note, the design could be improved further if the background thread waited on a threading.Condition each iteration. This would allow the main thread to notify the daemon thread to wake-up and check the status of the event, avoiding any prolonged waiting in the main thread. This would be especially useful if the background thread only ran every minute or hour.
Next, let’s look at how we might automate the stopping of the daemon thread.
Overwhelmed by the python concurrency APIs?
Find relief, download my FREE Python Concurrency Mind Maps
Example of Stopping a Daemon At Exit
The previous example showed how we can stop a daemon thread using a thread.Event, but it required a manual intervention by the main thread.
We can automate this and leave the main thread free to focus on the main task of the application.
This can be achieved by defining a function that sets the threading.Event and waits on the threading.Thread instance of the background task. We can then have this function called by the Python process automatically just prior to the Process terminating.
This capability is provided by the atexit module and by registering functions to call at the exit of the Python process via the atexit.register() function.
This function takes the name of the callable to execute, as well as any arguments that may need to be passed to the callable.
We can define a function named stop_background() that takes the threading.Event and threading.Thread instances as arguments then sets the event and waits on the thread.
The complete function is listed below.
|
1 2 3 4 5 6 7 8 |
# stop the background task gracefully before exit def stop_background(stop_event, thread): print('At exit stopping') # request the background thread stop stop_event.set() # wait for the background thread to stop thread.join() print('At exit done') |
We can then update the main thread to register this function.
|
1 2 3 |
... # register the at exit atexit.register(stop_background, stop_event, thread) |
And that’s it.
Tying this together, the complete example of stopping a daemon thread at exit is listed below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 |
# SuperFastPython.com # example of stopping daemon thread background task at exit from time import sleep from threading import Thread from threading import Event import atexit # background task def task(event): # run forever while not event.is_set(): # run every 2 seconds sleep(2) # perform task print('Background performing task') print('Background done') # stop the background task gracefully before exit def stop_background(stop_event, thread): print('At exit stopping') # request the background thread stop stop_event.set() # wait for the background thread to stop thread.join() print('At exit done') # prepare state for stopping the background stop_event = Event() # create and start the background thread thread = Thread(target=task, args=(stop_event,), daemon=True, name="Background") thread.start() # register the at exit atexit.register(stop_background, stop_event, thread) # run the main thread for a while print('Main thread running...') sleep(10) print('Main thread stopping') |
Running the example first creates and starts the daemon thread to run the background task.
The background task runs every two seconds as before, checking the status of the event each iteration.
The main thread blocks for ten seconds then terminates.
Right before the Python process is terminated, our stop_background() function is called. This function sets the event and joins the thread instance, waiting for it to terminate.
The background thread notices that the event is set, exits its loop and terminates.
Finally, the Python process is terminated.
This shows how we can automatically trigger background daemon threads to close gracefully.
|
1 2 3 4 5 6 7 8 9 10 |
Main thread running... Background performing task Background performing task Background performing task Background performing task Main thread stopping At exit stopping Background performing task Background done At exit done |
Further Reading
This section provides additional resources that you may find helpful.
Python Threading Books
- Python Threading Jump-Start, Jason Brownlee (my book!)
- Threading API Interview Questions
- Threading Module API Cheat Sheet
I also recommend specific chapters in the following books:
- Python Cookbook, David Beazley and Brian Jones, 2013.
- See: Chapter 12: Concurrency
- Effective Python, Brett Slatkin, 2019.
- See: Chapter 7: Concurrency and Parallelism
- Python in a Nutshell, Alex Martelli, et al., 2017.
- See: Chapter: 14: Threads and Processes
Guides
- Python Threading: The Complete Guide
- Python ThreadPoolExecutor: The Complete Guide
- Python ThreadPool: The Complete Guide
APIs
References
Takeaways
You now know how to stop daemon threads in Python.
Do you have any questions?
Ask your questions in the comments below and I will do my best to answer.
Photo by Matt Bennett on Unsplash
Thanks for the article?
What is the point of doing a .join() on a daemon process? If you’re waiting for that thread to finish before you handoff control to the main thread, then you could’ve just made that thread non-daemon right?
Which would suggest you would only want to spin up daemon thread for tasks that you don’t need to be atomic.
You may want to join a daemon if you expicltly shut it down, e.g. while closing a server or service.