PEP 433 – Easier suppression of file descriptor inheritance
- Author:
- Victor Stinner <vstinner at python.org>
- Status:
- Superseded
- Type:
- Standards Track
- Created:
- 10-Jan-2013
- Python-Version:
- 3.4
- Superseded-By:
- 446
Abstract
Add a new optional cloexec parameter on functions creating file descriptors, add different ways to change default values of this parameter, and add four new functions:
os.get_cloexec(fd)os.set_cloexec(fd, cloexec=True)sys.getdefaultcloexec()sys.setdefaultcloexec(cloexec)
Rationale
A file descriptor has a close-on-exec flag which indicates if the file descriptor will be inherited or not.
On UNIX, if the close-on-exec flag is set, the file descriptor is not inherited: it will be closed at the execution of child processes; otherwise the file descriptor is inherited by child processes.
On Windows, if the close-on-exec flag is set, the file descriptor is not
inherited; the file descriptor is inherited by child processes if the
close-on-exec flag is cleared and if CreateProcess() is called with
the bInheritHandles parameter set to TRUE (when
subprocess.Popen is created with close_fds=False for example).
Windows does not have “close-on-exec” flag but an inheritance flag which
is just the opposite value. For example, setting close-on-exec flag
means clearing the HANDLE_FLAG_INHERIT flag of a handle.
Status in Python 3.3
On UNIX, the subprocess module closes file descriptors greater than 2 by default since Python 3.2 [1]. All file descriptors created by the parent process are automatically closed in the child process.
xmlrpc.server.SimpleXMLRPCServer sets the close-on-exec flag of
the listening socket, the parent class socketserver.TCPServer
does not set this flag.
There are other cases creating a subprocess or executing a new program
where file descriptors are not closed: functions of the os.spawn*()
and the os.exec*() families and third party modules calling
exec() or fork() + exec(). In this case, file descriptors
are shared between the parent and the child processes which is usually
unexpected and causes various issues.
This PEP proposes to continue the work started with the change in the subprocess in Python 3.2, to fix the issue in any code, and not just code using subprocess.
Inherited file descriptors issues
Closing the file descriptor in the parent process does not close the related resource (file, socket, …) because it is still open in the child process.
The listening socket of TCPServer is not closed on exec(): the child
process is able to get connection from new clients; if the parent closes
the listening socket and create a new listening socket on the same
address, it would get an “address already is used” error.
Not closing file descriptors can lead to resource exhaustion: even if the parent closes all files, creating a new file descriptor may fail with “too many files” because files are still open in the child process.
See also the following issues:
- Issue #2320: Race condition in subprocess using stdin (2008)
- Issue #3006: subprocess.Popen causes socket to remain open after close (2008)
- Issue #7213: subprocess leaks open file descriptors between Popen instances causing hangs (2009)
- Issue #12786: subprocess wait() hangs when stdin is closed (2011)
Security
Leaking file descriptors is a major security vulnerability. An untrusted child process can read sensitive data like passwords and take control of the parent process though leaked file descriptors. It is for example a known vulnerability to escape from a chroot.
See also the CERT recommendation: