source: trunk/essentials/dev-lang/python/Lib/subprocess.py@ 3951

Last change on this file since 3951 was 3225, checked in by bird, 19 years ago

Python 2.5
File size: 41.4 KB
Line 
1# subprocess - Subprocesses with accessible I/O streams
2#
3# For more information about this module, see PEP 324.
4#
5# This module should remain compatible with Python 2.2, see PEP 291.
6#
7# Copyright (c) 2003-2005 by Peter Astrand <[email protected]>
8#
9# Licensed to PSF under a Contributor Agreement.
10# See http://www.python.org/2.4/license for licensing details.
11
12r"""subprocess - Subprocesses with accessible I/O streams
13
14This module allows you to spawn processes, connect to their
15input/output/error pipes, and obtain their return codes. This module
16intends to replace several other, older modules and functions, like:
17
18os.system
19os.spawn*
20os.popen*
21popen2.*
22commands.*
23
24Information about how the subprocess module can be used to replace these
25modules and functions can be found below.
26
27
28
29Using the subprocess module
30===========================
31This module defines one class called Popen:
32
33class Popen(args, bufsize=0, executable=None,
34 stdin=None, stdout=None, stderr=None,
35 preexec_fn=None, close_fds=False, shell=False,
36 cwd=None, env=None, universal_newlines=False,
37 startupinfo=None, creationflags=0):
38
39
40Arguments are:
41
42args should be a string, or a sequence of program arguments. The
43program to execute is normally the first item in the args sequence or
44string, but can be explicitly set by using the executable argument.
45
46On UNIX, with shell=False (default): In this case, the Popen class
47uses os.execvp() to execute the child program. args should normally
48be a sequence. A string will be treated as a sequence with the string
49as the only item (the program to execute).
50
51On UNIX, with shell=True: If args is a string, it specifies the
52command string to execute through the shell. If args is a sequence,
53the first item specifies the command string, and any additional items
54will be treated as additional shell arguments.
55
56On Windows: the Popen class uses CreateProcess() to execute the child
57program, which operates on strings. If args is a sequence, it will be
58converted to a string using the list2cmdline method. Please note that
59not all MS Windows applications interpret the command line the same
60way: The list2cmdline is designed for applications using the same
61rules as the MS C runtime.
62
63bufsize, if given, has the same meaning as the corresponding argument
64to the built-in open() function: 0 means unbuffered, 1 means line
65buffered, any other positive value means use a buffer of
66(approximately) that size. A negative bufsize means to use the system
67default, which usually means fully buffered. The default value for
68bufsize is 0 (unbuffered).
69
70stdin, stdout and stderr specify the executed programs' standard
71input, standard output and standard error file handles, respectively.
72Valid values are PIPE, an existing file descriptor (a positive
73integer), an existing file object, and None. PIPE indicates that a
74new pipe to the child should be created. With None, no redirection
75will occur; the child's file handles will be inherited from the
76parent. Additionally, stderr can be STDOUT, which indicates that the
77stderr data from the applications should be captured into the same
78file handle as for stdout.
79
80If preexec_fn is set to a callable object, this object will be called
81in the child process just before the child is executed.
82
83If close_fds is true, all file descriptors except 0, 1 and 2 will be
84closed before the child process is executed.
85
86if shell is true, the specified command will be executed through the
87shell.
88
89If cwd is not None, the current directory will be changed to cwd
90before the child is executed.
91
92If env is not None, it defines the environment variables for the new
93process.
94
95If universal_newlines is true, the file objects stdout and stderr are
96opened as a text files, but lines may be terminated by any of '\n',
97the Unix end-of-line convention, '\r', the Macintosh convention or
98'\r\n', the Windows convention. All of these external representations
99are seen as '\n' by the Python program. Note: This feature is only
100available if Python is built with universal newline support (the
101default). Also, the newlines attribute of the file objects stdout,
102stdin and stderr are not updated by the communicate() method.
103
104The startupinfo and creationflags, if given, will be passed to the
105underlying CreateProcess() function. They can specify things such as
106appearance of the main window and priority for the new process.
107(Windows only)
108
109
110This module also defines two shortcut functions:
111
112call(*popenargs, **kwargs):
113 Run command with arguments. Wait for command to complete, then
114 return the returncode attribute.
115
116 The arguments are the same as for the Popen constructor. Example:
117
118 retcode = call(["ls", "-l"])
119
120check_call(*popenargs, **kwargs):
121 Run command with arguments. Wait for command to complete. If the
122 exit code was zero then return, otherwise raise
123 CalledProcessError. The CalledProcessError object will have the
124 return code in the returncode attribute.
125
126 The arguments are the same as for the Popen constructor. Example:
127
128 check_call(["ls", "-l"])
129
130Exceptions
131----------
132Exceptions raised in the child process, before the new program has
133started to execute, will be re-raised in the parent. Additionally,
134the exception object will have one extra attribute called
135'child_traceback', which is a string containing traceback information
136from the childs point of view.
137
138The most common exception raised is OSError. This occurs, for
139example, when trying to execute a non-existent file. Applications
140should prepare for OSErrors.
141
142A ValueError will be raised if Popen is called with invalid arguments.
143
144check_call() will raise CalledProcessError, if the called process
145returns a non-zero return code.
146
147
148Security
149--------
150Unlike some other popen functions, this implementation will never call
151/bin/sh implicitly. This means that all characters, including shell
152metacharacters, can safely be passed to child processes.
153
154
155Popen objects
156=============
157Instances of the Popen class have the following methods:
158
159poll()
160 Check if child process has terminated. Returns returncode
161 attribute.
162
163wait()
164 Wait for child process to terminate. Returns returncode attribute.
165
166communicate(input=None)
167 Interact with process: Send data to stdin. Read data from stdout
168 and stderr, until end-of-file is reached. Wait for process to
169 terminate. The optional stdin argument should be a string to be
170 sent to the child process, or None, if no data should be sent to
171 the child.
172
173 communicate() returns a tuple (stdout, stderr).
174
175 Note: The data read is buffered in memory, so do not use this
176 method if the data size is large or unlimited.
177
178The following attributes are also available:
179
180stdin
181 If the stdin argument is PIPE, this attribute is a file object
182 that provides input to the child process. Otherwise, it is None.
183
184stdout
185 If the stdout argument is PIPE, this attribute is a file object
186 that provides output from the child process. Otherwise, it is
187 None.
188
189stderr
190 If the stderr argument is PIPE, this attribute is file object that
191 provides error output from the child process. Otherwise, it is
192 None.
193
194pid
195 The process ID of the child process.
196
197returncode
198 The child return code. A None value indicates that the process
199 hasn't terminated yet. A negative value -N indicates that the
200 child was terminated by signal N (UNIX only).
201
202
203Replacing older functions with the subprocess module
204====================================================
205In this section, "a ==> b" means that b can be used as a replacement
206for a.
207
208Note: All functions in this section fail (more or less) silently if
209the executed program cannot be found; this module raises an OSError
210exception.
211
212In the following examples, we assume that the subprocess module is
213imported with "from subprocess import *".
214
215
216Replacing /bin/sh shell backquote
217---------------------------------
218output=`mycmd myarg`
219==>
220output = Popen(["mycmd", "myarg"], stdout=PIPE).communicate()[0]
221
222
223Replacing shell pipe line
224-------------------------
225output=`dmesg | grep hda`
226==>
227p1 = Popen(["dmesg"], stdout=PIPE)
228p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE)
229output = p2.communicate()[0]
230
231
232Replacing os.system()
233---------------------
234sts = os.system("mycmd" + " myarg")
235==>
236p = Popen("mycmd" + " myarg", shell=True)
237pid, sts = os.waitpid(p.pid, 0)
238
239Note:
240
241* Calling the program through the shell is usually not required.
242
243* It's easier to look at the returncode attribute than the
244 exitstatus.
245
246A more real-world example would look like this:
247
248try:
249 retcode = call("mycmd" + " myarg", shell=True)
250 if retcode < 0:
251 print >>sys.stderr, "Child was terminated by signal", -retcode
252 else:
253 print >>sys.stderr, "Child returned", retcode
254except OSError, e:
255 print >>sys.stderr, "Execution failed:", e
256
257
258Replacing os.spawn*
259-------------------
260P_NOWAIT example:
261
262pid = os.spawnlp(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg")
263==>
264pid = Popen(["/bin/mycmd", "myarg"]).pid
265
266
267P_WAIT example:
268
269retcode = os.spawnlp(os.P_WAIT, "/bin/mycmd", "mycmd", "myarg")
270==>
271retcode = call(["/bin/mycmd", "myarg"])
272
273
274Vector example:
275
276os.spawnvp(os.P_NOWAIT, path, args)
277==>
278Popen([path] + args[1:])
279
280
281Environment example:
282
283os.spawnlpe(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg", env)
284==>
285Popen(["/bin/mycmd", "myarg"], env={"PATH": "/usr/bin"})
286
287
288Replacing os.popen*
289-------------------
290pipe = os.popen(cmd, mode='r', bufsize)
291==>
292pipe = Popen(cmd, shell=True, bufsize=bufsize, stdout=PIPE).stdout
293
294pipe = os.popen(cmd, mode='w', bufsize)
295==>
296pipe = Popen(cmd, shell=True, bufsize=bufsize, stdin=PIPE).stdin
297
298
299(child_stdin, child_stdout) = os.popen2(cmd, mode, bufsize)
300==>
301p = Popen(cmd, shell=True, bufsize=bufsize,
302 stdin=PIPE, stdout=PIPE, close_fds=True)
303(child_stdin, child_stdout) = (p.stdin, p.stdout)
304
305
306(child_stdin,
307 child_stdout,
308 child_stderr) = os.popen3(cmd, mode, bufsize)
309==>
310p = Popen(cmd, shell=True, bufsize=bufsize,
311 stdin=PIPE, stdout=PIPE, stderr=PIPE, close_fds=True)
312(child_stdin,
313 child_stdout,
314 child_stderr) = (p.stdin, p.stdout, p.stderr)
315
316
317(child_stdin, child_stdout_and_stderr) = os.popen4(cmd, mode, bufsize)
318==>
319p = Popen(cmd, shell=True, bufsize=bufsize,
320 stdin=PIPE, stdout=PIPE, stderr=STDOUT, close_fds=True)
321(child_stdin, child_stdout_and_stderr) = (p.stdin, p.stdout)
322
323
324Replacing popen2.*
325------------------
326Note: If the cmd argument to popen2 functions is a string, the command
327is executed through /bin/sh. If it is a list, the command is directly
328executed.
329
330(child_stdout, child_stdin) = popen2.popen2("somestring", bufsize, mode)
331==>
332p = Popen(["somestring"], shell=True, bufsize=bufsize
333 stdin=PIPE, stdout=PIPE, close_fds=True)
334(child_stdout, child_stdin) = (p.stdout, p.stdin)
335
336
337(child_stdout, child_stdin) = popen2.popen2(["mycmd", "myarg"], bufsize, mode)
338==>
339p = Popen(["mycmd", "myarg"], bufsize=bufsize,
340 stdin=PIPE, stdout=PIPE, close_fds=True)
341(child_stdout, child_stdin) = (p.stdout, p.stdin)
342
343The popen2.Popen3 and popen3.Popen4 basically works as subprocess.Popen,
344except that:
345
346* subprocess.Popen raises an exception if the execution fails
347* the capturestderr argument is replaced with the stderr argument.
348* stdin=PIPE and stdout=PIPE must be specified.
349* popen2 closes all filedescriptors by default, but you have to specify
350 close_fds=True with subprocess.Popen.
351
352
353"""
354
355import sys
356mswindows = (sys.platform == "win32")
357
358import os
359import types
360import traceback
361
362# Exception classes used by this module.
363class CalledProcessError(Exception):
364 """This exception is raised when a process run by check_call() returns
365 a non-zero exit status. The exit status will be stored in the
366 returncode attribute."""
367 def __init__(self, returncode, cmd):
368 self.returncode = returncode
369 self.cmd = cmd
370 def __str__(self):
371 return "Command '%s' returned non-zero exit status %d" % (self.cmd, self.returncode)
372
373
374if mswindows:
375 import threading
376 import msvcrt
377 if 0: # <-- change this to use pywin32 instead of the _subprocess driver
378 import pywintypes
379 from win32api import GetStdHandle, STD_INPUT_HANDLE, \
380 STD_OUTPUT_HANDLE, STD_ERROR_HANDLE
381 from win32api import GetCurrentProcess, DuplicateHandle, \
382 GetModuleFileName, GetVersion
383 from win32con import DUPLICATE_SAME_ACCESS, SW_HIDE
384 from win32pipe import CreatePipe
385 from win32process import CreateProcess, STARTUPINFO, \
386 GetExitCodeProcess, STARTF_USESTDHANDLES, \
387 STARTF_USESHOWWINDOW, CREATE_NEW_CONSOLE
388 from win32event import WaitForSingleObject, INFINITE, WAIT_OBJECT_0
389 else:
390 from _subprocess import *
391 class STARTUPINFO:
392 dwFlags = 0
393 hStdInput = None
394 hStdOutput = None
395 hStdError = None
396 wShowWindow = 0
397 class pywintypes:
398 error = IOError
399else:
400 import select
401 import errno
402 import fcntl
403 import pickle
404
405__all__ = ["Popen", "PIPE", "STDOUT", "call", "check_call", "CalledProcessError"]
406
407try:
408 MAXFD = os.sysconf("SC_OPEN_MAX")
409except:
410 MAXFD = 256
411
412# True/False does not exist on 2.2.0
413try:
414 False
415except NameError:
416 False = 0
417 True = 1
418
419_active = []
420
421def _cleanup():
422 for inst in _active[:]:
423 if inst.poll(_deadstate=sys.maxint) >= 0:
424 try:
425 _active.remove(inst)
426 except ValueError:
427 # This can happen if two threads create a new Popen instance.
428 # It's harmless that it was already removed, so ignore.
429 pass
430
431PIPE = -1
432STDOUT = -2
433
434
435def call(*popenargs, **kwargs):
436 """Run command with arguments. Wait for command to complete, then
437 return the returncode attribute.
438
439 The arguments are the same as for the Popen constructor. Example:
440
441 retcode = call(["ls", "-l"])
442 """
443 return Popen(*popenargs, **kwargs).wait()
444
445
446def check_call(*popenargs, **kwargs):
447 """Run command with arguments. Wait for command to complete. If
448 the exit code was zero then return, otherwise raise
449 CalledProcessError. The CalledProcessError object will have the
450 return code in the returncode attribute.
451
452 The arguments are the same as for the Popen constructor. Example:
453
454 check_call(["ls", "-l"])
455 """
456 retcode = call(*popenargs, **kwargs)
457 cmd = kwargs.get("args")
458 if cmd is None:
459 cmd = popenargs[0]
460 if retcode:
461 raise CalledProcessError(retcode, cmd)
462 return retcode
463
464
465def list2cmdline(seq):
466 """
467 Translate a sequence of arguments into a command line
468 string, using the same rules as the MS C runtime:
469
470 1) Arguments are delimited by white space, which is either a
471 space or a tab.
472
473 2) A string surrounded by double quotation marks is
474 interpreted as a single argument, regardless of white space
475 contained within. A quoted string can be embedded in an
476 argument.
477
478 3) A double quotation mark preceded by a backslash is
479 interpreted as a literal double quotation mark.
480
481 4) Backslashes are interpreted literally, unless they
482 immediately precede a double quotation mark.
483
484 5) If backslashes immediately precede a double quotation mark,
485 every pair of backslashes is interpreted as a literal
486 backslash. If the number of backslashes is odd, the last
487 backslash escapes the next double quotation mark as
488 described in rule 3.
489 """
490
491 # See
492 # http://msdn.microsoft.com/library/en-us/vccelng/htm/progs_12.asp
493 result = []
494 needquote = False
495 for arg in seq:
496 bs_buf = []
497
498 # Add a space to separate this argument from the others
499 if result:
500 result.append(' ')
501
502 needquote = (" " in arg) or ("\t" in arg)
503 if needquote:
504 result.append('"')
505
506 for c in arg:
507 if c == '\\':
508 # Don't know if we need to double yet.
509 bs_buf.append(c)
510 elif c == '"':
511 # Double backspaces.
512 result.append('\\' * len(bs_buf)*2)
513 bs_buf = []
514 result.append('\\"')
515 else:
516 # Normal char
517 if bs_buf:
518 result.extend(bs_buf)
519 bs_buf = []
520 result.append(c)
521
522 # Add remaining backspaces, if any.
523 if bs_buf:
524 result.extend(bs_buf)
525
526 if needquote:
527 result.extend(bs_buf)
528 result.append('"')
529
530 return ''.join(result)
531
532
533class Popen(object):
534 def __init__(self, args, bufsize=0, executable=None,
535 stdin=None, stdout=None, stderr=None,
536 preexec_fn=None, close_fds=False, shell=False,
537 cwd=None, env=None, universal_newlines=False,
538 startupinfo=None, creationflags=0):
539 """Create new Popen instance."""
540 _cleanup()
541
542 self._child_created = False
543 if not isinstance(bufsize, (int, long)):
544 raise TypeError("bufsize must be an integer")
545
546 if mswindows:
547 if preexec_fn is not None:
548 raise ValueError("preexec_fn is not supported on Windows "
549 "platforms")
550 if close_fds:
551 raise ValueError("close_fds is not supported on Windows "
552 "platforms")
553 else:
554 # POSIX
555 if startupinfo is not None:
556 raise ValueError("startupinfo is only supported on Windows "
557 "platforms")
558 if creationflags != 0:
559 raise ValueError("creationflags is only supported on Windows "
560 "platforms")
561
562 self.stdin = None
563 self.stdout = None
564 self.stderr = None
565 self.pid = None
566 self.returncode = None
567 self.universal_newlines = universal_newlines
568
569 # Input and output objects. The general principle is like
570 # this:
571 #
572 # Parent Child
573 # ------ -----
574 # p2cwrite ---stdin---> p2cread
575 # c2pread <--stdout--- c2pwrite
576 # errread <--stderr--- errwrite
577 #
578 # On POSIX, the child objects are file descriptors. On
579 # Windows, these are Windows file handles. The parent objects
580 # are file descriptors on both platforms. The parent objects
581 # are None when not using PIPEs. The child objects are None
582 # when not redirecting.
583
584 (p2cread, p2cwrite,
585 c2pread, c2pwrite,
586 errread, errwrite) = self._get_handles(stdin, stdout, stderr)
587
588 self._execute_child(args, executable, preexec_fn, close_fds,
589 cwd, env, universal_newlines,
590 startupinfo, creationflags, shell,
591 p2cread, p2cwrite,
592 c2pread, c2pwrite,
593 errread, errwrite)
594
595 if p2cwrite:
596 self.stdin = os.fdopen(p2cwrite, 'wb', bufsize)
597 if c2pread:
598 if universal_newlines:
599 self.stdout = os.fdopen(c2pread, 'rU', bufsize)
600 else:
601 self.stdout = os.fdopen(c2pread, 'rb', bufsize)
602 if errread:
603 if universal_newlines:
604 self.stderr = os.fdopen(errread, 'rU', bufsize)
605 else:
606 self.stderr = os.fdopen(errread, 'rb', bufsize)
607
608
609 def _translate_newlines(self, data):
610 data = data.replace("\r\n", "\n")
611 data = data.replace("\r", "\n")
612 return data
613
614
615 def __del__(self):
616 if not self._child_created:
617 # We didn't get to successfully create a child process.
618 return
619 # In case the child hasn't been waited on, check if it's done.
620 self.poll(_deadstate=sys.maxint)
621 if self.returncode is None and _active is not None:
622 # Child is still running, keep us alive until we can wait on it.
623 _active.append(self)
624
625
626 def communicate(self, input=None):
627 """Interact with process: Send data to stdin. Read data from
628 stdout and stderr, until end-of-file is reached. Wait for
629 process to terminate. The optional input argument should be a
630 string to be sent to the child process, or None, if no data
631 should be sent to the child.
632
633 communicate() returns a tuple (stdout, stderr)."""
634
635 # Optimization: If we are only using one pipe, or no pipe at
636 # all, using select() or threads is unnecessary.
637 if [self.stdin, self.stdout, self.stderr].count(None) >= 2:
638 stdout = None
639 stderr = None
640 if self.stdin:
641 if input:
642 self.stdin.write(input)
643 self.stdin.close()
644 elif self.stdout:
645 stdout = self.stdout.read()
646 elif self.stderr:
647 stderr = self.stderr.read()
648 self.wait()
649 return (stdout, stderr)
650
651 return self._communicate(input)
652
653
654 if mswindows:
655 #
656 # Windows methods
657 #
658 def _get_handles(self, stdin, stdout, stderr):
659 """Construct and return tupel with IO objects:
660 p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite
661 """
662 if stdin is None and stdout is None and stderr is None:
663 return (None, None, None, None, None, None)
664
665 p2cread, p2cwrite = None, None
666 c2pread, c2pwrite = None, None
667 errread, errwrite = None, None
668
669 if stdin is None:
670 p2cread = GetStdHandle(STD_INPUT_HANDLE)
671 elif stdin == PIPE:
672 p2cread, p2cwrite = CreatePipe(None, 0)
673 # Detach and turn into fd
674 p2cwrite = p2cwrite.Detach()
675 p2cwrite = msvcrt.open_osfhandle(p2cwrite, 0)
676 elif isinstance(stdin, int):
677 p2cread = msvcrt.get_osfhandle(stdin)
678 else:
679 # Assuming file-like object
680 p2cread = msvcrt.get_osfhandle(stdin.fileno())
681 p2cread = self._make_inheritable(p2cread)
682
683 if stdout is None:
684 c2pwrite = GetStdHandle(STD_OUTPUT_HANDLE)
685 elif stdout == PIPE:
686 c2pread, c2pwrite = CreatePipe(None, 0)
687 # Detach and turn into fd
688 c2pread = c2pread.Detach()
689 c2pread = msvcrt.open_osfhandle(c2pread, 0)
690 elif isinstance(stdout, int):
691 c2pwrite = msvcrt.get_osfhandle(stdout)
692 else:
693 # Assuming file-like object
694 c2pwrite = msvcrt.get_osfhandle(stdout.fileno())
695 c2pwrite = self._make_inheritable(c2pwrite)
696
697 if stderr is None:
698 errwrite = GetStdHandle(STD_ERROR_HANDLE)
699 elif stderr == PIPE:
700 errread, errwrite = CreatePipe(None, 0)
701 # Detach and turn into fd
702 errread = errread.Detach()
703 errread = msvcrt.open_osfhandle(errread, 0)
704 elif stderr == STDOUT:
705 errwrite = c2pwrite
706 elif isinstance(stderr, int):
707 errwrite = msvcrt.get_osfhandle(stderr)
708 else:
709 # Assuming file-like object
710 errwrite = msvcrt.get_osfhandle(stderr.fileno())
711 errwrite = self._make_inheritable(errwrite)
712
713 return (p2cread, p2cwrite,
714 c2pread, c2pwrite,
715 errread, errwrite)
716
717
718 def _make_inheritable(self, handle):
719 """Return a duplicate of handle, which is inheritable"""
720 return DuplicateHandle(GetCurrentProcess(), handle,
721 GetCurrentProcess(), 0, 1,
722 DUPLICATE_SAME_ACCESS)
723
724
725 def _find_w9xpopen(self):
726 """Find and return absolut path to w9xpopen.exe"""
727 w9xpopen = os.path.join(os.path.dirname(GetModuleFileName(0)),
728 "w9xpopen.exe")
729 if not os.path.exists(w9xpopen):
730 # Eeek - file-not-found - possibly an embedding
731 # situation - see if we can locate it in sys.exec_prefix
732 w9xpopen = os.path.join(os.path.dirname(sys.exec_prefix),
733 "w9xpopen.exe")
734 if not os.path.exists(w9xpopen):
735 raise RuntimeError("Cannot locate w9xpopen.exe, which is "
736 "needed for Popen to work with your "
737 "shell or platform.")
738 return w9xpopen
739
740
741 def _execute_child(self, args, executable, preexec_fn, close_fds,
742 cwd, env, universal_newlines,
743 startupinfo, creationflags, shell,
744 p2cread, p2cwrite,
745 c2pread, c2pwrite,
746 errread, errwrite):
747 """Execute program (MS Windows version)"""
748
749 if not isinstance(args, types.StringTypes):
750 args = list2cmdline(args)
751
752 # Process startup details
753 if startupinfo is None:
754 startupinfo = STARTUPINFO()
755 if None not in (p2cread, c2pwrite, errwrite):
756 startupinfo.dwFlags |= STARTF_USESTDHANDLES
757 startupinfo.hStdInput = p2cread
758 startupinfo.hStdOutput = c2pwrite
759 startupinfo.hStdError = errwrite
760
761 if shell:
762 startupinfo.dwFlags |= STARTF_USESHOWWINDOW
763 startupinfo.wShowWindow = SW_HIDE
764 comspec = os.environ.get("COMSPEC", "cmd.exe")
765 args = comspec + " /c " + args
766 if (GetVersion() >= 0x80000000L or
767 os.path.basename(comspec).lower() == "command.com"):
768 # Win9x, or using command.com on NT. We need to
769 # use the w9xpopen intermediate program. For more
770 # information, see KB Q150956
771 # (http://web.archive.org/web/20011105084002/http://support.microsoft.com/support/kb/articles/Q150/9/56.asp)
772 w9xpopen = self._find_w9xpopen()
773 args = '"%s" %s' % (w9xpopen, args)
774 # Not passing CREATE_NEW_CONSOLE has been known to
775 # cause random failures on win9x. Specifically a
776 # dialog: "Your program accessed mem currently in
777 # use at xxx" and a hopeful warning about the
778 # stability of your system. Cost is Ctrl+C wont
779 # kill children.
780 creationflags |= CREATE_NEW_CONSOLE
781
782 # Start the process
783 try:
784 hp, ht, pid, tid = CreateProcess(executable, args,
785 # no special security
786 None, None,
787 # must inherit handles to pass std
788 # handles
789 1,
790 creationflags,
791 env,
792 cwd,
793 startupinfo)
794 except pywintypes.error, e:
795 # Translate pywintypes.error to WindowsError, which is
796 # a subclass of OSError. FIXME: We should really
797 # translate errno using _sys_errlist (or simliar), but
798 # how can this be done from Python?
799 raise WindowsError(*e.args)
800
801 # Retain the process handle, but close the thread handle
802 self._child_created = True
803 self._handle = hp
804 self.pid = pid
805 ht.Close()
806
807 # Child is launched. Close the parent's copy of those pipe
808 # handles that only the child should have open. You need
809 # to make sure that no handles to the write end of the
810 # output pipe are maintained in this process or else the
811 # pipe will not close when the child process exits and the
812 # ReadFile will hang.
813 if p2cread is not None:
814 p2cread.Close()
815 if c2pwrite is not None:
816 c2pwrite.Close()
817 if errwrite is not None:
818 errwrite.Close()
819
820
821 def poll(self, _deadstate=None):
822 """Check if child process has terminated. Returns returncode
823 attribute."""
824 if self.returncode is None:
825 if WaitForSingleObject(self._handle, 0) == WAIT_OBJECT_0:
826 self.returncode = GetExitCodeProcess(self._handle)
827 return self.returncode
828
829
830 def wait(self):
831 """Wait for child process to terminate. Returns returncode
832 attribute."""
833 if self.returncode is None:
834 obj = WaitForSingleObject(self._handle, INFINITE)
835 self.returncode = GetExitCodeProcess(self._handle)
836 return self.returncode
837
838
839 def _readerthread(self, fh, buffer):
840 buffer.append(fh.read())
841
842
843 def _communicate(self, input):
844 stdout = None # Return
845 stderr = None # Return
846
847 if self.stdout:
848 stdout = []
849 stdout_thread = threading.Thread(target=self._readerthread,
850 args=(self.stdout, stdout))
851 stdout_thread.setDaemon(True)
852 stdout_thread.start()
853 if self.stderr:
854 stderr = []
855 stderr_thread = threading.Thread(target=self._readerthread,
856 args=(self.stderr, stderr))
857 stderr_thread.setDaemon(True)
858 stderr_thread.start()
859
860 if self.stdin:
861 if input is not None:
862 self.stdin.write(input)
863 self.stdin.close()
864
865 if self.stdout:
866 stdout_thread.join()
867 if self.stderr:
868 stderr_thread.join()
869
870 # All data exchanged. Translate lists into strings.
871 if stdout is not None:
872 stdout = stdout[0]
873 if stderr is not None:
874 stderr = stderr[0]
875
876 # Translate newlines, if requested. We cannot let the file
877 # object do the translation: It is based on stdio, which is
878 # impossible to combine with select (unless forcing no
879 # buffering).
880 if self.universal_newlines and hasattr(file, 'newlines'):
881 if stdout:
882 stdout = self._translate_newlines(stdout)
883 if stderr:
884 stderr = self._translate_newlines(stderr)
885
886 self.wait()
887 return (stdout, stderr)
888
889 else:
890 #
891 # POSIX methods
892 #
893 def _get_handles(self, stdin, stdout, stderr):
894 """Construct and return tupel with IO objects:
895 p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite
896 """
897 p2cread, p2cwrite = None, None
898 c2pread, c2pwrite = None, None
899 errread, errwrite = None, None
900
901 if stdin is None:
902 pass
903 elif stdin == PIPE:
904 p2cread, p2cwrite = os.pipe()
905 elif isinstance(stdin, int):
906 p2cread = stdin
907 else:
908 # Assuming file-like object
909 p2cread = stdin.fileno()
910
911 if stdout is None:
912 pass
913 elif stdout == PIPE:
914 c2pread, c2pwrite = os.pipe()
915 elif isinstance(stdout, int):
916 c2pwrite = stdout
917 else:
918 # Assuming file-like object
919 c2pwrite = stdout.fileno()
920
921 if stderr is None:
922 pass
923 elif stderr == PIPE:
924 errread, errwrite = os.pipe()
925 elif stderr == STDOUT:
926 errwrite = c2pwrite
927 elif isinstance(stderr, int):
928 errwrite = stderr
929 else:
930 # Assuming file-like object
931 errwrite = stderr.fileno()
932
933 return (p2cread, p2cwrite,
934 c2pread, c2pwrite,
935 errread, errwrite)
936
937
938 def _set_cloexec_flag(self, fd):
939 try:
940 cloexec_flag = fcntl.FD_CLOEXEC
941 except AttributeError:
942 cloexec_flag = 1
943
944 old = fcntl.fcntl(fd, fcntl.F_GETFD)
945 fcntl.fcntl(fd, fcntl.F_SETFD, old | cloexec_flag)
946
947
948 def _close_fds(self, but):
949 for i in xrange(3, MAXFD):
950 if i == but:
951 continue
952 try:
953 os.close(i)
954 except:
955 pass
956
957
958 def _execute_child(self, args, executable, preexec_fn, close_fds,
959 cwd, env, universal_newlines,
960 startupinfo, creationflags, shell,
961 p2cread, p2cwrite,
962 c2pread, c2pwrite,
963 errread, errwrite):
964 """Execute program (POSIX version)"""
965
966 if isinstance(args, types.StringTypes):
967 args = [args]
968
969 if shell:
970 args = ["/bin/sh", "-c"] + args
971
972 if executable is None:
973 executable = args[0]
974
975 # For transferring possible exec failure from child to parent
976 # The first char specifies the exception type: 0 means
977 # OSError, 1 means some other error.
978 errpipe_read, errpipe_write = os.pipe()
979 self._set_cloexec_flag(errpipe_write)
980
981 self.pid = os.fork()
982 self._child_created = True
983 if self.pid == 0:
984 # Child
985 try:
986 # Close parent's pipe ends
987 if p2cwrite:
988 os.close(p2cwrite)
989 if c2pread:
990 os.close(c2pread)
991 if errread:
992 os.close(errread)
993 os.close(errpipe_read)
994
995 # Dup fds for child
996 if p2cread:
997 os.dup2(p2cread, 0)
998 if c2pwrite:
999 os.dup2(c2pwrite, 1)
1000 if errwrite:
1001 os.dup2(errwrite, 2)
1002
1003 # Close pipe fds. Make sure we don't close the same
1004 # fd more than once, or standard fds.
1005 for fd in set((p2cread, c2pwrite, errwrite))-set((0,1,2)):
1006 if fd: os.close(fd)
1007
1008 # Close all other fds, if asked for
1009 if close_fds:
1010 self._close_fds(but=errpipe_write)
1011
1012 if cwd is not None:
1013 os.chdir(cwd)
1014
1015 if preexec_fn:
1016 apply(preexec_fn)
1017
1018 if env is None:
1019 os.execvp(executable, args)
1020 else:
1021 os.execvpe(executable, args, env)
1022
1023 except:
1024 exc_type, exc_value, tb = sys.exc_info()
1025 # Save the traceback and attach it to the exception object
1026 exc_lines = traceback.format_exception(exc_type,
1027 exc_value,
1028 tb)
1029 exc_value.child_traceback = ''.join(exc_lines)
1030 os.write(errpipe_write, pickle.dumps(exc_value))
1031
1032 # This exitcode won't be reported to applications, so it
1033 # really doesn't matter what we return.
1034 os._exit(255)
1035
1036 # Parent
1037 os.close(errpipe_write)
1038 if p2cread and p2cwrite:
1039 os.close(p2cread)
1040 if c2pwrite and c2pread:
1041 os.close(c2pwrite)
1042 if errwrite and errread:
1043 os.close(errwrite)
1044
1045 # Wait for exec to fail or succeed; possibly raising exception
1046 data = os.read(errpipe_read, 1048576) # Exceptions limited to 1 MB
1047 os.close(errpipe_read)
1048 if data != "":
1049 os.waitpid(self.pid, 0)
1050 child_exception = pickle.loads(data)
1051 raise child_exception
1052
1053
1054 def _handle_exitstatus(self, sts):
1055 if os.WIFSIGNALED(sts):
1056 self.returncode = -os.WTERMSIG(sts)
1057 elif os.WIFEXITED(sts):
1058 self.returncode = os.WEXITSTATUS(sts)
1059 else:
1060 # Should never happen
1061 raise RuntimeError("Unknown child exit status!")
1062
1063
1064 def poll(self, _deadstate=None):
1065 """Check if child process has terminated. Returns returncode
1066 attribute."""
1067 if self.returncode is None:
1068 try:
1069 pid, sts = os.waitpid(self.pid, os.WNOHANG)
1070 if pid == self.pid:
1071 self._handle_exitstatus(sts)
1072 except os.error:
1073 if _deadstate is not None:
1074 self.returncode = _deadstate
1075 return self.returncode
1076
1077
1078 def wait(self):
1079 """Wait for child process to terminate. Returns returncode
1080 attribute."""
1081 if self.returncode is None:
1082 pid, sts = os.waitpid(self.pid, 0)
1083 self._handle_exitstatus(sts)
1084 return self.returncode
1085
1086
1087 def _communicate(self, input):
1088 read_set = []
1089 write_set = []
1090 stdout = None # Return
1091 stderr = None # Return
1092
1093 if self.stdin:
1094 # Flush stdio buffer. This might block, if the user has
1095 # been writing to .stdin in an uncontrolled fashion.
1096 self.stdin.flush()
1097 if input:
1098 write_set.append(self.stdin)
1099 else:
1100 self.stdin.close()
1101 if self.stdout:
1102 read_set.append(self.stdout)
1103 stdout = []
1104 if self.stderr:
1105 read_set.append(self.stderr)
1106 stderr = []
1107
1108 while read_set or write_set:
1109 rlist, wlist, xlist = select.select(read_set, write_set, [])
1110
1111 if self.stdin in wlist:
1112 # When select has indicated that the file is writable,
1113 # we can write up to PIPE_BUF bytes without risk
1114 # blocking. POSIX defines PIPE_BUF >= 512
1115 bytes_written = os.write(self.stdin.fileno(), input[:512])
1116 input = input[bytes_written:]
1117 if not input:
1118 self.stdin.close()
1119 write_set.remove(self.stdin)
1120
1121 if self.stdout in rlist:
1122 data = os.read(self.stdout.fileno(), 1024)
1123 if data == "":
1124 self.stdout.close()
1125 read_set.remove(self.stdout)
1126 stdout.append(data)
1127
1128 if self.stderr in rlist:
1129 data = os.read(self.stderr.fileno(), 1024)
1130 if data == "":
1131 self.stderr.close()
1132 read_set.remove(self.stderr)
1133 stderr.append(data)
1134
1135 # All data exchanged. Translate lists into strings.
1136 if stdout is not None:
1137 stdout = ''.join(stdout)
1138 if stderr is not None:
1139 stderr = ''.join(stderr)
1140
1141 # Translate newlines, if requested. We cannot let the file
1142 # object do the translation: It is based on stdio, which is
1143 # impossible to combine with select (unless forcing no
1144 # buffering).
1145 if self.universal_newlines and hasattr(file, 'newlines'):
1146 if stdout:
1147 stdout = self._translate_newlines(stdout)
1148 if stderr:
1149 stderr = self._translate_newlines(stderr)
1150
1151 self.wait()
1152 return (stdout, stderr)
1153
1154
1155def _demo_posix():
1156 #
1157 # Example 1: Simple redirection: Get process list
1158 #
1159 plist = Popen(["ps"], stdout=PIPE).communicate()[0]
1160 print "Process list:"
1161 print plist
1162
1163 #
1164 # Example 2: Change uid before executing child
1165 #
1166 if os.getuid() == 0:
1167 p = Popen(["id"], preexec_fn=lambda: os.setuid(100))
1168 p.wait()
1169
1170 #
1171 # Example 3: Connecting several subprocesses
1172 #
1173 print "Looking for 'hda'..."
1174 p1 = Popen(["dmesg"], stdout=PIPE)
1175 p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE)
1176 print repr(p2.communicate()[0])
1177
1178 #
1179 # Example 4: Catch execution error
1180 #
1181 print
1182 print "Trying a weird file..."
1183 try:
1184 print Popen(["/this/path/does/not/exist"]).communicate()
1185 except OSError, e:
1186 if e.errno == errno.ENOENT:
1187 print "The file didn't exist. I thought so..."
1188 print "Child traceback:"
1189 print e.child_traceback
1190 else:
1191 print "Error", e.errno
1192 else:
1193 print >>sys.stderr, "Gosh. No error."
1194
1195
1196def _demo_windows():
1197 #
1198 # Example 1: Connecting several subprocesses
1199 #
1200 print "Looking for 'PROMPT' in set output..."
1201 p1 = Popen("set", stdout=PIPE, shell=True)
1202 p2 = Popen('find "PROMPT"', stdin=p1.stdout, stdout=PIPE)
1203 print repr(p2.communicate()[0])
1204
1205 #
1206 # Example 2: Simple execution of program
1207 #
1208 print "Executing calc..."
1209 p = Popen("calc")
1210 p.wait()
1211
1212
1213if __name__ == "__main__":
1214 if mswindows:
1215 _demo_windows()
1216 else:
1217 _demo_posix()
Note: See TracBrowser for help on using the repository browser.