17.5.1. Using the subprocess Module

The recommended approach to invoking subprocesses is to use the run() function for all use cases it can handle. For more advanced use cases, the underlying Popen interface can be used directly.

The run() function was added in Python 3.5; if you need to retain compatibility with older versions, see the Older high-level API section.

subprocess.run(args, *, stdin=None, input=None, stdout=None, stderr=None, shell=False, timeout=None, check=False)

Run the command described by args. Wait for command to complete, then return a CompletedProcess instance.

The arguments shown above are merely the most common ones, described below in Frequently Used Arguments (hence the use of keyword-only notation in the abbreviated signature). The full function signature is largely the same as that of the Popen constructor - apart from timeout, input and check, all the arguments to this function are passed through to that interface.

This does not capture stdout or stderr by default. To do so, pass PIPE for the stdout and/or stderr arguments.

The timeout argument is passed to Popen.communicate(). If the timeout expires, the child process will be killed and waited for. The TimeoutExpired exception will be re-raised after the child process has terminated.

The input argument is passed to Popen.communicate() and thus to the subprocess’s stdin. If used it must be a byte sequence, or a string if universal_newlines=True. When used, the internal Popen object is automatically created with stdin=PIPE, and the stdin argument may not be used as well.

If check is True, and the process exits with a non-zero exit code, a CalledProcessError exception will be raised. Attributes of that exception hold the arguments, the exit code, and stdout and stderr if they were captured.

Examples:

>>> subprocess.run(["ls", "-l"])  # doesn't capture output
CompletedProcess(args=['ls', '-l'], returncode=0)

>>> subprocess.run("exit 1", shell=True, check=True)
Traceback (most recent call last):
  ...
subprocess.CalledProcessError: Command 'exit 1' returned non-zero exit status 1

>>> subprocess.run(["ls", "-l", "/dev/null"], stdout=subprocess.PIPE)
CompletedProcess(args=['ls', '-l', '/dev/null'], returncode=0,
stdout=b'crw-rw-rw- 1 root root 1, 3 Jan 23 16:23 /dev/null\n')

New in version 3.5.

class subprocess.CompletedProcess

The return value from run(), representing a process that has finished.

args

The arguments used to launch the process. This may be a list or a string.

returncode

Exit status of the child process. Typically, an exit status of 0 indicates that it ran successfully.

A negative value -N indicates that the child was terminated by signal N (POSIX only).

stdout

Captured stdout from the child process. A bytes sequence, or a string if run() was called with universal_newlines=True. None if stdout was not captured.

If you ran the process with stderr=subprocess.STDOUT, stdout and stderr will be combined in this attribute, and stderr will be None.

stderr

Captured stderr from the child process. A bytes sequence, or a string if run() was called with universal_newlines=True. None if stderr was not captured.

check_returncode()

If returncode is non-zero, raise a CalledProcessError.

New in version 3.5.

subprocess.DEVNULL

Special value that can be used as the stdin, stdout or stderr argument to Popen and indicates that the special file os.devnull will be used.

New in version 3.3.

subprocess.PIPE

Special value that can be used as the stdin, stdout or stderr argument to Popen and indicates that a pipe to the standard stream should be opened. Most useful with Popen.communicate().

subprocess.STDOUT

Special value that can be used as the stderr argument to Popen and indicates that standard error should go into the same handle as standard output.

exception subprocess.SubprocessError

Base class for all other exceptions from this module.

New in version 3.3.

exception subprocess.TimeoutExpired

Subclass of SubprocessError, raised when a timeout expires while waiting for a child process.

cmd

Command that was used to spawn the child process.

timeout

Timeout in seconds.

output

Output of the child process if it was captured by run() or check_output(). Otherwise, None.

stdout

Alias for output, for symmetry with stderr.

stderr

Stderr output of the child process if it was captured by run(). Otherwise, None.

New in version 3.3.

Changed in version 3.5: stdout and stderr attributes added

exception subprocess.CalledProcessError

Subclass of SubprocessError, raised when a process run by check_call() or check_output() returns a non-zero exit status.

returncode

Exit status of the child process.

cmd

Command that was used to spawn the child process.

output

Output of the child process if it was captured by run() or check_output(). Otherwise, None.

stdout

Alias for output, for symmetry with stderr.

stderr

Stderr output of the child process if it was captured by run(). Otherwise, None.

Changed in version 3.5: stdout and stderr attributes added

17.5.2. Security Considerations

Unlike some other popen functions, this implementation will never implicitly call a system shell. This means that all characters, including shell metacharacters, can safely be passed to child processes. If the shell is invoked explicitly, via shell=True, it is the application’s responsibility to ensure that all whitespace and metacharacters are quoted appropriately to avoid shell injection vulnerabilities.

When using shell=True, the shlex.quote() function can be used to properly escape whitespace and shell metacharacters in strings that are going to be used to construct shell commands.

17.5.3. Popen Objects

Instances of the Popen class have the following methods:

Popen.poll()

Check if child process has terminated. Set and return returncode attribute.

Popen.wait(timeout=None)

Wait for child process to terminate. Set and return returncode attribute.

If the process does not terminate after timeout seconds, raise a TimeoutExpired exception. It is safe to catch this exception and retry the wait.

Note

This will deadlock when using stdout=PIPE or stderr=PIPE and the child process generates enough output to a pipe such that it blocks waiting for the OS pipe buffer to accept more data. Use Popen.communicate() when using pipes to avoid that.

Note

The function is implemented using a busy loop (non-blocking call and short sleeps). Use the asyncio module for an asynchronous wait: see asyncio.create_subprocess_exec.

Changed in version 3.3: timeout was added.

Deprecated since version 3.4: Do not use the endtime parameter. It is was unintentionally exposed in 3.3 but was left undocumented as it was intended to be private for internal use. Use timeout instead.

Popen.communicate(input=None, timeout=None)

Interact with process: Send data to stdin. Read data from stdout and stderr, until end-of-file is reached. Wait for process to terminate. The optional input argument should be data to be sent to the child process, or None, if no data should be sent to the child. The type of input must be bytes or, if universal_newlines was True, a string.

communicate() returns a tuple (stdout_data, stderr_data). The data will be bytes or, if universal_newlines was True, strings.

Note that if you want to send data to the process’s stdin, you need to create the Popen object with stdin=PIPE. Similarly, to get anything other than None in the result tuple, you need to give stdout=PIPE and/or stderr=PIPE too.

If the process does not terminate after timeout seconds, a TimeoutExpired exception will be raised. Catching this exception and retrying communication will not lose any output.

The child process is not killed if the timeout expires, so in order to cleanup properly a well-behaved application should kill the child process and finish communication:

proc = subprocess.Popen(...)
try:
    outs, errs = proc.communicate(timeout=15)
except TimeoutExpired:
    proc.kill()
    outs, errs = proc.communicate()

Note

The data read is buffered in memory, so do not use this method if the data size is large or unlimited.

Changed in version 3.3: timeout was added.

Popen.send_signal(signal)

Sends the signal signal to the child.

Note

On Windows, SIGTERM is an alias for terminate(). CTRL_C_EVENT and CTRL_BREAK_EVENT can be sent to processes started with a creationflags parameter which includes CREATE_NEW_PROCESS_GROUP.

Popen.terminate()

Stop the child. On Posix OSs the method sends SIGTERM to the child. On Windows the Win32 API function TerminateProcess() is called to stop the child.

Popen.kill()

Kills the child. On Posix OSs the function sends SIGKILL to the child. On Windows kill() is an alias for terminate().

The following attributes are also available:

Popen.args

The args argument as it was passed to Popen – a sequence of program arguments or else a single string.

New in version 3.3.

Popen.stdin

If the stdin argument was PIPE, this attribute is a writeable stream object as returned by open(). If the universal_newlines argument was True, the stream is a text stream, otherwise it is a byte stream. If the stdin argument was not PIPE, this attribute is None.

Popen.stdout

If the stdout argument was PIPE, this attribute is a readable stream object as returned by open(). Reading from the stream provides output from the child process. If the universal_newlines argument was True, the stream is a text stream, otherwise it is a byte stream. If the stdout argument was not PIPE, this attribute is None.

Popen.stderr

If the stderr argument was PIPE, this attribute is a readable stream object as returned by open(). Reading from the stream provides error output from the child process. If the universal_newlines argument was True, the stream is a text stream, otherwise it is a byte stream. If the stderr argument was not PIPE, this attribute is None.

Popen.pid

The process ID of the child process.

Note that if you set the shell argument to True, this is the process ID of the spawned shell.

Popen.returncode

The child return code, set by poll() and wait() (and indirectly by communicate()). A None value indicates that the process hasn’t terminated yet.

A negative value -N indicates that the child was terminated by signal N (POSIX only).

17.5.4. Windows Popen Helpers

The STARTUPINFO class and following constants are only available on Windows.

class subprocess.STARTUPINFO

Partial support of the Windows STARTUPINFO structure is used for Popen creation.

dwFlags

A bit field that determines whether certain STARTUPINFO attributes are used when the process creates a window.

si = subprocess.STARTUPINFO()
si.dwFlags = subprocess.STARTF_USESTDHANDLES | subprocess.STARTF_USESHOWWINDOW

hStdInput

If dwFlags specifies STARTF_USESTDHANDLES, this attribute is the standard input handle for the process. If STARTF_USESTDHANDLES is not specified, the default for standard input is the keyboard buffer.

hStdOutput

If dwFlags specifies STARTF_USESTDHANDLES, this attribute is the standard output handle for the process. Otherwise, this attribute is ignored and the default for standard output is the console window’s buffer.

hStdError

If dwFlags specifies STARTF_USESTDHANDLES, this attribute is the standard error handle for the process. Otherwise, this attribute is ignored and the default for standard error is the console window’s buffer.

wShowWindow

If dwFlags specifies STARTF_USESHOWWINDOW, this attribute can be any of the values that can be specified in the nCmdShow parameter for the ShowWindow function, except for SW_SHOWDEFAULT. Otherwise, this attribute is ignored.

SW_HIDE is provided for this attribute. It is used when Popen is called with shell=True.

17.5.5. Older high-level API

Prior to Python 3.5, these three functions comprised the high level API to subprocess. You can now use run() in many cases, but lots of existing code calls these functions.

subprocess.call(args, *, stdin=None, stdout=None, stderr=None, shell=False, timeout=None)

Run the command described by args. Wait for command to complete, then return the returncode attribute.

This is equivalent to:

run(...).returncode

(except that the input and check parameters are not supported)

The arguments shown above are merely the most common ones. The full function signature is largely the same as that of the Popen constructor - this function passes all supplied arguments other than timeout directly through to that interface.

Note

Do not use stdout=PIPE or stderr=PIPE with this function. The child process will block if it generates enough output to a pipe to fill up the OS pipe buffer as the pipes are not being read from.

Changed in version 3.3: timeout was added.

subprocess.check_call(args, *, stdin=None, stdout=None, stderr=None, shell=False, timeout=None)

Run command with arguments. Wait for command to complete. If the return code was zero then return, otherwise raise CalledProcessError. The CalledProcessError object will have the return code in the returncode attribute.

This is equivalent to:

run(..., check=True)

(except that the input parameter is not supported)

The arguments shown above are merely the most common ones. The full function signature is largely the same as that of the Popen constructor - this function passes all supplied arguments other than timeout directly through to that interface.

Note

Do not use stdout=PIPE or stderr=PIPE with this function. The child process will block if it generates enough output to a pipe to fill up the OS pipe buffer as the pipes are not being read from.

Changed in version 3.3: timeout was added.

subprocess.check_output(args, *, stdin=None, stderr=None, shell=False, universal_newlines=False, timeout=None)

Run command with arguments and return its output.

If the return code was non-zero it raises a CalledProcessError. The CalledProcessError object will have the return code in the returncode attribute and any output in the output attribute.

This is equivalent to:

run(..., check=True, stdout=PIPE).stdout

The arguments shown above are merely the most common ones. The full function signature is largely the same as that of run() - most arguments are passed directly through to that interface. However, explicitly passing input=None to inherit the parent’s standard input file handle is not supported.

By default, this function will return the data as encoded bytes. The actual encoding of the output data may depend on the command being invoked, so the decoding to text will often need to be handled at the application level.

This behaviour may be overridden by setting universal_newlines to True as described above in Frequently Used Arguments.

To also capture standard error in the result, use stderr=subprocess.STDOUT:

>>> subprocess.check_output(
...     "ls non_existent_file; exit 0",
...     stderr=subprocess.STDOUT,
...     shell=True)
'ls: non_existent_file: No such file or directory\n'

New in version 3.1.

Changed in version 3.3: timeout was added.

Changed in version 3.4: Support for the input keyword argument was added.

17.5.6. Replacing Older Functions with the subprocess Module

In this section, “a becomes b” means that b can be used as a replacement for a.

In the following examples, we assume that the relevant functions have already been imported from the subprocess module.

output=`mycmd myarg`
# becomes
output = check_output(["mycmd", "myarg"])

output=`dmesg | grep hda`
# becomes
p1 = Popen(["dmesg"], stdout=PIPE)
p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE)
p1.stdout.close()  # Allow p1 to receive a SIGPIPE if p2 exits.
output = p2.communicate()[0]

output=`dmesg | grep hda`
# becomes
output=check_output("dmesg | grep hda", shell=True)

sts = os.system("mycmd" + " myarg")
# becomes
sts = call("mycmd" + " myarg", shell=True)

try:
    retcode = call("mycmd" + " myarg", shell=True)
    if retcode < 0:
        print("Child was terminated by signal", -retcode, file=sys.stderr)
    else:
        print("Child returned", retcode, file=sys.stderr)
except OSError as e:
    print("Execution failed:", e, file=sys.stderr)

pid = os.spawnlp(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg")
==>
pid = Popen(["/bin/mycmd", "myarg"]).pid

retcode = os.spawnlp(os.P_WAIT, "/bin/mycmd", "mycmd", "myarg")
==>
retcode = call(["/bin/mycmd", "myarg"])

os.spawnvp(os.P_NOWAIT, path, args)
==>
Popen([path] + args[1:])

os.spawnlpe(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg", env)
==>
Popen(["/bin/mycmd", "myarg"], env={"PATH": "/usr/bin"})

(child_stdin, child_stdout) = os.popen2(cmd, mode, bufsize)
==>
p = Popen(cmd, shell=True, bufsize=bufsize,
          stdin=PIPE, stdout=PIPE, close_fds=True)
(child_stdin, child_stdout) = (p.stdin, p.stdout)

(child_stdin,
 child_stdout,
 child_stderr) = os.popen3(cmd, mode, bufsize)
==>
p = Popen(cmd, shell=True, bufsize=bufsize,
          stdin=PIPE, stdout=PIPE, stderr=PIPE, close_fds=True)
(child_stdin,
 child_stdout,
 child_stderr) = (p.stdin, p.stdout, p.stderr)

(child_stdin, child_stdout_and_stderr) = os.popen4(cmd, mode, bufsize)
==>
p = Popen(cmd, shell=True, bufsize=bufsize,
          stdin=PIPE, stdout=PIPE, stderr=STDOUT, close_fds=True)
(child_stdin, child_stdout_and_stderr) = (p.stdin, p.stdout)

pipe = os.popen(cmd, 'w')
...
rc = pipe.close()
if rc is not None and rc >> 8:
    print("There were some errors")
==>
process = Popen(cmd, stdin=PIPE)
...
process.stdin.close()
if process.wait() != 0:
    print("There were some errors")

(child_stdout, child_stdin) = popen2.popen2("somestring", bufsize, mode)
==>
p = Popen("somestring", shell=True, bufsize=bufsize,
          stdin=PIPE, stdout=PIPE, close_fds=True)
(child_stdout, child_stdin) = (p.stdout, p.stdin)

(child_stdout, child_stdin) = popen2.popen2(["mycmd", "myarg"], bufsize, mode)
==>
p = Popen(["mycmd", "myarg"], bufsize=bufsize,
          stdin=PIPE, stdout=PIPE, close_fds=True)
(child_stdout, child_stdin) = (p.stdout, p.stdin)

17.5.7. Legacy Shell Invocation Functions

This module also provides the following legacy functions from the 2.x commands module. These operations implicitly invoke the system shell and none of the guarantees described above regarding security and exception handling consistency are valid for these functions.

subprocess.getstatusoutput(cmd)

Return (status, output) of executing cmd in a shell.

Execute the string cmd in a shell with Popen.check_output() and return a 2-tuple (status, output). Universal newlines mode is used; see the notes on Frequently Used Arguments for more details.

A trailing newline is stripped from the output. The exit status for the command can be interpreted according to the rules for the C function wait(). Example:

>>> subprocess.getstatusoutput('ls /bin/ls')
(0, '/bin/ls')
>>> subprocess.getstatusoutput('cat /bin/junk')
(256, 'cat: /bin/junk: No such file or directory')
>>> subprocess.getstatusoutput('/bin/junk')
(256, 'sh: /bin/junk: not found')

Availability: POSIX & Windows

Changed in version 3.3.4: Windows support added

subprocess.getoutput(cmd)

Return output (stdout and stderr) of executing cmd in a shell.

Like getstatusoutput(), except the exit status is ignored and the return value is a string containing the command’s output. Example:

>>> subprocess.getoutput('ls /bin/ls')
'/bin/ls'

Availability: POSIX & Windows

Changed in version 3.3.4: Windows support added

17.5.8. Notes