How to Wait for subprocess(es) to finish in Python

# Table of Contents

1. How to Wait for subprocess(es) to finish in Python
2. Using the subprocess.call() method to wait for a process to complete
3. Using the subprocess.run() method to wait for a process to complete
4. Using the psutil module to wait for multiple processes to terminate

# How to Wait for subprocess(es) to finish in Python

You can use the Popen.wait() method to wait for subprocesses to finish.

The method waits for a child process to terminate and returns the exit code.

Copied!
import subprocess

p1 = subprocess.Popen(['echo', 'bobbyhadz.com'])
p2 = subprocess.Popen(['echo', 'google.com'])


exit_codes = [p.wait() for p in (p1, p2)]

print(exit_codes)

We used the subprocess.Popen class to execute a child program in a new process.

The arguments the Popen class takes should be a sequence of program arguments or a single string or path-like object.

By default, the program to execute is the first item in args if args is a sequence.

The Popen.wait() method waits for a child process to terminate and returns the exit code.

We used a list comprehension to call the wait() method on each Popen object and store the exit codes.

Copied!
import subprocess

p1 = subprocess.Popen(['echo', 'bobbyhadz.com'])
p2 = subprocess.Popen(['echo', 'google.com'])


exit_codes = [p.wait() for p in (p1, p2)]

print(exit_codes)

However, you can also manually call the wait() method.

Copied!
import subprocess

p1 = subprocess.Popen(['echo', 'bobbyhadz.com'])
p2 = subprocess.Popen(['echo', 'google.com'])

exit_code1 = p1.wait()
print(exit_code1)

exit_code2 = p2.wait()
print(exit_code2)

The wait method can also be passed a timeout argument.

If the process doesn't terminate after timeout seconds, then a TimeoutExpired exception is raised.

Copied!
import subprocess

p1 = subprocess.Popen(['echo', 'bobbyhadz.com'])
p2 = subprocess.Popen(['echo', 'google.com'])

exit_code1 = p1.wait(timeout=5)
print(exit_code1)

exit_code2 = p2.wait(timeout=5)
print(exit_code2)

You can catch the exception and retry the wait.

Copied!
import subprocess

p1 = subprocess.Popen(['echo', 'bobbyhadz.com'])


def wait_child_process():
    exit_code1 = p1.wait(timeout=5)
    print(exit_code1)


try:
    wait_child_process()
except subprocess.TimeoutExpired:
    wait_child_process()

If the process hasn't terminated after 5 seconds, a TimeoutExpired exception is raised and we try to call the wait() method again.

# Using the subprocess.call() method to wait for a process to complete

You can also use the subprocess.call() method to wait for subprocesses to complete.

Copied!
import subprocess

exit_code1 = subprocess.call(['echo', 'bobbyhadz.com'])
exit_code2 = subprocess.call(['echo', 'google.com'])

print(exit_code1)
print(exit_code2)

The subprocess.call() method runs the supplied command and waits for it to complete.

The method returns the exit code.

# Using the subprocess.run() method to wait for a process to complete

You can also use the subprocess.run() method to wait for a subprocess to complete.

Copied!
import subprocess

p1 = subprocess.run(['echo', 'bobbyhadz.com'], timeout=3, check=True)
p2 = subprocess.run(['echo', 'google.com'], timeout=3, check=True)

print(p1)
print(p2)

The subprocess.run() method runs the supplied command and waits for it to complete.

The method returns a CompletedProcess instance.

If the specified timeout expires, the child process is terminated and waited for.

The TimeoutExpired exception is re-raised after the child process has terminated.

If the check argument is set to True and the process exits with a non-zero exit code, a CalledProcessError exception is raised.

The subprocess.run() method is synchronous, so the Python interpreter waits for the subprocess to finish before proceeding to the next command.

# Using the psutil module to wait for multiple processes to terminate

You can also use the psutil module to wait for multiple processes to terminate.

First, install the module by running the following command.

Copied!
pip install psutil

# or with pip3
pip3 install psutil

If you run into issues when installing psutil, follow the instructions in this article.

You can now import and use the module as follows.

Copied!
import subprocess
import psutil

p1 = subprocess.Popen(['echo', 'bobbyhadz.com'])
p2 = subprocess.Popen(['echo', 'google.com'])


processes = [psutil.Process(p1.pid), psutil.Process(p2.pid)]
print(processes)

# waits for multiple processes to terminate
gone, alive = psutil.wait_procs(processes, timeout=3)

print(gone)
print(alive)

The gone and alive variables store lists that indicate which processes are gone and which ones are still alive.

The processes variable stores a list of psutil.Process objects.

You can also set a callback function that is run when a process is terminated.

Copied!
import subprocess
import psutil

p1 = subprocess.Popen(['echo', 'bobbyhadz.com'])
p2 = subprocess.Popen(['echo', 'google.com'])

processes = [psutil.Process(p1.pid), psutil.Process(p2.pid)]


def on_terminate(proc):
    print(f"process {proc} terminated")


gone, alive = psutil.wait_procs(
    processes,
    timeout=3,
    callback=on_terminate
)

The on_terminate function gets called with the psutil.Process objects.

You can view more examples of using the psutil module in the package's Pypi page.

# Additional Resources

You can learn more about the related topics by checking out the following tutorials:

• ModuleNotFoundError: No module named 'psutil' in Python
• Yes/No question with user input in Python
• Solve Python error: subprocess-exited-with-error