Python: Running more than one process at once

Running many scripts at the same time may not come up on a daily basis and may not even be a good idea, but here is a great little code snippet. I had a few scripts that could run in parallel to save time without too much of a performance hit (the scripts grab data from external sources).

from subprocess import Popen
files = ['file1.py',
         'file2.py',
         'file3.py',
         'file4.py',
         'file5.py']

threads = []
for file in files:
    t = Popen(file, shell=True)
    threads.append(t)

[x.wait() for x in threads]

 

This code runs five files at the same time and waits for all five to finish before continuing... Lovely.

Enjoy !