This is a follow up to this question, but if I want to pass an argument to
subprocess, how can I get the output in real time? This is what I currently have; I also tried replacing
call from the
subprocess module and this just leads to the script hanging.
from subprocess import Popen, PIPE, STDOUT cmd = 'rsync --rsh=ssh -rv --files-from=- thisdir/ servername:folder/' p = Popen(cmd.split(), stdout=PIPE, stdin=PIPE, stderr=STDOUT) subfolders = 'n'.join(['subfolder1','subfolder2']) output = p.communicate(input=subfolders) print output
In the former question where I did not have to pass
stdin I was suggested to use
p.stdout.readline, there there is no room there to pipe anything to
Addendum: This works for the transfer, but I see the output only at the end and I would like to see the details of the transfer while it’s happening.
In order to grab stdout from the subprocess in real time you need to decide exactly what behavior you want; specifically, you need to decide whether you want to deal with the output line-by-line or character-by-character, and whether you want to block while waiting for output or be able to do something else while waiting.
It looks like it will probably suffice for your case to read the output in line-buffered fashion, blocking until each complete line comes in, which means the convenience functions provided by
subprocess are good enough:
p = subprocess.Popen(some_cmd, stdout=subprocess.PIPE) # Grab stdout line by line as it becomes available. This will loop until # p terminates. while p.poll() is None: l = p.stdout.readline() # This blocks until it receives a newline. print l # When the subprocess terminates there might be unconsumed output # that still needs to be processed. print p.stdout.read()
If you need to write to the stdin of the process, just use another pipe:
p = subprocess.Popen(some_cmd, stdout=subprocess.PIPE, stdin=subprocess.PIPE) # Send input to p. p.stdin.write("some inputn") p.stdin.flush() # Now start grabbing output. while p.poll() is None: l = p.stdout.readline() print l print p.stdout.read()
Pace the other answer, there’s no need to indirect through a file in order to pass input to the subprocess.
something like this I think
from subprocess import Popen, PIPE, STDOUT p = Popen('c:/python26/python printingTest.py', stdout = PIPE, stderr = PIPE) for line in iter(p.stdout.readline, ''): print line p.stdout.close()
using an iterator will return live results basically ..
in order to send input to stdin you would need something like
other_input = "some extra input stuff" with open("to_input.txt","w") as f: f.write(other_input) p = Popen('c:/python26/python printingTest.py < some_input_redirection_thing', stdin = open("to_input.txt"), stdout = PIPE, stderr = PIPE)
this would be similar to the linux shell command of
%prompt%> some_file.o < cat to_input.txt
see alps answer for better passing to stdin
If you pass all your input before starting reading the output and if by “real-time” you mean whenever the subprocess flushes its stdout buffer:
from subprocess import Popen, PIPE, STDOUT cmd = 'rsync --rsh=ssh -rv --files-from=- thisdir/ servername:folder/' p = Popen(cmd.split(), stdout=PIPE, stdin=PIPE, stderr=STDOUT, bufsize=1) subfolders = 'n'.join(['subfolder1','subfolder2']) p.stdin.write(subfolders) p.stdin.close() # eof for line in iter(p.stdout.readline, ''): print line, # do something with the output here p.stdout.close() rc = p.wait()