I am very new to Python so please forgive the following basic code and problem, but I have been trying to figure out what is causing the error I am getting (I have even looked at similar threads on S.O.) but can’t get past my issue.
Here is what I am trying to do:
- loop through a folder of CSV files
- search for a ‘keyword’ and delete all lines containing the ‘keyword’
- save output to a separate folder
Here is my code:
import os, fnmatch import shutil src_dir = "C:/temp/CSV" target_dir = "C:/temp/output2" keyword = "KEYWORD" for f in os.listdir(src_dir): os.path.join(src_dir, f) with open(f): for line in f: if keyword not in line: write(line) shutil.copy2(os.path.join(src_dir, f), target_dir)
Here is the error I am getting:
IOError: [Errno 2] No such file or directory: 'POS_03217_20120309_153244.csv'
I have confirmed that the folder and file do exist. What is causing the
IOError and how to I resolve it? Also, is there anything else wrong with my code that would prevent me from performing the entire task?
Hmm, there are a few things going wrong here.
for f in os.listdir(src_dir): os.path.join(src_dir, f)
You’re not storing the result of
join. This should be something like:
for f in os.listdir(src_dir): f = os.path.join(src_dir, f)
This open call is is the cause of your
IOError. (Because without storing the result of the
f was still just ‘file.csv’, not ‘src_dir/file.csv’.)
Also, the syntax:
is close, but the syntax isn’t quite right. It should be
with open(file_name) as file_object:. Then, you use to the
file_object to perform read or write operations.
You told python what you wanted to write, but not where to write it. Write is a method on the file object. Try
Edit: You’re also clobbering your input file. You probably want to
open the output file and write lines to it as you’re reading them in from the input file.
See: input / output in python.
Even though @Ignacio gave you a straightforward solution, I thought I might add an answer that gives you some more details about the issues with your code…
# You are not saving this result into a variable to reuse os.path.join(src_dir, f) # Should be src_path = os.path.join(src_dir, f) # you open the file but you dont again use a variable to reference with open(f) # should be with open(src_path) as fh # this is actually just looping over each character # in each result of your os.listdir for line in f # you should loop over lines in the open file handle for line in fh # write? Is this a method you wrote because its not a python builtin function write(line) # write to the file fh.write(line)
with open(os.path.join(src_dir, f)) as fin: for line in fin:
Also, you never output to a new file.
Just as an FYI, here is my working code:
src_dir = "C:\temp\CSV\" target_dir = "C:\temp\output2\" keyword = "KEYWORD" for f in os.listdir(src_dir): file_name = os.path.join(src_dir, f) out_file = os.path.join(target_dir, f) with open(file_name, "r+") as fi, open(out_file, "w") as fo: for line in fi: if keyword not in line: fo.write(line)
Thanks again to everyone for all the great feedback!
I got this error and fixed by appending the directory path in the loop. script not in the same directory as the files. dr1 =”~/test” directory variable