Writing a list to a file with Python

Posted on

Solving problem is about exposing yourself to as many situations as possible like Writing a list to a file with Python and practice these strategies over and over. With time, it becomes second nature and a natural way you approach any problems in general. Big or small, always start with a plan, use other strategies mentioned here till you are confident and ready to code the solution.
In this post, my aim is to share an overview the topic about Writing a list to a file with Python, which can be followed any time. Take easy to follow this discuss.

Writing a list to a file with Python

Is this the cleanest way to write a list to a file, since writelines() doesn’t insert newline characters?

file.writelines(["%sn" % item  for item in list])

It seems like there would be a standard way…

Answer #1:

You can use a loop:

with open('your_file.txt', 'w') as f:
    for item in my_list:
        f.write("%sn" % item)

In Python 2, you can also use

with open('your_file.txt', 'w') as f:
    for item in my_list:
        print >> f, item

If you’re keen on a single function call, at least remove the square brackets [], so that the strings to be printed get made one at a time (a genexp rather than a listcomp) — no reason to take up all the memory required to materialize the whole list of strings.

Answered By: Alex Martelli

Answer #2:

What are you going to do with the file? Does this file exist for humans, or other programs with clear interoperability requirements?

If you are just trying to serialize a list to disk for later use by the same python app, you should be pickleing the list.

import pickle
with open('outfile', 'wb') as fp:
    pickle.dump(itemlist, fp)

To read it back:

with open ('outfile', 'rb') as fp:
    itemlist = pickle.load(fp)

Answer #3:

The simpler way is:

with open("outfile", "w") as outfile:
    outfile.write("n".join(itemlist))

You could ensure that all items in item list are strings using a generator expression:

with open("outfile", "w") as outfile:
    outfile.write("n".join(str(item) for item in itemlist))

Remember that all itemlist list need to be in memory, so, take care about the memory consumption.

Answered By: osantana

Answer #4:

Using Python 3 and Python 2.6+ syntax:

with open(filepath, 'w') as file_handler:
    for item in the_list:
        file_handler.write("{}n".format(item))

This is platform-independent. It also terminates the final line with a newline character, which is a UNIX best practice.

Starting with Python 3.6, "{}n".format(item) can be replaced with an f-string: f"{item}n".

Answered By: orluke

Answer #5:

Yet another way. Serialize to json using simplejson (included as json in python 2.6):

>>> import simplejson
>>> f = open('output.txt', 'w')
>>> simplejson.dump([1,2,3,4], f)
>>> f.close()

If you examine output.txt:

[1, 2, 3, 4]

This is useful because the syntax is pythonic, it’s human readable, and it can be read by other programs in other languages.

Answered By: Jason Baker

Answer #6:

I thought it would be interesting to explore the benefits of using a genexp, so here’s my take.

The example in the question uses square brackets to create a temporary list, and so is equivalent to:

file.writelines( list( "%sn" % item for item in list ) )

Which needlessly constructs a temporary list of all the lines that will be written out, this may consume significant amounts of memory depending on the size of your list and how verbose the output of str(item) is.

Drop the square brackets (equivalent to removing the wrapping list() call above) will instead pass a temporary generator to file.writelines():

file.writelines( "%sn" % item for item in list )

This generator will create newline-terminated representation of your item objects on-demand (i.e. as they are written out). This is nice for a couple of reasons:

  • Memory overheads are small, even for very large lists
  • If str(item) is slow there’s visible progress in the file as each item is processed

This avoids memory issues, such as:

In [1]: import os
In [2]: f = file(os.devnull, "w")
In [3]: %timeit f.writelines( "%sn" % item for item in xrange(2**20) )
1 loops, best of 3: 385 ms per loop
In [4]: %timeit f.writelines( ["%sn" % item for item in xrange(2**20)] )
ERROR: Internal Python error in the inspect module.
Below is the traceback from this internal error.
Traceback (most recent call last):
...
MemoryError

(I triggered this error by limiting Python’s max. virtual memory to ~100MB with ulimit -v 102400).

Putting memory usage to one side, this method isn’t actually any faster than the original:

In [4]: %timeit f.writelines( "%sn" % item for item in xrange(2**20) )
1 loops, best of 3: 370 ms per loop
In [5]: %timeit f.writelines( ["%sn" % item for item in xrange(2**20)] )
1 loops, best of 3: 360 ms per loop

(Python 2.6.2 on Linux)

Answered By: RobM

Answer #7:

Because i’m lazy….

import json
a = [1,2,3]
with open('test.txt', 'w') as f:
    f.write(json.dumps(a))
#Now read the file back into a Python list object
with open('test.txt', 'r') as f:
    a = json.loads(f.read())
Answered By: Big Sam

Answer #8:

Serialize list into text file with comma sepparated value

mylist = dir()
with open('filename.txt','w') as f:
    f.write( ','.join( mylist ) )
Answered By: themadmax

Leave a Reply

Your email address will not be published.