Using genfromtxt to import csv data with missing values in numpy

Posted on

Question :

Using genfromtxt to import csv data with missing values in numpy

I have a csv file that looks something like this (actual file has many more columns and rows):


Say the name of the file is info.csv
If I try to import this using

data = numpy.genfromtxt('info.csv', delimiter = ',')

then I get the following error:

ValueError: Some errors were detected ! Line #4 (got 1 columns instead of 5)

If I use,

data = numpy.genfromtxt('info.csv', delimiter = ',', skip_footer = 1) 

both lines with data 16 and with data 11, 12, 13, 14, 15 are skipped. I don’t understand why the line with 11, 12, 13, 14, 15 is being skipped. I would appreciate any help on how I can appropriately use the genfromtxt to import first three lines in the above file.


Answer #1:

if you can ignore the 16 at the end of the file try using the

invalid_raise (bool, optional) parameter
if set to False it ignores all incomplete lines without throwing an exception

see here (its the last parameter before the examples)

Answered By: Nikolaus Gradwohl

Answer #2:

The command filling_values also helped me. I set it to zero. Thus each empty value is set to zero. It probably doesn’t always make sense, but maybe it will help you.

Answered By: Dark

Leave a Reply

Your email address will not be published. Required fields are marked *