Read CSV with Scanner()

My csv is getting read into the System.out, but I’ve noticed that any text with a space gets moved into the next line (as a return n)

Here’s how my csv starts:

first,last,email,address 1, address 2
john,smith,blah@blah.com,123 St. Street,
Jane,Smith,blech@blech.com,4455 Roger Cir,apt 2

After running my app, any cell with a space (address 1), gets thrown onto the next line.

import java.io.File;
import java.io.FileNotFoundException;
import java.util.Scanner;

public class main {

    public static void main(String[] args) {
        // -define .csv file in app
        String fileNameDefined = "uploadedcsv/employees.csv";
        // -File class needed to turn stringName to actual file
        File file = new File(fileNameDefined);

        try{
            // -read from filePooped with Scanner class
            Scanner inputStream = new Scanner(file);
            // hashNext() loops line-by-line
            while(inputStream.hasNext()){
                //read single line, put in string
                String data = inputStream.next();
                System.out.println(data + "***");

            }
            // after loop, close scanner
            inputStream.close();


        }catch (FileNotFoundException e){

            e.printStackTrace();
        }

    }
}

So here’s the result in the console:

first,last,email,address 
1,address 
2
john,smith,blah@blah.com,123 
St. 
Street,
Jane,Smith,blech@blech.com,4455 
Roger 
Cir,apt 
2

Am I using Scanner incorrectly?

Please stop writing faulty CSV parsers!

I’ve seen hundreds of CSV parsers and so called tutorials for them online.

Nearly every one of them gets it wrong!

This wouldn’t be such a bad thing as it doesn’t affect me but people who try to write CSV readers and get it wrong tend to write CSV writers, too. And get them wrong as well. And these ones I have to write parsers for.

Please keep in mind that CSV (in order of increasing not so obviousness):

1. can have quoting characters around values
2. can have other quoting characters than “
3. can even have other quoting characters than ” and ‘
4. can have no quoting characters at all
5. can even have quoting characters on some values and none on others
6. can have other separators than , and ;
7. can have whitespace between seperators and (quoted) values
8. can have other charsets than ascii
9. should have the same number of values in each row, but doesn’t always
10. can contain empty fields, either quoted: "foo","","bar" or not: "foo",,"bar"
11. can contain newlines in values
12. can not contain newlines in values if they are not delimited
13. can not contain newlines between values
14. can have the delimiting character within the value if properly escaped
15. does not use backslash to escape delimiters but…
16. uses the quoting character itself to escape it, e.g. Frodo's Ring will be 'Frodo''s Ring'
17. can have the quoting character at beginning or end of value, or even as only character ("foo""", """bar", """")
18. can even have the quoted character within the not quoted value; this one is not escaped

If you think this is obvious not a problem, then think again. I’ve seen every single one of these items implemented wrongly. Even in major software packages. (e.g. Office-Suites, CRM Systems)

There are good and correctly working out-of-the-box CSV readers and writers out there:

• opencsv
• Ostermiller Java Utilities
• Apache Commons CSV

If you insist on writing your own at least read the (very short) RFC for CSV.

scanner.useDelimiter(",");

This should work.

import java.io.File;
import java.io.FileNotFoundException;
import java.util.Scanner;


public class TestScanner {

    public static void main(String[] args) throws FileNotFoundException {
        Scanner scanner = new Scanner(new File("/Users/pankaj/abc.csv"));
        scanner.useDelimiter(",");
        while(scanner.hasNext()){
            System.out.print(scanner.next()+"|");
        }
        scanner.close();
    }

}

For CSV File:

a,b,c d,e
1,2,3 4,5
X,Y,Z A,B

Output is:

a|b|c d|e
1|2|3 4|5
X|Y|Z A|B|

Scanner.next() does not read a newline but reads the next token, delimited by whitespace (by default, if useDelimiter() was not used to change the delimiter pattern). To read a line use Scanner.nextLine().

Once you read a single line you can use String.split(",") to separate the line into fields. This enables identification of lines that do not consist of the required number of fields. Using useDelimiter(","); would ignore the line-based structure of the file (each line consists of a list of fields separated by a comma). For example:

while (inputStream.hasNextLine())
{
    String line = inputStream.nextLine();
    String[] fields = line.split(",");
    if (fields.length >= 4) // At least one address specified.
    {
        for (String field: fields) System.out.print(field + "|");
        System.out.println();
    }
    else
    {
        System.err.println("Invalid record: " + line);
    }
}

As already mentioned, using a CSV library is recommended. For one, this (and useDelimiter(",") solution) will not correctly handle quoted identifiers containing , characters.

I agree with Scheintod that using an existing CSV library is a good idea to have RFC-4180-compliance from the start. Besides the mentioned OpenCSV and Oster Miller, there are a series of other CSV libraries out there. If you’re interested in performance, you can take a look at the uniVocity/csv-parsers-comparison. It shows that

• uniVocity CSV parser
• SimpleFlatMapper CSV parser
• Jackson CSV parser

are consistently the fastest using either JDK 6, 7, 8, or 9. The study did not find any RFC 4180 compatibility issues in any of those three. Both OpenCSV and Oster Miller are found to be about twice as slow as those.

I’m not in any way associated with the author(s), but concerning the uniVocity CSV parser, the study might be biased due to its author being the same as of that parser.

To note, the author of SimpleFlatMapper has also published a performance comparison comparing only those three.

Split nextLine() by this delimiter:
(?=([^"]*""[^""]*"")*[^""]*$)"").