Opening a file in *nix full of invisible line breaks(\r)

Feb 15th, 2011 | By | Category: Linux / Freebsd

So i was given a massive 90MB CSV to go through today and much to my chagrin the output of wc -l was 1.  Ugh.  But copying and pasting sections of it clearly showed linebreaks however as far as linux was concerned it was all one line.  After a bit of examining and some help from the good boys at $awk on freenode I foudn that there were hidden linebreaks in there that didn’t show up in terminal for whatever reason and they looked like this ‘\r’

So I quickly filtered it through awk with this: awk ‘{gsub(/\r/,”\n”); print}’ file1.csv > files2.csv

and viola, I now have a CSV with 56,000 lines instead of 1 🙂

PS – in case your wondering dos2unix doesn’t do jack in this circumstance 🙂

Tags: , , , , , , ,

Leave a Comment