The way I have my partitions setup is 50GB of EXT4 for linux and 300GB of NTFS for general file storage so that I can access them if I dual boot in windows or anything else int he future.. Compatibility was the name of the game. Because of this I end up storing most of my working files on the NTFS and working off that drive keeping the EXT4 for linux system files only. Well that was until today when I happened to have top running while executing a new script I am just finishing off.. The script does a LOT of search and replacing with sed, queries with awk and some other bash fun. This particular one was taking a 24MB file and breaking it down into 2.5MB chunks based on the contents of the file. Searches for the 1200th instance of a marker, takes all the content above that, puts it into a new file and then deletes it from the original. Rinse repeat, rinse repeat.. Anyways top was showing that ntfs-3g was hogging 90% of the cpu while this was going on so out of curiosity I redid the job this time piping it through PV to get a timer on it. End result: 5 minutes 25 seconds.
Sooo I wondered how long would it take if I moved all my stuff over to my EXT4 partition and ran it there. End result: 2 minutes and 44 seconds. Boooya, I’ve just cut down the processing time of my scripts by half just by running them on my EXT4 partition.
Just thought I would share for all those people wanting results from a real life performance test, it’s pretty substantial.
