CSV isn’t the best choice for storage, but which is better Feather or Parquet?
I was reading this post Feather vs CSV by Raj.k, and I thought, Feather is very good, but there is Parquet, that’s also very good, so which of them is better?
To answer this question, a simple script was made to convert a CSV file (649Mb) to Feather and Parquet, measure the new size and read time.
The code to convert CSV to Feather:
and here is the result:
73.41% reduction.
Time to read Feather: 1.51255.
The code to convert CSV to Parquet:
and here is the result:
85.58% reduction.
Time to read parquet: 2.18953.
Well, which one is the best?
Depends, if you want to save some storage and the reading time doesn’t matter, then Parquet is the best option, although if both things are important, then Feather is better, but the size will be a little bigger, in this test the difference between files was 46% if we compare Parquet vs Feather and the read time was 31% faster.
You can choose between these two options, but the CSV files are definitely not an option for saving your files to a storage.
