Are you familiar with Parquet ? you can't do that at all, you need to rewrite the whole file.

Yeah , I poorly phrased it - I meant in an ideal situation with the benefits of parquet like columnar file structure. I very much understand that it’s not possible on parquet today for the reasons you mentioned and others.