Skip to content
Snippets Groups Projects

Write parquet with rust backend to use full statistics option and better optimize row groups

Merged Write parquet with rust backend to use full statistics option and better optimize row groups
mdefende/gpfs-policy:bugfix-change-to-rust-backend-for-parquet-export into patch-v0.5.1
Merged Matthew K Defenderfer requested to merge mdefende/gpfs-policy:bugfix-change-to-rust-backend-for-parquet-export into patch-v0.5.1
Compare
and
1 file
+ 1
− 1
Compare changes
  • Side-by-side
  • Inline
src/rc_gpfs/policy/convert.py
+ 1
− 1
@@ -88,4 +88,4 @@ def convert(
.sort('path')
)
df.write_parquet(output_path,use_pyarrow=True)
\ No newline at end of file
df.write_parquet(output_path,statistics='full')
\ No newline at end of file