Convert churn calculation to lazyframe backend
Merged
requested to merge mdefende/gpfs-policy:ref-calculate-churn-with-lazyframes into patch-v0.5.2
Improved performance of churn calculation by making the pipeline perform on lazyframes all the way through. Had to change some of the logic, but it's all equivalent at the end. A set of lazyframe plans are created by looping through the given acq values and all collected in parallel at the end. This allow polars to plan to only read each acq dataset once even though most will be used in multiple comparisons. Performance is fairly comparable to previous versions without needing to use GPUs.