perf: Collapse cross-joins to faster joins

[coastalwhite]

This PR adds the collapse_joins optimization pass.

This collapses a join and filters into a faster join. For example, a.join(b, how='cross').filter(pl.col.l == pl.col.r) can be collapsed into a.join(b, how='inner', left_on=pl.col.l, right_on=pl.col.r) if l is a column of a and r is a column of b.

This currently only collapses cross joins into inner or iejoin, but theoretically other joins could be simplified as well.

closes #18753

[coastalwhite]

If you want I can write a small fuzzer to try and further verify the correctness of this.

[codecov]

Codecov Report

Attention: Patch coverage is 89.36782% with 37 lines in your changes missing coverage. Please review.

Project coverage is 79.88%. Comparing base (25f84e4) to head (34516b5).
Report is 45 commits behind head on main.

Files with missing lines	Patch %	Lines
.../polars-plan/src/plans/optimizer/collapse_joins.rs	90.71%	30 Missing ⚠️
crates/polars-utils/src/idx_vec.rs	0.00%	3 Missing ⚠️
py-polars/polars/lazyframe/frame.py	25.00%	3 Missing ⚠️
py-polars/polars/functions/lazy.py	50.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #18633      +/-   ##
==========================================
+ Coverage   79.84%   79.88%   +0.03%     
==========================================
  Files        1518     1519       +1     
  Lines      205577   205925     +348     
  Branches     2892     2892              
==========================================
+ Hits       164144   164497     +353     
+ Misses      40885    40880       -5     
  Partials      548      548              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[ritchie46]

Currently it doesn't account for suffixed names. If we do support that, I can remove much of the logic in the DSL -> JOIN conversion and use this optimization pass directly there.

There is some logic there that deals with suffixed names. It is a bit of bookkeeping as duplicate names get suffixed post-join, but don't have that suffixed name pre-join. (They could have a name that has the suffix though. 🙈 ).

[coastalwhite]

Currently it doesn't account for suffixed names. If we do support that, I can remove much of the logic in the DSL -> JOIN conversion and use this optimization pass directly there.

There is some logic there that deals with suffixed names. It is a bit of bookkeeping as duplicate names get suffixed post-join, but don't have that suffixed name pre-join. (They could have a name that has the suffix though. 🙈 ).

This is now taken into account. I also added a very broad test that verifies the correctness up to a certain extent.

[mcrumiller]

I think this will close #18619, no? (I know you opened it, don't want it to remain orphaned).

[coastalwhite]

I don’t think it will. It is less pushing down predicates through a join and more about putting predicates into a join.