Move EnforceSorting into datafusion-physical-optimizer crate

[buraksenn]

Which issue does this PR close?

Closes #14185

Rationale for this change

Extract remaining physical optimizer out of core

What changes are included in this PR?

Move EnforceSorting out of core package

Are these changes tested?

Existing tests that were moved to integration tests

[buraksenn]

I've these errors in replace in replace_with_order_preserving_variants tests:

expected:

[
    "SortExec: expr=[a@0 ASC NULLS LAST], preserve_partitioning=[false]",
    "  CoalescePartitionsExec",
    "    RepartitionExec: partitioning=Hash([c@1], 8), input_partitions=8",
    "      RepartitionExec: partitioning=RoundRobinBatch(8), input_partitions=1",
    "        StreamingTableExec: partition_sizes=1, projection=[a, c, d], infinite_source=true, output_ordering=[a@0 ASC NULLS LAST]",
]
actual:

[
    "SortExec: expr=[a@0 ASC NULLS LAST], preserve_partitioning=[false]",
    "  CoalescePartitionsExec",
    "    RepartitionExec: partitioning=Hash([c@2], 8), input_partitions=8",
    "      RepartitionExec: partitioning=RoundRobinBatch(8), input_partitions=1",
    "        StreamingTableExec: partition_sizes=1, projection=[a, b, c, d], infinite_source=true, output_ordering=[a@0 ASC NULLS LAST]",
]

Not sure why but projection changes so I'm working on fixing it

[buraksenn]

cc @berkaysynnada

[berkaysynnada]

cc @berkaysynnada

There are two function having the same name (stream_exec_ordered), one in replace_with_order_preserving_variants.rs with a built-in projection, and one in core/src/test/mod.rs, w/o a projection, which differs the test results. You've mixed up them. I guess you've deleted one of them actually.

[alamb]

Thank you @buraksenn and @berkaysynnada

I did some spot checks on this PR and I think it looks EPIC -- very nice. The fact that no plans change gives me confidence the refactoring is good.

I took the liberty of pushing a commit that removed a chance to the datafusion-testing pin:

But otherwise I think this is ready to merge. I will leave it open a bit longer to give @berkaysynnada a chance to comment if they want.

[xudong963]

I did some random checks, lgtm

[berkaysynnada]

LGTM, thank you @buraksenn. I'll continue with the #14190