fix: remove unnecessary metadata action when overwriting partition

[PeterKeDer]

Description

Fixes spurious metadata action when write_deltalake is called with mode overwrite, using a predicate and with a string partition column. This is undesirable because all concurrent writes will fail and need to be retried due to this metadata action.

This is caused by the schema != table_schema check. table_schema is from calling table.input_schema() which converts string partition columns to dictionary and causes it to be different from schema.

We fix this issue by comparing it using try_cast_batch, so the behavior becomes identical to writing with mode='append'.

To replicate (on deltalake==0.20.1):

from deltalake import DeltaTable, write_deltalake
import polars as pl

df1 = pl.DataFrame({'id': ['a', 'b'], 'val': [1,2]})

write_deltalake('testtable1', df1.to_arrow(), schema_mode='merge', mode='overwrite', partition_by=['id'])
write_deltalake('testtable1', df1.to_arrow(), schema_mode='merge', mode='overwrite', partition_by=['id'])
write_deltalake('testtable1', df1.to_arrow(), schema_mode='merge', mode='overwrite', partition_by=['id'])

If we look at the latter 2 transaction JSONs, they will have a metadata action indicating a schema change, even though the schema is identical.

[houqp]

Would be good to add a test as follow up.

[ion-elgreco]

Try cast batch is too lenient, you can already see it in the test failure.

[houqp]

There are a couple issues that are worth digging into:

• can_cast_types seems like the wrong function to use in try_cast_batch considering it allows incompatible type downcasting, i.e. casting from i64 to i8.
• The failing rust test test_issue_2105 has the id column type defined as PrimitiveType::Integer, which based on delta spec should map to 4 bytes int (i32). But the test expects the query result to have the type ArrowDataType::Int64. Is this expected? The original MR that introduced this test has the expected type defined as ArrowDataType::Int32, which seems more reasonable, but it got changed to i64 in later schema casting MR.
• The failed python test test_parse_stats_with_new_schema is definitely a valid test failure that we should fix @PeterKeDer . Although it might be related to the same use of can_cast_types in try_cast_batch issue mentioned in my first point.