python-v0.22.3

What's Changed

• fix: attempt to workaround problems reading actions as empty rather than null by @rtyler in #3045

Full Changelog: python-v0.22.2...python-v0.22.3

python-v0.22.2: QueryBuilder

What's Changed

• fix: check top level nullability during write by @ion-elgreco in #3026
• feat: read delta table using datafusion by @PeterKeDer in #2922
• chore: bump for another python and core release by @rtyler in #3038

Full Changelog: python-v0.22.1...python-v0.22.2

python-v0.22.1

What's Changed

• fix: jsonwriter and recordbatchwriter to respect stats skipping by @jusjosj in #2989
• chore: bump the python version for release by @rtyler in #3024
• chore: refactor some of the writebuilder preconditions into the appropriate function by @rtyler in #3022
• fix(python): pass ending_timestamp correctly by @ion-elgreco in #3025
• chore(deps): update hashbrown requirement from 0.14.3 to 0.15.2 by @dependabot in #3028
• fix: workaround for Add actions being read slightly differently out of parquet files by @rtyler in #3031

Full Changelog: python-v0.22.0...python-v0.22.1

python-v0.22.0

What's Changed

• chore: bump the python package version to 0.21.0 by @rtyler in #2967
• fix: enable readerFeatures in minReaderVersion 3 by @rjancewicz in #2970
• chore(deps): update which requirement from 6 to 7 by @dependabot in #2971
• perf: close partition writers concurrently by @alexwilcoxson-rel in #2984
• chore(deps): update thiserror requirement from 1 to 2 by @dependabot in #2985
• perf: batch json decode checkpoint actions when writing to parquet by @alexwilcoxson-rel in #2983
• docs: fix typo by @RyRyRyNguyen in #2977
• fix: update object_store to 0.10.2 by @thomasfrederikhoeck in #2994
• fix: cache credential resolution with the AWS credential provider by @rtyler in #2987
• fix: jsonwriter should checkpoint by default by @jusjosj in #2993
• fix: correctly recognize existing delta tables using the transaction log by @stretchadito in #3005
• fix: fixed the deprecation warnings in spot check step of the build by @vksx in #3007
• chore: upgrade to datafusion 43 by @ion-elgreco in #2886
• docs: fix the verify table existence example in usage docs by @vksx in #3003
• fix: decimal stat rounding overflow by @gruuya in #2975
• feat: upgrade to delta_kernel 0.4.1 🎉 by @rtyler in #3016
• chore: include license files in published crates by @ankane in #3009
• docs: explain the value of deltalake on first page of docs by @braaannigan in #3017
• feat: provide direct TableProvider integration in datafusion-python by @timsaucer in #3012
• fix: expression with dates to string conversation by @JonasDev1 in #3019
• chore: fixed a bunch of warnings and deprecations by @hntd187 in #3020
• fix: num rows statistics by @lewiszlw in #2990
• docs: mention AWS_ENDPOINT_URL_DYNAMODB by @maxitg in #3021
• feat: override dynamodb config by @thomas-chauvet in #3011

New Contributors

• @rjancewicz made their first contribution in #2970
• @RyRyRyNguyen made their first contribution in #2977
• @stretchadito made their first contribution in #3005
• @vksx made their first contribution in #3007
• @ankane made their first contribution in #3009
• @timsaucer made their first contribution in #3012
• @lewiszlw made their first contribution in #2990
• @maxitg made their first contribution in #3021
• @thomas-chauvet made their first contribution in #3011

Full Changelog: python-v0.21.0...python-v0.22.0

python-v0.21.0

What's Changed

• fix: avoid reference after move when building w/o datafusion by @rtyler in #2917
• docs: updating docs for MinIO and Docker, with working conditional put support by @rwhaling in #2895
• feat: get earliest version by @ion-elgreco in #2797
• fix: cdc merge union schema order by @ion-elgreco in #2919
• chore: update the changelog with the latest rust merges by @rtyler in #2918
• fix: datafusion sanity checker passes when all files filtered out by @adamfaulkner-at in #2830
• fix: ListingSchemaProvider not forwarding custom storage_options by @Nordalf in #2924
• fix: update createdTime during schema evolution by @ion-elgreco in #2926
• fix: properly handle the different flavors of storage options keys by @rtyler in #2931
• chore: introduce some more writer tests to demonstrate schema promotion by @rtyler in #2935
• fix: panics when creating delta tables in threads by @PeterKeDer in #2940
• docs: minor fixes to the architecture documentation by @0x26res in #2934
• feat(python): reading/writing transaction identifiers in python by @PeterKeDer in #2942
• fix: improve errors on field cast failures by @jkylling in #2932
• fix: dramatically reduce checkpoint memory consumption by @rtyler in #2956
• chore: enable a dummy DCO app to unblock the merge queue by @rtyler in #2959
• docs: add merge guide by @avriiil in #2953
• fix: ensure the checkpoint decoder is regularly flushed by @rtyler in #2960
• chore: fix many warnings across the codebase by @hntd187 in #2955
• chore: be less verbose about zorder by @rtyler in #2963

New Contributors

• @rwhaling made their first contribution in #2895
• @adamfaulkner-at made their first contribution in #2830
• @Nordalf made their first contribution in #2924
• @0x26res made their first contribution in #2934

Full Changelog: python-v0.20.2...python-v0.21.0

rust-v0.20.1

What's Changed

• chore: pin the Rust baseline version to 1.80 by @rtyler in #2842
• fix: pin the build-dependencies for Python to a slightly older vendored openssl by @rtyler in #2856
• fix: escaped columns in dataskippingstatscolumns by @ion-elgreco in #2855
• fix: set put mode to overwrite in mount backend by @ion-elgreco in #2861
• fix(rust): scan schema fix for predicate by @sherlockbeard in #2869
• chore: attempt to ignore all dependabot checks for arrow and datafusion by @rtyler in #2870
• fix: stats is optional in add action by @jkylling in #2841
• fix: prepare the next 🦀 release with fixed version ranges by @rtyler in #2875
• fix: pin broken dependencies and changes in 0.19.1 by @rtyler in #2878
• chore: exclude parquet from dependabot as well by @rtyler in #2874
• chore: rearrange github actions a bit by @rtyler in #2868
• chore: cleanup codecov defaults by @rtyler in #2876
• chore(deps): update sqlparser requirement from 0.50 to 0.51 by @dependabot in #2881
• chore: set max_retries in CommitProperties by @helanto in #2826
• fix(python, rust): use require files by @ion-elgreco in #2809
• fix: re-enable optional old casting behavior in merge by @ion-elgreco in #2853
• fix: conditionally disable enable_io non-unix based systems by @hntd187 in #2884
• docs: fix typo in delta-lake-dagster by @jessy1092 in #2883
• feat: add support for pyarrow.ExtensionType by @fecet in #2885
• feat: improve AWS credential loading between S3 and DynamoDb code paths by @rtyler in #2887
• feat(python, rust): add feature operation by @ion-elgreco in #2712
• fix: convert last checkpoint json keys to camelCase by @feniljain in #2889
• fix: ensure log store correctly identifies existing delta tables by @rtyler in #2890
• docs: updating usage/managing-tables on optimizing tables by @VillePuuska in #2892
• fix: double-encode paths during zorder optimize when they contain special characters by @rtyler in #2897
• fix: check lowercase config keys with/without aws prefix in s3logstorefactory by @ion-elgreco in #2896
• docs: typo fix by @avriiil in #2902
• docs: introduce GCS backend docs by @avriiil in #2903
• docs: correct Minio docs fix typo by @avriiil in #2904
• refactor: exposing CommitConflictError enum by @Filip-Dziuba in #2907
• docs: add adls backend docs by @avriiil in #2913
• chore: bump the patch version for a 0.20.1 release by @rtyler in #2916

New Contributors

• @helanto made their first contribution in #2826
• @jessy1092 made their first contribution in #2883
• @fecet made their first contribution in #2885
• @feniljain made their first contribution in #2889
• @VillePuuska made their first contribution in #2892
• @Filip-Dziuba made their first contribution in #2907

Full Changelog: python-v0.19.2...rust-v0.20.1

python-v0.20.1

Bug Fixes

• fix: convert last checkpoint json keys to camelCase by @feniljain in #2889
• fix: ensure log store correctly identifies existing delta tables by @rtyler in #2890
• fix: double-encode paths during zorder optimize when they contain special characters by @rtyler in #2897
• fix: check lowercase config keys with/without aws prefix in s3logstorefactory by @ion-elgreco in #2896

Other Changes

• refactor: exposing CommitConflictError enum by @Filip-Dziuba in #2907
• docs: add adls backend docs by @avriiil in #2913
• chore: bump the patch version for a 0.20.1 release by @rtyler in #2916
• docs: typo fix by @avriiil in #2902
• docs: introduce GCS backend docs by @avriiil in #2903
• docs: correct Minio docs fix typo by @avriiil in #2904
• docs: updating usage/managing-tables on optimizing tables by @VillePuuska in #2892

New Contributors

• @feniljain made their first contribution in #2889
• @VillePuuska made their first contribution in #2892
• @Filip-Dziuba made their first contribution in #2907

Full Changelog: python-v0.20.0...python-v0.20.1

python-v0.20.0

New features

• feat(python, rust): add feature operation by @ion-elgreco in #2712
• feat: add support for pyarrow.ExtensionType by @fecet in #2885
• feat: improve AWS credential loading between S3 and DynamoDb code paths by @rtyler in #2887

Bug Fixes

• fix: pin the build-dependencies for Python to a slightly older vendored openssl by @rtyler in #2856
• fix: escaped columns in dataskippingstatscolumns by @ion-elgreco in #2855
• fix: set put mode to overwrite in mount backend by @ion-elgreco in #2861
• fix(rust): scan schema fix for predicate by @sherlockbeard in #2869
• fix(python, rust): use require files by @ion-elgreco in #2809
• fix: re-enable optional old casting behavior in merge by @ion-elgreco in #2853
• fix: conditionally disable enable_io non-unix based systems by @hntd187 in #2884
• fix: stats is optional in add action by @jkylling in #2841
• fix: prepare the next 🦀 release with fixed version ranges by @rtyler in #2875
• fix: pin broken dependencies and changes in 0.19.1 by @rtyler in #2878

Other Changes

• chore: pin the Rust baseline version to 1.80 by @rtyler in #2842
• chore: attempt to ignore all dependabot checks for arrow and datafusion by @rtyler in #2870
• docs: fix typo in delta-lake-dagster by @jessy1092 in #2883
• chore: exclude parquet from dependabot as well by @rtyler in #2874
• chore: rearrange github actions a bit by @rtyler in #2868
• chore: cleanup codecov defaults by @rtyler in #2876
• chore(deps): update sqlparser requirement from 0.50 to 0.51 by @dependabot in #2881
• chore: set max_retries in CommitProperties by @helanto in #2826

New Contributors

• @helanto made their first contribution in #2826
• @jessy1092 made their first contribution in #2883
• @fecet made their first contribution in #2885

Full Changelog: python-v0.19.2...python-v0.20.0

python-v0.19.2: objectstore conditional put

New features

• perf: conditional put for default log store (e.g. azure, gcs, minio, cloudflare) by @ion-elgreco in #2813
• feat: make Add::get_json_stats public by @gruuya in #2822
• refactor(python): add pymergebuilder by @ion-elgreco in #2823
• feat: public method to get partitions for DeltaTable (#2671) by @omkar-foss in #2816
• feat(rust): add operationMetrics to WRITE by @gavinmead in #2838

Bug Fixes

• fix: enable feature flags to which deltalake-core build tokio with enable_io by @rtyler in #2803
• fix(rust): set token provider explicitly by @ion-elgreco in #2817
• fix(python, rust): allow in pushdowns in early_filter by @ion-elgreco in #2807
• fix: use table config target file size, expose target_file_size in python by @ion-elgreco in #2811

Other Changes

• chore(python): raise not implemented in from_data_catalog by @ion-elgreco in #2799
• docs: added WriterProperties documentation by @sherlockbeard in #2804
• chore(python): remove deprecated or duplicated functions by @ion-elgreco in #2801
• test(python): fix optimize call in benchmark by @ion-elgreco in #2812
• docs: fix documentation about max_spill_size by @junhl in #2835
• refactor(python): post_commit_hook_properties derive by @ion-elgreco in #2824
• docs: fix docstring of set_table_properties by @astrojuanlu in #2820
• chore: enable codecov reporting by @rtyler in #2836
• chore(aws): use backon to replace backoff by @Xuanwo in #2840
• chore: update python by @ion-elgreco in #2845
• docs: concurrent writes permission missing by @poguez in #2846

New Contributors

• @junhl made their first contribution in #2835
• @Xuanwo made their first contribution in #2840
• @gavinmead made their first contribution in #2838
• @poguez made their first contribution in #2846

Full Changelog: python-v0.19.1...python-v0.19.2

python-v0.19.1: separate IO runtime

New features

• feat: configurable IO runtime by @ion-elgreco in #2789
• feat(python, rust): added statistics_truncate_length in WriterProperties by @sherlockbeard in #2784
• feat(python, rust): add ColumnProperties And rework in python WriterProperties by @sherlockbeard in #2793

Bug Fixes

• fix: pin maturin verison by @ion-elgreco in #2778
• fix(rust): max_spill_size default value by @mrjsj in #2795
• fix: trim trailing slash in url storage options (#2656) by @omkar-foss in #2775

Other Changes

• chore: update the changelog with the 0.19.0 release by @rtyler in #2774
• style: more consistent imports by @roeap in #2786
• chore: remove some file_actions call sites by @roeap in #2787
• chore(deps): update sqlparser requirement from 0.49 to 0.50 by @dependabot in #2792

New Contributors

• @mrjsj made their first contribution in #2795

Full Changelog: python-v0.19.0...python-v0.19.1