Skip to content

Commit

Permalink
address PR comments
Browse files Browse the repository at this point in the history
  • Loading branch information
ravi-kumar-pilla committed Dec 17, 2024
1 parent 4ca50fc commit d900b97
Show file tree
Hide file tree
Showing 2 changed files with 6 additions and 5 deletions.
2 changes: 1 addition & 1 deletion kedro-datasets/RELEASE.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,11 +19,11 @@
- Improved the dependency management for Spark-based datasets by refactoring the Spark and Databricks utility functions used across the datasets.
- Added deprecation warning for `tracking.MetricsDataset` and `tracking.JSONDataset`.
- Moved `kedro-catalog` JSON schemas from Kedro core to `kedro-datasets`.
- Removed file handling using Ibis's backends from `ibis.TableDataset`. `ibis.FileDataset` will handle loading and saving files using Ibis's backends.

## Breaking Changes

- Demoted `video.VideoDataset` from core to experimental dataset.
- Removed file handling capabilities from `ibis.TableDataset`. Use `ibis.FileDataset` to load and save files with an Ibis backend instead.

## Community contributions

Expand Down
9 changes: 5 additions & 4 deletions kedro-datasets/tests/ibis/test_table_dataset.py
Original file line number Diff line number Diff line change
Expand Up @@ -100,10 +100,11 @@ def test_exists(self, table_dataset, dummy_table):
table_dataset.save(dummy_table)
assert table_dataset.exists()

def test_load_extra_params(self):
"""Test overriding the load arguments."""
table_dataset = TableDataset(table_name="test", load_args={"database": "test"})
assert "database" in table_dataset._load_args
@pytest.mark.parametrize("load_args", [{"database": "test"}], indirect=True)
def test_load_extra_params(self, table_dataset, load_args):
"""Test overriding the default load arguments."""
for key, value in load_args.items():
assert table_dataset._load_args[key] == value

@pytest.mark.parametrize("save_args", [{"materialized": "table"}], indirect=True)
def test_save_extra_params(self, table_dataset, save_args, dummy_table, database):
Expand Down

0 comments on commit d900b97

Please sign in to comment.