Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Incorrect default for ColumnConfig. run_for_all_data_type and ColumnConfig. limit_data_type #85

Open
mikita-sakalouski opened this issue Oct 29, 2024 · 0 comments
Assignees
Labels
bug Something isn't working
Milestone

Comments

@mikita-sakalouski
Copy link
Contributor

Describe the bug

Currently we are using the following default values for ColumnConfig. run_for_all_data_type and ColumnConfig. limit_data_type :

run_for_all_data_type: Optional[List[SparkDatatype]] = [None]
limit_data_type: Optional[List[SparkDatatype]] = [None]
data_type_strict_mode: bool = False

and we have checks for validating that run_for_all_data_type exists:

if columns[0] == "*" and not run_for_all_data_type:
            raise ValueError("Cannot use '*' as a column name when no run_for_all_data_type is set")

but [None] is always exists, as it is list with 1 element which is equal None.

Steps to Reproduce

  1. Initiate the ColumnConfig class
  2. Call run_for_all_data_type attribute
  3. Check for existing of attribute

Expected behavior

Default value should be changed to None and code should be fixed to provide correct behavior.

@mikita-sakalouski mikita-sakalouski added the bug Something isn't working label Oct 29, 2024
@mikita-sakalouski mikita-sakalouski self-assigned this Oct 29, 2024
@mikita-sakalouski mikita-sakalouski added this to the 0.9.0 milestone Oct 29, 2024
@dannymeijer dannymeijer modified the milestones: 0.9.0, 0.10.0 Nov 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
Status: No status
Development

When branches are created from issues, their pull requests are automatically linked.

2 participants