Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[enhancement] Refactor onedal/datatypes in preparation for dlpack support #2195

Open
wants to merge 13 commits into
base: main
Choose a base branch
from

Conversation

icfaust
Copy link
Contributor

@icfaust icfaust commented Nov 28, 2024

Description

PR #1568 is too large and too out of date to use for refactoring the onedal table object. Subsequent PRs which introduced sycl_usm_array support mixed in additional C++ code in an unsustainable/ poorly extensible manner. This PR tries to find an intermediate solution which does not rewrite onedal/datatypes but only shifts the code in a way which is clearer and better separated following #1568 in spirit. This way adding dlpack support to table will be separated cleanly from numpy and sycl_usm_array. It defines additional namespaces so that it is clear in the code which conversion mechanism is being used where. It begins standardization of the naming schemes of functions and files for simple extensibility in the future (say supporting other array or possibly dataframe types).

Therefore, because it is only namespace definitions and file movements, performance benchmarks aren't necessary.

NOTE: A function convert_from_sua_iface is renamed convert_to_table in the sycl_usm namespace.

NOTE: changes to pickling for incremental algorithms must also be done. It is clear with the changes what types of inputs/outputs are expected for __getstate__ and __setstate__, which will default to numpy (not explicitly discussed in those PRs, but may be important in future array_api discussions).

NOTE: there are definitely inconsistencies between the sycl_usm and numpy implementations, and changes to those implementations in any functional manner should be done in a follow-up PR/ticket. They are likely to be time-intensive and require benchmarking.


PR should start as a draft, then move to ready for review state after CI is passed and all applicable checkboxes are closed.
This approach ensures that reviewers don't spend extra time asking for regular requirements.

You can remove a checkbox as not applicable only if it doesn't relate to this PR in any way.
For example, PR with docs update doesn't require checkboxes for performance while PR with any change in actual code should have checkboxes and justify how this code change is expected to affect performance (or justification should be self-evident).

Checklist to comply with before moving PR from draft:

PR completeness and readability

  • I have reviewed my changes thoroughly before submitting this pull request.
  • I have commented my code, particularly in hard-to-understand areas.
  • I have updated the documentation to reflect the changes or created a separate PR with update and provided its number in the description, if necessary.
  • Git commit message contains an appropriate signed-off-by string (see CONTRIBUTING.md for details).
  • I have added a respective label(s) to PR if I have a permission for that.
  • I have resolved any merge conflicts that might occur with the base branch.

Testing

  • I have run it locally and tested the changes extensively.
  • All CI jobs are green or I have provided justification why they aren't.
  • I have extended testing suite if new functionality was introduced in this PR.

Performance

  • I have measured performance for affected algorithms using scikit-learn_bench and provided at least summary table with measured data, if performance change is expected.
  • I have provided justification why performance has changed or why changes are not expected.
  • I have provided justification why quality metrics have changed or why changes are not expected.
  • I have extended benchmarking suite and provided corresponding scikit-learn_bench PR if new measurable functionality was introduced in this PR.

@icfaust
Copy link
Contributor Author

icfaust commented Jan 26, 2025

Introducing dlpack first requires refactoring the datatypes directory in order to reduce tech debt and simplify reviewing. This PR will drop dlpack support to a second PR and will focus on refactoring directory.

@icfaust icfaust closed this Jan 26, 2025
@icfaust icfaust force-pushed the dev/dlpack_to_table branch from 18a9d4d to 058372f Compare January 26, 2025 21:50
@icfaust icfaust reopened this Jan 26, 2025
@icfaust icfaust changed the title [WIP] add dlpack support [WIP] Refactor onedal/datatypes in preparation for dlpack support Jan 26, 2025
@icfaust
Copy link
Contributor Author

icfaust commented Jan 26, 2025

/intelci: run

@icfaust icfaust marked this pull request as ready for review January 26, 2025 23:44
Copy link

codecov bot commented Jan 26, 2025

Codecov Report

Attention: Patch coverage is 16.21622% with 31 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
onedal/basic_statistics/basic_statistics.cpp 8.33% 0 Missing and 11 partials ⚠️
onedal/decomposition/pca.cpp 0.00% 2 Missing and 7 partials ⚠️
onedal/covariance/covariance.cpp 14.28% 0 Missing and 6 partials ⚠️
onedal/linear_model/linear_model.cpp 25.00% 0 Missing and 3 partials ⚠️
onedal/datatypes/table.cpp 60.00% 0 Missing and 2 partials ⚠️
Flag Coverage Δ
azure 77.96% <ø> (ø)
github 72.07% <16.21%> (+1.05%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
onedal/datatypes/table.cpp 60.52% <60.00%> (ø)
onedal/linear_model/linear_model.cpp 55.88% <25.00%> (ø)
onedal/covariance/covariance.cpp 56.73% <14.28%> (-0.56%) ⬇️
onedal/decomposition/pca.cpp 43.57% <0.00%> (-0.32%) ⬇️
onedal/basic_statistics/basic_statistics.cpp 41.93% <8.33%> (ø)

... and 5 files with indirect coverage changes

icfaust added a commit to icfaust/scikit-learn-intelex that referenced this pull request Jan 27, 2025
@icfaust icfaust changed the title [WIP] Refactor onedal/datatypes in preparation for dlpack support [enhancement] Refactor onedal/datatypes in preparation for dlpack support Jan 27, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant