Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improvement: add a "division by zero" check in chunked loss handling in kd_losses.py #2225

Open
insop opened this issue Jan 2, 2025 · 3 comments
Assignees
Labels
enhancement New feature or request

Comments

@insop
Copy link
Contributor

insop commented Jan 2, 2025

Suggested improvement:

It would be good to add a "division by zero" check in chunked loss handling in kd_losses.py.

Context:
This is based on the discussion in PR #2094.

Potential issue:
ForwardKLWithChunkedOutputLoss does not check for division by zero, while the non-chunked version does.

@insop insop changed the title add a "division by zero" check in chunked loss handling in kd_losses.py Improvement: add a "division by zero" check in chunked loss handling in kd_losses.py Jan 2, 2025
@joecummings joecummings added the enhancement New feature or request label Jan 6, 2025
@joecummings
Copy link
Contributor

Great catch! We'd definitely welcome a small PR for this if you want to do it, otherwise we can try to get to it soon.

@insop
Copy link
Contributor Author

insop commented Jan 6, 2025

I can make a change. @felipemello1 , let me know if you prefer to do it or have a change already.

@felipemello1
Copy link
Contributor

That would be awesome. Thanks @insop :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants