Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bugfix generic-k code in top-k with softmax #1993

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

t4c1
Copy link

@t4c1 t4c1 commented Dec 17, 2024

Fixes a bug in generic top-k softmax EVT implementation that resulted in wrong results when k != 2 and k != 4.

This allows removal of the static assert requiring k to be either 2 or 4. Comment in example 61 is also fixed to reflect that any k value is now supported.

@alihassanijr
Copy link
Contributor

@t4c1 Thank you for submitting this patch.

Just a note on the assertion, it's there more as a warning to users that the generic sort comes with serious performance implications. Due to the control flow associated with it, it introduces somewhat heavy branching and register spilling, resulting in a smaller improvement over the baseline.

I'll let @hwu36 chime in on whether we want to get rid of the assertion.

@hwu36
Copy link
Collaborator

hwu36 commented Dec 26, 2024

@alihassanijr , maybe leave it there but make the text clearer about the consequences?

@alihassanijr
Copy link
Contributor

@hwu36 leave the assert there with a better message, or remove and make the doc clearer about the consequences?

@hwu36
Copy link
Collaborator

hwu36 commented Dec 26, 2024

Leave it there and make the message better so people willing to try can try it.

@t4c1
Copy link
Author

t4c1 commented Jan 15, 2025

I addressed the review comments some time ago. Is something else required to get this merged?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants