Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The loss computed in the last step is significantly smaller compared to the previous steps #709

Open
rayuron opened this issue Mar 5, 2024 · 0 comments

Comments

@rayuron
Copy link

rayuron commented Mar 5, 2024

tfrs.tasks.Retrieval uses the tf.keras.losses.CategoricalCrossentropy loss function by default.
I think that the number of classes in tf.keras.losses.CategoricalCrossentropy equals the batch size when we don't set the num_hard_negatives.

Usually, the batch size in the last step is smaller than the batch size we have set.
Therefore, it seems that the loss in the last steps during training is smaller than in the previous steps.

I was confused by the small loss values because the logs only display the loss from the last step when outputting the loss for each epoch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant