Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BtS accumulation with non-multiples #542

Open
MaxiBoether opened this issue Jun 22, 2024 · 0 comments
Open

BtS accumulation with non-multiples #542

MaxiBoether opened this issue Jun 22, 2024 · 0 comments

Comments

@MaxiBoether
Copy link
Contributor

Currently, Batch-Then-Sample is limited to batch sizes and selection ratios where batch size % batch_size * ratio == 0, e.g. 50% selection ratio. This is because from each batch we select ratio * size samples, and then build a new batch after n batches.

In Sample-Then-Batch, we don't have this limitation. It's a bit unclear to me whether we can find a solution in which the number of samples we train on is exactly identical between StB and BtS. One idea for BtS is to round the target size per batch (up or down?), and then at some point we selected a little too much points. We can use them for the next batch. However, it is not clear to me if this will lead to identical behavior to StB. We should synchronize the number of trained batches to be able to fairly compare BtS and StB.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant