No weight decay for norm/bias/tokens/etc. for LTDETR#598
No weight decay for norm/bias/tokens/etc. for LTDETR#598
Conversation
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 27398c54ff
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
src/lightly_train/_task_models/dinov2_ltdetr_object_detection/train_model.py
Outdated
Show resolved
Hide resolved
src/lightly_train/_task_models/dinov3_ltdetr_object_detection/train_model.py
Outdated
Show resolved
Hide resolved
…ns' into trn-1803-ltdetr-no-weight-decay-for-bias-norms-and-tokens
|
@codex review |
|
/review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e4f91aff42
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
src/lightly_train/_task_models/dinov2_ltdetr_object_detection/train_model.py
Show resolved
Hide resolved
There was a problem hiding this comment.
Pull request overview
This PR refactors weight decay handling for LTDETR object detection models (both DINOv2 and DINOv3 variants) by replacing regex-based parameter grouping with explicit module-based parameter grouping. The change uses optimizer_helpers.get_weight_decay_parameters() to identify parameters that should not have weight decay applied (such as biases, normalization layers, tokens, and embeddings).
Changes:
- Replaced regex pattern matching with explicit module-based parameter collection from backbone, encoder, and decoder
- Split parameters into groups with and without weight decay for backbone, detector, and default parameters
- Removed unused
remodule import and addedoptimizer_helpersimport
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 10 comments.
| File | Description |
|---|---|
| src/lightly_train/_task_models/dinov3_ltdetr_object_detection/train_model.py | Refactored get_optimizer method to use module-based parameter grouping instead of regex, splitting parameters by weight decay requirements |
| src/lightly_train/_task_models/dinov2_ltdetr_object_detection/train_model.py | Applied the same refactoring as DINOv3 variant for consistency |
What has changed and why?
Handle LTDETR weight decay and optim groups
How has it been tested?
Did you update CHANGELOG.md?
Did you update the documentation?