Skip to content

No weight decay for norm/bias/tokens/etc. for LTDETR#598

Open
guarin wants to merge 12 commits intomainfrom
trn-1803-ltdetr-no-weight-decay-for-bias-norms-and-tokens
Open

No weight decay for norm/bias/tokens/etc. for LTDETR#598
guarin wants to merge 12 commits intomainfrom
trn-1803-ltdetr-no-weight-decay-for-bias-norms-and-tokens

Conversation

@guarin
Copy link
Contributor

@guarin guarin commented Feb 3, 2026

What has changed and why?

Handle LTDETR weight decay and optim groups

How has it been tested?

  • Manually

Did you update CHANGELOG.md?

  • Yes
  • Not needed (internal change)

Did you update the documentation?

  • Yes
  • Not needed (internal change without effects for user)

Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 27398c54ff

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@guarin
Copy link
Contributor Author

guarin commented Feb 5, 2026

@codex review

@guarin
Copy link
Contributor Author

guarin commented Feb 5, 2026

/review

Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e4f91aff42

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Base automatically changed from guarin-trn-1803-no-weight-decay-for-bias-norms-and-tokens to main February 5, 2026 15:07
Copilot AI review requested due to automatic review settings February 5, 2026 16:47
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR refactors weight decay handling for LTDETR object detection models (both DINOv2 and DINOv3 variants) by replacing regex-based parameter grouping with explicit module-based parameter grouping. The change uses optimizer_helpers.get_weight_decay_parameters() to identify parameters that should not have weight decay applied (such as biases, normalization layers, tokens, and embeddings).

Changes:

  • Replaced regex pattern matching with explicit module-based parameter collection from backbone, encoder, and decoder
  • Split parameters into groups with and without weight decay for backbone, detector, and default parameters
  • Removed unused re module import and added optimizer_helpers import

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 10 comments.

File Description
src/lightly_train/_task_models/dinov3_ltdetr_object_detection/train_model.py Refactored get_optimizer method to use module-based parameter grouping instead of regex, splitting parameters by weight decay requirements
src/lightly_train/_task_models/dinov2_ltdetr_object_detection/train_model.py Applied the same refactoring as DINOv3 variant for consistency

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant