Incremental learning

[BossCrab-jyj]

Search before asking

• I have searched the YOLOv5 issues and found no similar feature requests.

Description

I searched for all the related content on incremental learning in the issue tracker and conducted many experiments, but I found that the performance of incremental learning was poor. In the current object detection project using YOLO, we often need to continuously add new data to the original dataset to improve the model's accuracy. However, retraining the model using the entire dataset each time requires significant computational cost. Therefore, I attempted to use incremental learning to solve this problem. I trained a model using 30,000 images and obtained the best model, then added 50 new images to the original 30,000 for retraining. The number of classes in the new data was consistent with the original data. I initially thought that training for 50 epochs would suffice, but unexpectedly, the mAP kept decreasing. After training for 20 epochs, the mAP dropped from 0.92 to 0.1. I then reduced the learning rate to one-tenth of the original value, but the performance was still poor. On top of this, I froze the backbone layers and only trained the head, but the results were still unsatisfactory. Now, I have no choice but to retrain the model using the official pre-trained weights.
I'm not sure if my approach is incorrect or if YOLO doesn't support incremental learning, but if this issue can be resolved, YOLO would see a significant improvement.

Use case

No response

Additional

No response

Are you willing to submit a PR?

• Yes I'd like to help by submitting a PR!

[UltralyticsAssistant]

👋 Hello @BossCrab-jyj, thank you for your interest in YOLOv5 🚀! This is an automated response to help you get support faster. An Ultralytics engineer will review your issue and assist you soon.

Please visit our ⭐️ Tutorials to get started, where you can find quickstart guides for simple tasks like Custom Data Training all the way to advanced concepts like Hyperparameter Evolution.

If this is a 🐛 Bug Report, please provide a minimum reproducible example (MRE) to help us debug it:

• Exact training/validation commands you ran and full logs🧪
• YOLOv5 version/commit, PyTorch/CUDA versions, and OS🛠️
• A tiny sample of your dataset (a few images + labels) that reproduces the issue🗂️
• Your hyperparameters and any changes to defaults (e.g., LR, epochs, frozen layers)⚙️

If this is a custom training ❓ Question, please provide as much information as possible, including dataset image examples and training logs, and verify you are following our Tips for Best Training Results.

Requirements

Python>=3.8.0 with all requirements.txt installed including PyTorch>=1.8. To get started:

git clone https://github.com/ultralytics/yolov5  # clone
cd yolov5
pip install -r requirements.txt  # install

Environments

YOLOv5 may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):

• Notebooks with free GPU:
• Google Cloud Deep Learning VM. See GCP Quickstart Guide
• Amazon Deep Learning AMI. See AWS Quickstart Guide
• Docker Image. See Docker Quickstart Guide

Status

If this badge is green, all YOLOv5 GitHub Actions Continuous Integration (CI) tests are currently passing. CI tests verify correct operation of YOLOv5 training, validation, inference, export and benchmarks on macOS, Windows, and Ubuntu every 24 hours and on every commit.

Thanks for the detailed description on incremental learning and the performance drop you observed. To help us investigate, please share the MRE details above along with:

• Baseline checkpoints used and the exact retraining procedure (commands, epochs)🔁
• Validation set composition and whether it changed between runs📊
• Any freezing settings and the final training/validation metrics per epoch📉

Once we have this, we can better triage and guide next steps. 🙏

[pderrenger]

Thanks for the detailed context. YOLOv5 doesn’t include built‑in continual/incremental learning algorithms; the supported workflow is periodic fine‑tuning on a merged dataset. The 0.92→0.10 mAP collapse is most often a class‑ID/name mismatch or a changed val split, not LR. Please sanity‑check first, then continue training from your last good checkpoint:

sanity checks

python val.py --weights best.pt --data data_orig.yaml
python val.py --weights best.pt --data data_merged.yaml

incremental update (merged data + small LR; oversample new images if needed)

python train.py --weights best.pt --data data_merged.yaml --epochs 10 --lr0 0.001 --image-weights

Keep data.yaml names/order identical across runs, ensure all label .txt IDs match, and include a small replay subset (e.g., 5–20%) of old images to avoid forgetting. Please pull the latest YOLOv5 and confirm the behavior persists. See the Fine‑tuning guide and our note on data‑drift mitigation for why this approach works in practice, and for general usage see the Ultralytics Docs.

[BossCrab-jyj]

Thank you for your reply. I think I found the problem that I need to strip the optimizer from the trained model first and then use it as a pre-trained model. This way, the training can converge normally.

[pderrenger]

Great catch—stale optimizer/scheduler state from resuming was likely the culprit. For incremental updates, start a fresh run (omit --resume) and load weights only; or strip once with: python -c "import torch; c=torch.load('best.pt','cpu'); c.pop('optimizer', None); torch.save(c,'best_noopt.pt')". See the Training mode docs for resume/checkpoint behavior.