Photo from fisheries.noaa.gov
On this repository, detection of the Black Sea Bass (Centropristis Striata) Fish Species and estimation the number of fish on each frame for the 2021 Machine Learning MATE ROV Microsoft Azure Computer Coding Challenge Task 2.
- The Dataset is created by using Roboflow.
- YOLOv5l is used for detection on the Custom Dataset.
- Fish detection is performed on the video stream that is provided by the Gray’s Reef National Marine Sanctuary.
- Number of detected fish is displayed on the right top corner of each frame
- About the Competition
- About the Model and the Data
- How to Detect Yourself?
- My Observations About the Task
- Discussion
Me and my friends, as DEU ROV TEAM, attend to the 2021 Machine Learning MATE ROV Microsoft Azure Computer Coding Challenge!
The Mission: It is cumbersome to do fish counts from video transects. Start with pre-built Artificial Intelligence models like Microsoft Computer Vision and Video Indexer OR take it to the next level by building your own machine learning (aka coding) models.
Task 2: Address this issue by harnessing the power of cloud computing, machine learning/artificial intelligence, and image recognition/processing. Create a program that counts how many fish are seen at any one instant during a video transect.
Video transect: Computer Coding Challenge Round #2 video from m the Gray’s Reef National Marine Sanctuary. Video courtesy of diver/videographer Randy Rudd.
To be able to detect Fish, the data is collected from the Video frame by frame but only 0th, 10th and 20th frames are extracted for each second to create a dataset with about 300 images.
The dataset is labeled and splitted as Train, Validate and Test.
Original version: TRAIN 270, VALID 41, TEST 8
Preprocessing: Resize 416x416
Augmented version: Flip Horizontal, Saturation Between -30% and +30%, Blur Up to 0.5px, TRAIN(808 images, 94%) VALID(41 images, 5%) TEST(8 images, 1%)
Then, another dataset version is also created by rebalancing the dataset and changing the augmentation process. This time the rates were as 95%(828 images), 5%(40 images) and 0%(1 images) respectively but the performance did not change dramatically.
Then, YOLOv5 tutorial of Ultralytics and Custom Dataset application tutorial of RoboFlow are followed to use YOLOv5l model for the detection of the fishes. Also you can check the research paper of You Only Look Once (YOLO).
Additionally, it was necessary to display number of fish. Therefore, another variable is inserted into detect.py file and putText is used, then the python file is saved as detect_fish.py.
Original detect.py is performed for training and the modified version detect_fish.py is performed to display counted fish number at the "Instead of Test Set" part.
You can simply follow my colab tutorial which was explained in detail.
- Download original YOLOv5 repository and extract the files - Install Dependencies
- Download the custom dataset
- Define Model Configuration and Architecture
Important! You need to replace and use my detect.py file, after training and testing, to display number of fish by yourself!
Besides, we preferred to use large one instead of s, m or x. You can use any of these as you wish. TIP
- Train Custom Yolov5 Detector
- Evaluate Custom Yolov5 Detector Performance
- Run Inference With Trained Weights
- Export Trained Weights for Future Inference
- Instead of Test Set (Use Individual Images or Videos)
It was quite fun and we have learned a lot by this project. The task was extremely challenging since the number of fish was above 50 at some parts and it was so hard to distinguish any fish that are small and at the background. However, the performance was quite enough for using only around 300 labeled images. The accuracy of detection can be improved by labeling more images and training longer without overfitting.
If you have any question and suggestion, please do not hesitate to contact me.