Mismatch between detections and class ids when using bytetracker

[rakonjac-uros]

Hello,

When using ByteTrack tracker I encountered and issue that the tracked objects and class ids / class names are sometimes mismatched. I see that in the detect_and_track function, all class ids are extracted and all detections are sent to the _tracker_update, but the tracker update filters out some of the targets with the box shape criteria resulting in a difference between numbers of bboxes_xyxy, ids and scores and number of class_ids returned by the detect_and_track function. Could this cause the error while displaying detections since the index of each bounding box does not match the index of the class id and ultimately class name?
If so, could a solution be to modify the _tracker_update to add an online_class_ids list and append the class ids extracted from dets, for each online target that passes the box shape criteria and return that list along with online_xyxys, online_ids, online_scores and use it later instead of the original class_ids?

Thanks in advance!

[smrfeld]

Not sure if it is related, but in other detectors like deepsort, there is also mismatch between the length of class_ids and bbox_xyxy_list,ids,scores in bbox_xyxy_list, ids, scores, class_ids = bbox_details

[smrfeld]

Here is a very simple test:

bbox_xyxy_list, ids, scores, class_ids = bbox_details
assert len(bbox_xyxy_list) == len(ids) == len(class_ids) == len(scores)

and here are the results for different trackers

    BYTETRACK = 0 # Not OK
    DEEPSORT = 1 # Not OK
    NORFAIR = 2 # OK
    MOTPY = 3 # OK
    OCSORT = 4 # OK
    STRONGSORT = 5  # OK

Both BYTETRACK and DEEPSORT are not working correctly from this bug