Detection and Tracking Process
The detection process outlined in Tracking Pipeline is detailed here. To localize the mice, we use a system known as You Only Look Once, developed by Joseph Redmon. Its speed and accuracy make it an ideal candidate for this application, but it does not handle everything for us. The tracking is up to us, and it is handled as follows.
YOLO, as it shall henceforth be called, is essentially a bounding-box regressor. We use this to obtain accurate localizations of the up to four mice within the frame, returned as a tuple of center location, width, and height. We can use this to continue tracks with a greedy approach, using Intersection Over Union as our main tracking information.
The most reliable method of continuing tracks is to calculate the intersection of each bounding box with the previous bounding box, and divide it by the union of the two bounding boxes. This provides us with an accurate likelihood that the new detection corresponds to the previous detection, but it does not provide enough information to handle all cases.
We assume that mice follow Newtonian physics generally. That is, they tend to either stay in motion or at rest, and when they are in motion they tend to be in motion in the same direction. In this manner, we determine the new velocity for each track if we assign the detection, and compare it to the old velocity of the track. The normalized delta of the velocity is subtracted from the IOU, ensuring that tracks are assigned in a more robust fashion.
Referencing Extending IOU Based Multi-Object Tracking by Visual Information, we determined a method for continuing tracks of mice where the detections have disappeared, by using a Visual Tracker (such as CSRT) to join gaps. The trackers are limited in both range and time, allowing them to track mice through occlusions without producing false positives.
When the tracks of mice disappear, it is not necessarily true that they cannot be re-identified when they return. In fact, if only one mouse's track disappears (unless it is by the specified entrance area), they can be identified at any point once they return. If two mice disappear, as long as they are not close by, they can potentially be re-identified upon return. Of course, the longer they are untracked for, the more likely that they will have moved into each others' old positions. To prevent this, we start up a counter for each mouse while they are partially lost, and remove them when the counter passes a defined threshold.
When mice walk over the RFID readers, the information is encoded as in Recording Process. We read this information with OpenCV and perform one of the following actions.
If a mouse walks over a reader and is confirmed as its own tag, then we simply update the validation point of each mouse.
When a dummy mouse walks over a reader and is confirmed as a currently lost mouse, then we assign this mouse the new tag and propagate the track of the dummy mouse onto the new tag.
In the case where two mice come near to each other, then come apart, then come together again, we have two potential error regions. If a mouse walks over a reader and is determined to be a tag other than its current reference, then we can determine the accurate track up until the most recent occlusion, and switch those tracks over. The earliest occlusions that come after the last RFID validation mark the end of the tracks we can be certain of, and those sections stay on the original tags. All of the track in the middle must be lost, as it cannot be determined.
In the case of an identity swap, but one of the mice is a dummy tag, then the swap works mostly the same, but the dummy information must be lost, as none of it can be confirmed.
Once the system has reached the end of a video, all the information in each MouseTracker object is saved to a JSON file for later processing.