auto-correct does not appear to assign recognized identities

[aperkes]

This could certainly be user error, but from what I can tell, visual identification has worked successfully, but auto-correct isn't changing the identities based on the recognition. I updated to the most recent version to confirm it wasn't an older bug.

As you can see in the image, it's very confident that the green fish should be fish0 (which is correct) but it has still assigned it as fish1

Full terminal readout when I run auto-correct, in case that's helpful: trex-output.txt

I'm using the most recent version of trex (1.1.6). The same thing was happening in 1.1.3 as well.

Thanks!

EDIT: For background, it's not just individual frames, but an entire tracks (>100 consecutive frames all with the same assigned locID) This also occurs both in sections (marked yellow or red) with stable tracks for all individuals.

Here's the source video, if you want to try to recreate the issue: https://drive.google.com/drive/folders/1Uopqn_he3FGuGDsPYrN_IBKJvcu7xlYa?usp=sharing

Parameters used:

-blob_size_range [1,6] -threshold 40 -track_intensity_range [50,150] -track_max_speed 30

This is just a little pilot test to get the pipeline set up, so I haven't made sure the time and distance measures are real-world accurate, but I could do that if that's important.

[mooch443]

Hey, thanks for posting this on GitHub as well, and thanks for the video! This usually helps a bunch when debugging/trying to help. I actually fixed an unrelated bug in the process :-)

As to your error: I think first of all that ther eare likely a few culprits with your system. Your fish are really big in-frame, meaning that the default recognition_image_size of [80,80] will probably not cut it. However, luckily we don't really need all that detail. I attached the settings I used - essentially I downscaled the images by half, fitting them inside a 64x64px frame.

With that I got a uniqueness of more than 90% within a few iterations (should be enough for 3 individuals). If you don't think that's enough, then manually set gpu_accepted_uniqueness to 0.95 (or something between 0.9-0.9999) and see how long that takes!

The second thing is that you have (also in your screenshot) a segment of consecutive frames that is extremely short - it's only 1 frame. TRex will not assign consecutive segments that have fewer than ~5 samples since that is likely not enough to determine identity with certainty. I plan to add an option so that you can change this, but currently you can't. Basically what's supposed to happen is that the "normal" tracking takes over and assigns correct/incorrect identities based on kinematics. If there is a mistake then it'll be corrected in consecutive assignments. So in general: Make sure your consecutives are as long as possible and only stop if you want them to - e.g. when individuals overlap or the individual is actually gone. Keeping blob_size_ranges tight, speed as well, but not too tight, should help here.

tracking settings:

blob_size_ranges = [[1,6]]
recognition_image_scale = 0.5
recognition_image_size = [64,64]
track_max_individuals = 3
track_max_speed = 30
track_threshold = 40

Does this solve your issue? :-)

[aperkes]

This is super helpful to know, thanks, I didn't realize the trajectories were split, that explains why it's not assigning it, but now I'm having trouble understanding a) why trajectories are split, and b) how to prevent them from being split.

I'm playing around with blob and speed (using your settings), but it's very persistent about making a new trajectories each frame. It only says "ended because: "ProbabilityTooSmall". What are the parameters that go into Probability? I understand that track max speed could factor in, but even setting their track max speed very close to the speed between frames, the frames end. Interestingly, even within the consec chosen for training, the frames are still often only 1 long, which doesn't make any sense to me. Were you able to get most of the tracks to be longer than 1 frame?

[mooch443]

I would think it might still be set too low. It isn't mapped 1:1 to the assignments. Some other factors include:

• the calculated probability is lower than track_trusted_probability and thus the assignment is untrusted when it comes to segment generation
• some of the factors might be squared. enable gui_show_probabilities to see a visualisation for the current probabilities for the focal individual
• potential overlaps

I did indeed not get many long segments, although this warrants some investigation on my part. I will update some of this behavior in the next version I think. For now, increasing the maximum speed to 100 seems to work, or lowering the trusted probability.

Also make sure that you set the cm_per_pixel. This can be done either during conversion by setting the meta_real_width parameter, or in TRex by clicking on-screen and selecting a known length (two points) within the empty space in tracking view via CTRL/CMD+left-click and then clicking the "set known length" button. This changes blob_size_ranges, max_speed etc. values, so be sure to always do this first.

I will also add this to the "documentation" project. All of this is important to know. Also, potentially I could say a few more things in the GUI when describing the issues on hover. (Also I generally dislike making things only visible upon magic hover, but I spontaneously didnt see a better way.)