deep_sort
deep_sort copied to clipboard
nwojke
tracking feature
HI if when t = 1, the deep sort detect the person A and save the feature of the person A , when t = 2, the deep sort detect unknown person ,it will use cnn to match the feature of the person A at t=1 and the feature of unknown person at t=2, if the result is ture (same person ) , the deep sort will save the feature of the person A at t=2 , and the feature of the person A at t=1 will forget , right??
thanks
@Phoebe-star the feature of the person A at t=1 will not be forgotten. if the person at t=2 and t=1 is matched, feature at t=2 will be added to variable features in class Track and feature at t=1 is still there, which is implemented in update function of class Track as self.features.append(detection.feature)
thanks~ so, if "the feature at t=1" and "the feature at t= 2" are same , but "the position of person at t=1" and "the position of person at t=2" are not same , the Deep Sort will let the person at t=1 and person at t=2 are different
@Phoebe-star when a person is moving, the position can't be same and the feature at t=1 and t=2 can't be exactly same, they are just similar under a pre-defined threshold which indicates these two person are a same person(with a same id)
sorry ~ my mean is "the position of person at t=1" and "the position of person at t=2" , are large distance ( like 200 pixel distance ) , because bounding box tracking fail. so even the features are same (for tracking success , it have already saved different view features ,like forward and back ) ,but the position are largely different , the Deep Sort will let the person at t=1 and person at t=2 are different , right? or it just similar under a pre-defined threshold which indicates these two person are a same person(with a same id)
@Phoebe-star The tracking result is determined by both feature similarity and the bounding box similarity (such as distance or IOU) in deep_sort, so you can't decide whether they are a same person just by either one of them. And for a new object, in the first n_init frame, feature similarity is not used, just bounding box IOU is used to determine whether they are a same person, you can get the idea by reading the code.
ok! thank you very much ~
but I have another problem
In, generate_detections.py
in line 175,
the tracking will get the "feature (dim 128) " and use the feature to compare cosine distance,
if I design another cnn to exact the feature( dim 1024) , is it available for current code (generate_detections.py) ?
P.S. I take away the features of line175 , and add the code ,
like features = my_network(bgr_image, rows[:, 2:6].copy())
and the features is dim (batch , 1024)
is the program still running? or it have to the features dim (batch, 128)
@Phoebe-star The program still can run, because the feature dimension is not hard-coded in the tracking code
why? for a new object, in the first n_init frame, feature similarity is not used. You mean first use IOU to determine whether they are a same person, if the IOU is small,then use cnn to get feature similarity.
If ,the person A is at the (10,20) of image , then she disappear in the next frame , and next frame she appear,but the person A is at the (600,500) of image. So the deep sort will let the person is not the same. Because it first use IOU, not feature.
@Phoebe-star three things you need to consider
- in the deep sort code, there are 3 types of tracks: tentative, confirmed and deleted. For a new object, it is classified as tentative in the first n_init frames
- only the tracks which are classified as confirmed will use feature similarity, other tracks will use IOU similarity
- for a new object, if it can't be matched using IOU similarity in every frame of the first n_init frames, it will be classified as deleted
you can find all the things above in the paper and code
do you know in the paper "SIMPLE ONLINE AND REALTIME TRACKING WITH A DEEP ASSOCIATION METRIC" listing1
after step 5 , I can not understand, QQ
In line 5 the loop goes over all n=1 ... max_age. Inside the loop, line 6 matches measurements against all tracks with age n. Lines 8 and 9 remove the matched tracks and measurements from the set to be matched in future iterations. Thus, the code matches tracks to measurements, giving priority to tracks of with lower age (i.e., tracks that have been seen more frequently).
thanks, do you know how to change the threshold for association admissible in the code?
is it the NearestNeighborDistanceMetric?
matching_threshold, what does it mean?
Sorry for the late reply. The threshold is here. Instead of changing the threshold you could also increase the motion model uncertainty of the Kalman filter here.
the
is "self._std_weight_position = 1. / 20" ?https://github.com/nwojke/deep_sort/blob/master/deep_sort/kalman_filter.py#L52
@tungduongbk When one track is missing, the state of this track is set to TrackState.Deleted, then this track won't be considered in following tracking process, but the track_id won't be deleted and it can't be used again.
Sorry for the late reply. S is the covariance of the (predicted) track state, projected into measurement space. It is denoted as S_k on Wikipedia.
Instead of IOU matching can we use feature similarity throughout the video. It may improve the accuracy of the system. Is it right?
why? for a new object, in the first n_init frame, feature similarity is not used. You mean first use IOU to determine whether they are a same person, if the IOU is small,then use cnn to get feature similarity.
If ,the person A is at the (10,20) of image , then she disappear in the next frame , and next frame she appear,but the person A is at the (600,500) of image. So the deep sort will let the person is not the same. Because it first use IOU, not feature.
@Phoebe-star The tracking result is determined by both feature similarity and the bounding box similarity (such as distance or IOU) in deep_sort, so you can't decide whether they are a same person just by either one of them. And for a new object, in the first
n_initframe, feature similarity is not used, just bounding box IOU is used to determine whether they are a same person, you can get the idea by reading the code.
how can we use only feature similarity from the start of the frame?
currently i am facing the issue of id switching frequently when two persons cross each other. How can i improve the accuracy?
From: chutki [email protected] Sent: 10 December 2019 10:39 To: nwojke/deep_sort [email protected] Cc: Jini Cherian [email protected]; Comment [email protected] Subject: Re: [nwojke/deep_sort] tracking feature (#48)
@Phoebe-starhttps://github.com/Phoebe-star The tracking result is determined by both feature similarity and the bounding box similarity (such as distance or IOU) in deep_sort, so you can't decide whether they are a same person just by either one of them. And for a new object, in the first n_init frame, feature similarity is not used, just bounding box IOU is used to determine whether they are a same person, you can get the idea by reading the code.
how can we use only feature similarity from the start of the frame?
— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/nwojke/deep_sort/issues/48?email_source=notifications&email_token=AJWEPKFNLN33XOTUNTHMFPDQX4QAZA5CNFSM4ETM3AEKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEGNTMKQ#issuecomment-563820074, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AJWEPKARCGHP3N6NO7HHVATQX4QAZANCNFSM4ETM3AEA.
@Phoebe-star the \lambda is set to 0 in the paper
hi do you know where can I change the lambda in the code? Thanks.
HI I am having the following issues: 1.In deep sort tracker if a Person -A is assigned an id 1. The person then leaves the frame. After few second the Person-A re-enters again. The tracker assigns the same id 1 to the person A. Is this the right behavior of the tracker. If not how to handle this? 2.The other case person-A is assigned an id -1,He leaves the frame and nobody in the frame for a few seconds. Then a new person B enters after few seconds. The same id -1 is assigned to the person -B. How to resolve this issue? Thanks in advance
Hii @sowmiyadharmalingam86
I am using deepSort for the vehicle tracking. I am having the same issue as you described in point 2. How can we not repeat the track id. Can anyone has any idea where to change in the algorithm so that track id wont be repeated. Thank you.
currently i am facing the issue of id switching frequently when two persons cross each other. How can i improve the accuracy? … ________________________________ From: chutki [email protected] Sent: 10 December 2019 10:39 To: nwojke/deep_sort [email protected] Cc: Jini Cherian [email protected]; Comment [email protected] Subject: Re: [nwojke/deep_sort] tracking feature (#48) @Phoebe-starhttps://github.com/Phoebe-star The tracking result is determined by both feature similarity and the bounding box similarity (such as distance or IOU) in deep_sort, so you can't decide whether they are a same person just by either one of them. And for a new object, in the first n_init frame, feature similarity is not used, just bounding box IOU is used to determine whether they are a same person, you can get the idea by reading the code. how can we use only feature similarity from the start of the frame? — You are receiving this because you commented. Reply to this email directly, view it on GitHub<#48?email_source=notifications&email_token=AJWEPKFNLN33XOTUNTHMFPDQX4QAZA5CNFSM4ETM3AEKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEGNTMKQ#issuecomment-563820074>, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AJWEPKARCGHP3N6NO7HHVATQX4QAZANCNFSM4ETM3AEA.
I'm facing this issue too, were you able to resolve this?
