open_model_zoo icon indicating copy to clipboard operation
open_model_zoo copied to clipboard

Suggestion for new model type

Open edward9112 opened this issue 5 years ago • 27 comments

I have a suggestion for a model for quite common use case: people detection in an overhead camera scene. Regular models like person-detection-retail just won't detect in such circumstances but this is the best camera position for non-occlusive people counting. It would be great if you add this type of model. I could assist with data collection if necessary.

Here is the image example: https://ibb.co/37Ybx5b

edward9112 avatar Nov 02 '20 17:11 edward9112

@snosov1 could you please review this request?

vladimir-dudnik avatar Nov 02 '20 19:11 vladimir-dudnik

Hey, @edward9112 !

We've considered such a use case previously. The closest we could get is the set of "crossroad" models that has a certain amount of "overhead" data in the training set. Though, on practice it still requires the camera to be somewhat higher up than you show on your example image.

That said, we could consider looking into it again. What kind of help in data collection do you imply?

snosov1 avatar Nov 05 '20 07:11 snosov1

Hi @snosov1 ! I can provide video footage or assist with image annotation. Here are some more typical examples: https://ibb.co/mBpwYjP https://ibb.co/8DYXH0q https://ibb.co/WP1D8jL

edward9112 avatar Nov 05 '20 08:11 edward9112

  1. What volumes of data are we talking about (minutes, hours, days, weeks)?
  2. What is the number of locations?
  3. Will you be able to make this data public under some permissive terms (otherwise, getting it into our premises will surely take a while and might not even be possible in the end)?

As for the annotation - we have the means to do somewhat large-scale annotation with our resources (given that we have satisfactory answers on the questions above).

snosov1 avatar Nov 05 '20 08:11 snosov1

  1. Days, probably weeks of video data
  2. Could be 5-6 plus I could try to scrape some public sources as well which should add 10-20 locations
  3. Not sure about this. Does sharing it via online channel for annotation only make it public? Or you add the data to some kind of public archive?

edward9112 avatar Nov 05 '20 08:11 edward9112

1-2. That sounds like something we can work with.

With regards to third item - the main reason I'm asking is that we, as a company, have to get data with clear terms on how we can use it. One of the simplest ways is if the dataset is public (i.e. accessible to general public, not only to us) with clear and non-restrictive terms of use/licensing. If you need to keep it between our parties - then there have to be some explicit license agreement between our companies (and if you're just you and not a company that we can have such kind of agreement with - then it's simply a no-go from our side).

snosov1 avatar Nov 05 '20 11:11 snosov1

Do you have a draft of such agreement? How do I make a dataset public?

edward9112 avatar Nov 05 '20 18:11 edward9112

Do you have a draft of such agreement?

No. When we purchase/acquire data - the providing company gives us its terms for review.

How do I make a dataset public?

You host the binaries on a service of your choice and have a website with the links, description, license/terms of use. Popular examples are - WIDERFace, MS COCO and many others.

snosov1 avatar Nov 06 '20 05:11 snosov1

Alright, I will let you know as soon as I collect the required data.

edward9112 avatar Nov 09 '20 11:11 edward9112

Thx! Looking forward to it!

snosov1 avatar Nov 09 '20 11:11 snosov1

Thx! Looking forward to it!

Hi @snosov1

I have collected the dataset, it's around 25GB of video footage with motion in it. Many scenes and camera positions. The average resolution is 640x360px.

Would that be sufficient?

edward9112 avatar Jan 26 '21 11:01 edward9112

Hey, @edward9112 !

Great news! The spec sounds great - we'll need to execute some experiments to see what we can achieve with it, though!

Please, refer any questions you might have with regards to sharing the data to @yssemaev . Once this is figured out and we can access the data - we'll plan the experiments accordingly.

snosov1 avatar Jan 26 '21 11:01 snosov1

Awesome!

@yssemaev can I send you the data sharing agreement draft? If so, can you provide your contact/email?

edward9112 avatar Jan 26 '21 12:01 edward9112

@edward9112, please contact me directly by email: yuri.semaev at intel.com to discuss details

yssemaev avatar Jan 26 '21 12:01 yssemaev

Hi, @yssemaev @snosov1 it looks like the agreement process is stalled. I never got any replies from your legal team.

edward9112 avatar Feb 15 '21 11:02 edward9112

Pinged legal team, waiting for response.

yssemaev avatar Feb 15 '21 11:02 yssemaev

@snosov1 @yssemaev it looks like the legal team's response is going to take forever. Is there another way to pass the materials to you without putting them in a public data bank permanently?

edward9112 avatar Feb 24 '21 15:02 edward9112

@snosov1 @yssemaev the legal team seem to have approved the agreement but then stopped responding. Is there any way to expedite?

edward9112 avatar Mar 19 '21 11:03 edward9112

@snosov1 @yssemaev Hi again, just a follow-up regarding the agreement. We spent a lot of time and resources collecting the data, so it would be unfortunate to just leave it behind.

edward9112 avatar Apr 16 '21 10:04 edward9112

@vladimir-dudnik @eaidova Any suggestions?

edward9112 avatar Apr 27 '21 12:04 edward9112

@edward9112 can you publish your data under permissive license on some public resource, like @snosov1 suggested in his comment?

vladimir-dudnik avatar Apr 28 '21 21:04 vladimir-dudnik

@snosov1 @vladimir-dudnik is it OK if we publish it on Kaggle.com ?

edward9112 avatar May 05 '21 17:05 edward9112

@edward9112 I think it should be OK if dataset is available on public resource under permissive license

vladimir-dudnik avatar May 06 '21 23:05 vladimir-dudnik

@snosov1 @vladimir-dudnik it looks like the agreement has been signed today. I will proceed with uploading the data for sharing.

edward9112 avatar May 07 '21 09:05 edward9112

@vladimir-dudnik @snosov1 @yssemaev data has been shared with you via Google Drive

edward9112 avatar May 17 '21 10:05 edward9112

@vladimir-dudnik @snosov1 @yssemaev any comments/feedback regarding data quality?

edward9112 avatar May 21 '21 11:05 edward9112

@vladimir-dudnik @snosov1 @yssemaev any feedback would be helpful. Should we add more data?

edward9112 avatar Jun 21 '21 12:06 edward9112