[RFC] New Ops in TorchVision

[datumbox]

🚀 The feature

Consider adding the following operators in TorchVision:

Layers

There is a separate ticket for tracking common layers: #4333

• [x] DropBlock Layer #5416
• [ ] AutoDropout Layer
• [ ] DropConnect Layer
• [ ] ShakeDrop Layer

Operators

• [ ] SoftNMS
• [ ] masks_to_boxes for N-dimensional masks

Losses

There is a separate ticket tracking Losses proposals: #2980

Schedulers & Optimizers (Core upstreaming)

• [ ] Polynomial LR scheduler
• [ ] Random Noise LR Scheduler
• [ ] LARS
• [ ] LAMB

[xiaohu2015]

maybe I can implement some features, eg DropBlock layer.

[datumbox]

@xiaohu2015 wow, I literally just slacked you to see if you are interested 😄

Great! If you want to send a PR for DropBlock it would be awesome. Let me know if you want me to create an issue for it so that other contributors know you are working on it already (else you can create one yourself or just rely on the PR; up to you!).

[lezwon]

hey @datumbox, can I take up SoftNMS implementation?

[datumbox]

@lezwon Thanks for offering help!

The SoftNMS would have to be implemented in C++ and CUDA because this is where we implement the standard NMS. Some additional discussion would be required to see exactly how this will be implemented and what its API would look like. As you understand this is quite a lot of work and it's not guaranteed that the feature will be merged. If you are up for it, we can discuss more. Just wanted to give you a heads up that this is a more risky feature to work on.

If the above doesn't sound too appealing, there are features listed at #5410 you might find fun to work on. Have a look and let me know if anything interests you. :)

[lezwon]

@datumbox Sure thing :) I'll pick up something from #5410

[oke-aditya]

I want to try DropConnect Layer. Any other info / implementation I could look to will be great :smiley:

[datumbox]

@oke-aditya There are a few reasons we haven't added DropConnect. According to the paper, here is Dropout: r = m * a(W v) and here is DropConnect: r = a ((M * W) u).

a: activation u: input M: bernoulli mask W: weights

As you see the M on the latter case is applied on the W, which means it makes for an awkward design of a layer. I believe you will have to implement different versions of it for Linear and Convs. Another issue with it is that it's quite old and not often used in SOTA research. These are some of the reasons we decided not to add it, at least on phase 1 and 2 of Batteries Included.