Metalhead.jl
Metalhead.jl copied to clipboard
FluxML
Computer vision models for Flux
This is an implementation of [EfficientNetv2](https://arxiv.org/abs/2104.00298). There's clearly some code duplication between the EffNets and the MobileNets, but figuring out how to unify that API and its design is perhaps...
Xref #187 > **Note** > This PR does not improve the existing textual documentation in any way; instead, it migrates the current documentation to `Documenter.jl`. See the new documentation in...
These changes will make it easier to port over pretrained weights for the models from PyTorch. - [ ] Right now, GoogLeNet matches the implementation in the paper, which does...
Convolution and BatchNorm layers have been fused during inference in many models, most notably [LeViT](https://github.com/facebookresearch/LeViT). It would be a good idea to have this as an option in the `conv_norm`...
`DropBlock` is a type of regularisation that tries to replace dropout. The [original paper](https://arxiv.org/abs/1810.12890) describes it as best used with a linear scaling rate across blocks in a model, as...
The current documentation format is a little weird and Publish exposes a lot of private functions in the API reference as well. There are several steps that need to be...
This modifies the current ViT API to add more options - notably, there is now a optional `prenorm/postnorm` toggle. There's also an option to make class tokens disappear completely, and...
Given that all the high-level model APIs implement `(m::Model)(x)`, `backbone(m::Model)` and `classifier(m::Model)`, would it make sense to have an abstract type `MetalheadModel` that the models can subtype? I take it...
With some experimentation, issues pop up when I use ViT on the GPU. Documenting these so that they can be tracked down and solved: - [x] Class tokens don't work...
I'm looking at Metalhead integration in MLJFlux. To do this well, I'm looking for some uniformity in the Metalhead.jl API that seems to be lacking. In particular, it would help...
