Awesome High-Order Neural Networks

This repository focuses on high-order neural network architecture in computer vision.

Transfomers

[ICLR2021] An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale

[ICML2021] Training data-efficient image transformers & distillation through attention

[ICCV2021] Swin Transformer: Hierarchical Vision Transformer Using Shifted Windows

ConvNets

[ICCV2017] SORT: Second-Order Response Transform for Visual Recognition

[NeurIPS2022] HorNet: Efficient High-Order Spatial Interactions with Recursive Gated Convolutions

[arxiv2022] Conv2Former: A Simple Transformer-Style ConvNet for Visual Recognition

[CVPR2019] Kervolutional Neural Networks

Bilinear Pooling

[ICCV2017] Factorized Bilinear Models for Image Recognition

[CVPR2019] Global Second-order Pooling Convolutional Networks

[TNNLS2021] Detachable Second-Order Pooling: Toward High-Performance First-Order Networks

Polynomial Neural Networks

Related Scholar: Grigorios G Chrysos

[TPAMI2021] Deep Polynomial Neural Networks

[CVPR2020] P-nets: Deep Polynomial Neural Networks

[ECCV2022] Augmenting Deep Classifiers with Polynomial Neural Networks

[ICLR2022] The Spectral Bias of Polynomial Neural Networks

[NeurIPS2022] Extrapolation and Spectral Bias of Neural Nets with Hadamard Product: a Polynomial Net Study

Library

[MLSys2022] QuadraLib: A Performant Quadratic Neural Network Library for Architecture Optimization and Design Exploration