Awesome_Mamba
Awesome_Mamba copied to clipboard
Published
20 hours ago •
xmindflow
xmindflow
A Comprehensive Survey of Mamba in Deep Learning
Awesome Mamba
:fire::fire: This is a curated collection of groundbreaking research papers and articles on Mamba Architecture in various domains like visual recognition, speech processing, and more. :fire::fire:
Welcome to the Mamba Architecture Repository! 🌟
🔍 Here, we explore the intricate world of Mamba architecture through a comprehensive collection of research papers, each shedding light on different facets of this versatile framework.
What is Mamba Architecture? 🤔
Mamba architecture is a pioneering framework in deep learning, known for its selective state spaces and efficiency in processing complex data structures. It stands at the forefront of innovation, pushing the boundaries of what's possible in sequence modelling, image processing, speech recognition, and much more.
Why This Repository? 🌈
- Curated Content: Each paper has been carefully selected to ensure you have access to high-quality research.
- Organized Access: We've categorized the papers into sections so you can easily navigate through the vast knowledge base.
- Community Driven: Contributions are welcome! If you have a paper or resource related to Mamba architecture, feel free to add it.
Contribute 🤝
We believe in the power of collaboration to drive research forward. If you'd like to contribute to this repository, please feel free to submit a pull request or open an issue. Whether it's adding new papers, improving documentation, or suggesting new categories, your input is invaluable.
Let's Dive In! 🚀
Ready to explore? Check out the Table of Contents to start your journey through the fascinating world of Mamba architecture. Happy exploring!
Thank you for visiting, and don't forget to stars ⭐ the repository if you find it useful!
Table of Contents
- Preliminaries and State Space Models
- Architecture Redesign
- Remote Sensing
- Speech Processing
- Video Processing
- Activity Recognition
- Image Enhancement
- Medical Imaging
- Image Segmentation
- Reinforcement Learning
- Natural Language Processing
- 3D Recognition
- Multi-Modal Understanding
- Time Series
- GNN
- Point Cloud
- Tabular Data
Preliminaries and State Space Models
- HiPPO: Recurrent Memory with Optimal Polynomial Projections [Github]
- S4: Efficiently Modeling Long Sequences with Structured State Spaces [Github]
- H3: Hungry Hungry Hippos: Toward Language Modeling with State Space Models [Github]
- LOCOST: State-Space Models for Long Document Abstractive Summarization [Github]
- Theoretical Foundations of Deep Selective State-Space Models
- S4++: Elevating Long Sequence Modeling with State Memory Reply
- Hieros: Hierarchical Imagination on Structured State Space Sequence World Models [Github]
- State Space Models as Foundation Models: A Control Theoretic Overview
- Selective Structured State-Spaces for Long-Form Video Understanding
- Retentive Network: A Successor to Transformer for Large Language Models[Github]
- Convolutional State Space Models for Long-Range Spatiotemporal Modeling[Github]
- Laughing Hyena Distillery: Extracting Compact Recurrences From Convolutions[Github]
- Resurrecting Recurrent Neural Networks for Long Sequences
- Hyena Hierarchy: Towards Larger Convolutional Language Models[Github]
Architecture Redesign
- Mamba: Linear-time sequence modeling with selective state spaces [Github]
- Locating and Editing Factual Associations in Mamba [Github]
- MambaMixer: Efficient Selective State Space Models with Dual Token and Channel Selection [Github]
- Jamba: A Hybrid Transformer-Mamba Language Model
- Mamba-ND: Selective State Space Modeling for Multi-Dimensional Data[Github]
- Incorporating Exponential Smoothing into MLP: A Simple but Effective Sequence Model [Github]
- PlainMamba: Improving Non-Hierarchical Mamba in Visual Recognition [Github]
- Understanding Robustness of Visual State Space Models for Image Classification
- Efficientvmamba: Atrous selective scan for light weight visual mamba [Github]
- Localmamba: Visual state space model with windowed selective scan [Github]
- Mamba4Rec: Towards Efficient Sequential Recommendation with Selective State Space Models [Github]
- The hidden attention of mamba models [Github]
- Learning method for S4 with Diagonal State Space Layers using Balanced Truncation
- BlackMamba: Mixture of Experts for State-Space Models[Github]
- MoE-Mamba: Efficient Selective State Space Models with Mixture of Experts[Github]
- Scalable Diffusion Models with State Space Backbone [Github]
- ZigMa: Zigzag Mamba Diffusion Model[Github]
- Spectral State Space Models
- Mamba-unet: Unet-like pure visual mamba for medical image segmentation [Github]
- Mambabyte: Token-free selective state space model
- Vmamba: Visual state space model [Github]
- Vision mamba: Efficient visual representation learning with bidirectional state space model [Github]
Remote Sensing
- ChangeMamba: Remote Sensing Change Detection with Spatio-Temporal State Space Model [Github]
- RS-Mamba for Large Remote Sensing Image Dense Prediction [Github]
- RS3Mamba: Visual State Space Model for Remote Sensing Images Semantic Segmentation [Github]
- HSIMamba: Hyperspectral Imaging Efficient Feature Learning with Bidirectional State Space for Classification [Github]
- Rsmamba: Remote sensing image classification with state space model [Github]
- HSIMamba: Hyperpsectral Imaging Efficient Feature Learning with Bidirectional State Space for Classification [Github]
- Samba: Semantic Segmentation of Remotely Sensed Images with State Space Model [Github]
Speech Processing
- SPMamba: State-space model is all you need in speech separation [Github]
- Dual-path Mamba: Short and Long-term Bidirectional Selective Structured State Space Models for Speech Separation [Github]
Video Processing
- SpikeMba: Multi-Modal Spiking Saliency Mamba for Temporal Video Grounding
- Video mamba suite: State space model as a versatile alternative for video understanding [Github]
- SSM Meets Video Diffusion Models: Efficient Video Generation with Structured State Spaces [Github]
- Videomamba: State space model for efficient video understanding [Github]
Activity Recognition
- HARMamba: Efficient Wearable Sensor Human Activity Recognition Based on Bidirectional Selective SSM
- VMRNN: Integrating Vision Mamba and LSTM for Efficient and Accurate Spatiotemporal Forecasting[Github]
Image Enhancement
- Aggregating Local and Global Features via Selective State Spaces Model for Efficient Image Deblurring
- Serpent: Scalable and Efficient Image Restoration via Multi-scale Structured State Space Models
- VmambaIR: Visual State Space Model for Image Restoration [Github]
- Activating Wider Areas in Image Super-Resolution [Github]
- MambaIR: A Simple Baseline for Image Restoration with State-Space Model [Github]
- Pan-Mamba: Effective pan-sharpening with State Space Model [Github]
- U-shaped Vision Mamba for Single Image Dehazing [Github]
Medical Imaging
- Vim4Path: Self-Supervised Vision Mamba for Histopathology Images [Github]
- VMambaMorph: a Visual Mamba-based Framework with Cross-Scan Module for Deformable 3D Image Registration [Github]
- UltraLight VM-UNet: Parallel Vision Mamba Significantly Reduces Parameters for Skin Lesion Segmentation [Github]
- Rotate to Scan: UNet-like Mamba with Triplet SSM Module for Medical Image Segmentation
- Integrating Mamba Sequence Model and Hierarchical Upsampling Network for Accurate Semantic Segmentation of Multiple Sclerosis Legion
- CMViM: Contrastive Masked Vim Autoencoder for 3D Multi-modal Representation Learning for AD classification
- H-vmunet: High-order vision mamba unet for medical image segmentation [Github]
- ProMamba: Prompt-Mamba for polyp segmentation
- Vm-unet-v2 rethinking vision mamba unet for medical image segmentation [Github]
- MD-Dose: A Diffusion Model based on the Mamba for Radiotherapy Dose Prediction [Github]
- Large Window-based Mamba UNet for Medical Image Segmentation: Beyond Convolution and Self-attention [Github]
- MambaMIL: Enhancing Long Sequence Modeling with Sequence Reordering in Computational Pathology [Github]
- Clinicalmamba: A generative clinical language model on longitudinal clinical notes [Github]
- Lightm-unet: Mamba assists in lightweight unet for medical image segmentation [Github]
- MedMamba: Vision Mamba for Medical Image Classification [Github]
- Weak-Mamba-UNet: Visual Mamba Makes CNN and ViT Work Better for Scribble-based Medical Image Segmentation [Github]
- P-Mamba: Marrying Perona Malik Diffusion with Mamba for Efficient Pediatric Echocardiographic Left Ventricular Segmentation
- Semi-Mamba-UNet: Pixel-Level Contrastive Cross-Supervised Visual Mamba-based UNet for Semi-Supervised Medical Image Segmentation [Github]
- FD-Vision Mamba for Endoscopic Exposure Correction [Github]
- Swin-umamba: Mamba-based unet with imagenet-based pretraining [Github]
- Vm-unet: Vision mamba unet for medical image segmentation[Github]
- Vivim: a video vision mamba for medical video object segmentation [Github]
- Segmamba: Long-range sequential modeling mamba for 3d medical image segmentation [Github]
- T-Mamba: Frequency-Enhanced Gated Long-Range Dependency for Tooth 3D CBCT Segmentation [Github]
- U-mamba: Enhancing long-range dependency for biomedical image segmentation [Github]
- MambaMorph: a Mamba-based Backbone with Contrastive Feature Learning for Deformable MR-CT Registration[Github]
- nnMamba: 3D Biomedical Image Segmentation, Classification and Landmark Detection with State Space Model[Github]
- MambaMIR: An Arbitrary-Masked Mamba for Joint Medical Image Reconstruction and Uncertainty Estimation[Github]
- ViM-UNet: Vision Mamba for Biomedical Segmentation[Github]
Image Segmentation
Reinforcement Learning
- Decision Mamba: Reinforcement Learning via Sequence Modeling with Selective State Spaces [Github]
- MAMBA: an Effective World Model Approach for Meta-Reinforcement Learning [Github]
Natural Language Processing
- RankMamba Benchmarking Mamba's Document Ranking Performance in the Era of Transformers [Github]
- Densemamba: State space models with dense hidden connection for efficient large language models [Github]
- Is Mamba Capable of In-Context Learning?
3D Recognition
- Gamba: Marry Gaussian Splatting with Mamba for single view 3D reconstruction
- Motion mamba: Efficient and long sequence motion generation with hierarchical and bidirectional selective ssm [Github]
Multi-Modal Understanding
- ReMamber: Referring Image Segmentation with Mamba Twister
- VL-Mamba: Exploring State Space Models for Multimodal Learning
- Cobra: Extending Mamba to Multi-Modal Large Language Model for Efficient Inference[Github]
- SurvMamba: State Space Model with Multi-grained Multi-modal Interaction for Survival Prediction
- MambaDFuse: A Mamba-based Dual-phase Model forMulti-modality Image Fusion
- APRICOT-Mamba: Acuity Prediction in Intensive Care Unit (ICU): Development and Validation of a Stability, Transitions, and Life-Sustaining Therapies Prediction Model
Time Series
- SiMBA: Simplified Mamba-Based Architecture for Vision and Multivariate Time series [Github]
- Is Mamba Effective for Time Series Forecasting? [Github]
- TimeMachine: A Time Series is Worth 4 Mambas for Long-term Forecasting [Github]
- MambaStock: Selective state space model for stock prediction [Github]
GNN
- STG-Mamba: Spatial-Temporal Graph Learning via Selective State Space Model
- Graph Mamba: Towards Learning on Graphs with State Space Models [Github]
- Graph-Mamba: Towards long-range graph sequence modeling with selective state spaces [Github]
- Recurrent Distance Filtering for Graph Representation Learning[Github]
- Modeling multivariate biosignals with graph neural networks and structured state space models[Github]
Point Cloud
- Point mamba: A novel point cloud backbone based on state space model with octree-based ordering strategy [Github]
- Point Could Mamba: Point Cloud Learning via State Space Model [Github]
- PointMamba: A Simple State Space Model for Point Cloud Analysis [Github]
- 3DMambaIPF: A State Space Model for Iterative Point Cloud Filtering via Differentiable Rendering
- 3DMambaComplete: Exploring Structured State Space Model for Point Cloud Completion
xmindflow