mixture-of-experts topic
MoE-LLaVA
Mixture-of-Experts for Large Vision-Language Models
PETL_AST
This is the official repository of the papers "Parameter-Efficient Transfer Learning of Audio Spectrogram Transformers" and "Efficient Fine-tuning of Audio Spectrogram Transformers via Soft Mixture of...
MC-SMoE
[ICLR 2024 Spotlight] Code for the paper "Merge, Then Compress: Demystify Efficient SMoE with Hints from Its Routing Policy"
leeroo_orchestrator
The implementation of "Leeroo Orchestrator: Elevating LLMs Performance Through Model Integration"
inferflow
Inferflow is an efficient and highly configurable inference engine for large language models (LLMs).
Chinese-Mixtral
中文Mixtral混合专家大模型（Chinese Mixtral MoE LLMs）
RealCompo
[NeurIPS 2024] RealCompo: Balancing Realism and Compositionality Improves Text-to-Image Diffusion Models
classifier
Machine learning code, derivatives calculation and optimization algorithms developed during the Machine Learning course at Universidade de Sao Paulo. All codes in Python, NumPy and Matplotlib with exa...
makeMoE
From scratch implementation of a sparse mixture of experts language model inspired by Andrej Karpathy's makemore :)
mergoo
A library for easily merging multiple LLM experts, and efficiently train the merged LLM.