long-context topic

Official release of InternLM2.5 base and chat models. 1M context support

InternLM

Code and documents of LongLoRA and LongAlpaca (ICLR 2024 Oral)

dvlab-research

[EMNLP 2024] LongAlign: A Recipe for Long Context Alignment of LLMs

THUDM

The code of our paper "InfLLM: Unveiling the Intrinsic Capacity of LLMs for Understanding Extremely Long Sequences with Training-Free Memory"

thunlp

[DEPRECIATED] Very fast, large music transformer with 8k sequence length, efficient heptabit MIDI notes encoding, true full MIDI instruments range, chords counters and outro tokens

asigalov61

[COLM 2024] TriForce: Lossless Acceleration of Long Sequence Generation with Hierarchical Speculative Decoding

Infini-AI-Lab

Implementation of 💍 Ring Attention, from Liu et al. at Berkeley AI, in Pytorch

lucidrains

PyTorch implementation of Infini-Transformer from "Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention" (https://arxiv.org/abs/2404.07143)

dingo-actual

ACL 2024 | LooGLE: Long Context Evaluation for Long-Context Language Models

bigai-nlco

open-source code for paper: Retrieval Head Mechanistically Explains Long-Context Factuality

nightdessert