Mamba is a sequence model architecture based on selective state space models, introduced by Albert Gu and Tri Dao in 2023. Instead of attention, it processes sequences with input-dependent state space operations, so compute grows linearly with sequence length and only a fixed-size state is kept during inference.
It is studied as an alternative to the transformer's quadratic cost, and open models of several sizes have been released. A successor architecture, Mamba-2, and hybrid models that interleave Mamba layers with attention layers have also been published.