AI21 Labs launches Jamba, an AI model that integrates two architectures: transformer and Mamba, which is based on the Structured State Space model (Kyle Wiggers/TechCrunch)

Kyle Wiggers / TechCrunch:

AI21 Labs launches Jamba, an AI model that integrates two architectures: transformer and Mamba, which is based on the Structured State Space model  —  Increasingly, the AI industry is moving toward generative AI models with longer contexts.  But models with large context windows tend to be compute-intensive.