Join us in Atlanta on April 10th and explore the landscape of security workforce. We will explore the vision, benefits, and use cases of AI for security teams. Request an invite here.
AI chip-maker SambaNova Systems has announced a significant achievement with its Samba-CoE v0.2 Large Language Model (LLM).
This model, operating at an impressive 330 tokens per second, outperforms several notable models from competitors such as the brand new DBRX from Databricks released just yesterday, MistralAI’s Mixtral-8x7B, and Grok-1 by Elon Musk’s xAI, among others.
What makes this achievement particularly notable is the efficiency of the model—it achieves these speeds without compromising on precision, and it requires only 8 sockets to operate.
Indeed, in our tests of the LLM, it produced responses to our inputs blindingly quickly, clocking in at 330.42 seconds for a 425-word answer about the Milky Way galaxy.
VB Event
The AI Impact Tour – Atlanta
Continuing our tour, we’re headed to Atlanta for the AI Impact Tour stop on April 10th. This exclusive, invite-only event, in partnership with Microsoft, will feature discussions on how generative AI is transforming the security workforce. Space is limited, so request an invite today.
A question about quantum computing yielded a similarly robust and fast response at a whopping 332.56 tokens delivered in one second.
This is presented as a more efficient alternative to configurations that might require 576 sockets and operate at lower bit rates.
Efficiency advancements
SambaNova’s emphasis on using a smaller number of sockets while maintaining high bit rates suggests a significant advancement in computing efficiency and model performance.
It is also teasing the upcoming release of Samba-CoE v0.3 in partnership with LeptonAI, indicating ongoing progress and innovation.
Additionally, SambaNova Systems highlights that the foundation of these advancements is built on open-source models from Samba-1 and the Sambaverse, employing a unique approach to ensembling and model merging.
This methodology not only underpins the current version but also suggests a scalable and innovative approach to future developments.
The comparison with other models like GoogleAI’s Gemma-7B, MistralAI’s Mixtral-8x7B, Meta’s llama2-70B, Alibaba Group’s Qwen-72B, TIIuae’s Falcon-180B, and BigScience’s BLOOM-176B, showcases Samba-CoE v0.2’s competitive edge in the field.
This announcement is likely to stir interest in the AI and machine learning communities, prompting discussions around efficiency, performance, and the future of AI model development.
Background on SambaNova
SambaNova Systems was found in Palo Alto, California in 2017 by three co-founders: Kunle Olukotun, Rodrigo Liang, and Christopher Ré.
Initially focusing on the creation of custom AI hardware chips, SambaNova’s ambition quickly expanded, encompassing a broader suite of offerings including machine learning services and a comprehensive enterprise AI training, development and deployment platform known as the SambaNova Suite in early 2023, and earlier this year, a 1-trillion-parameter AI model, Samba-1, made from 50 smaller models in a “Composition of Experts.”
This evolution from a hardware-centric start-up to a full-service AI innovator reflected the founders’ commitment to enabling scalable, accessible AI technologies.
As SambaNova carved its niche within the competitive AI industry, it positioned itself as a formidable contender to established giants like Nvidia, raising a $676 million Series D at a valuation of over $5 billion in 2021.
Today, the company competes with other dedicated AI chip startups such as Groq in addition to stalwarts like Nvidia.
