The Allen Institute for AI (AI2) on Thursday released a highly anticipated open-source large language model, along with the underlying data and code meant to significantly improve understanding of how generative AI actually works.
Open Language Model, or OLMo, is “a truly open-source, state-of-the-art large language model released alongside the pre-training data and training code, something no open models of this scale offer today,” AI2 said in its announcement. The goal is to “advance the science of language models collectively.”
It’s a major milestone for the Seattle-based non-profit institute, started by the late Microsoft co-founder Paul Allen in 2014. The effort contrasts with closed LLMs from OpenAI and others that don’t publicly offer training data and code.
OLMo is available on HuggingFace and GitHub.
AI2 says the ability to see the data and code will improve researcher efficiency and no longer require them to make assumptions about what’s happening inside the black box of the AI model, enabling scientific assessments instead. Eliminating redundancies improves energy efficiency in AI research, as well, AI2 says.
In a statement, Hanna Hajishirzi, OLMo project lead, described studying LLMs without access to the training data as “the equivalent of drug discovery without clinical trials or studying the solar system without a telescope.”
“With our new framework, researchers will finally be able to study the science of LLMs, which is critical to building the next generation of safe and trustworthy AI,” Hajishirzi said as part of the AI2 announcement.
AI2 said the project was made possible in part by its collaboration with the Kempner Institute for the Study of Natural and Artificial Intelligence at Harvard University, plus partners including AMD, CSC (Lumi Supercomputer), the Paul G. Allen School of Computer Science & Engineering at the University of Washington and Databricks.
This first version of the model is called OLMo 7B, a reference to the 7 billion parameters used in training the model. The more parameters, the greater the complexity and nuance of the responses. Previously, AI2 said OLMo would have 70 billion parameters; that is now expected to be the scale of the next version.
AI2 has been led since last summer by CEO Ali Farhadi, a University of Washington computer science professor and former Apple machine learning leader, who succeeded Oren Etzioni in the role.
