Biden appoints AI Safety Institute leaders as NIST funding concerns linger

Today the Biden administration named Elizabeth Kelly, a top White House aide who had been a “driving force” behind the President’s AI Executive Order (EO), as the director of the new US AI Safety Institute (USAISI) at the National Institute of Standards and Technology (NIST). In addition, Elham Tabassi was appointed as chief technology officer.

NIST, which is part of the US Department of Commerce, released an highly-anticipated AI risk management framework in January 2023 and was given a great deal of responsibility in the White House’s AI EO to “undertake an initiative for evaluating and auditing capabilities relating to Artificial Intelligence (AI) technologies and to develop a variety of guidelines, including for conducting AI red-teaming tests to enable deployment of safe, secure, and trustworthy systems.”

However, the leadership appointments appeared to be the first major announcement about the USAISI since it was established in November 2023 to “support the responsibilities assigned to the Department of Commerce” under the AI Executive Order. Since then, there have been few details disclosed about how the institute would work and where its funding would come from — especially since NIST itself, with reportedly a staff of about 3,400 and an annual budget of just over $1.6 billion — is known to be underfunded.

Source of NIST funding for AI safety is unclear

A bipartisan group of senators asked the Senate Appropriations Committee in January for $10 million of funding to help establish the U.S. Artificial Intelligence Safety Institute (USAISI) within NIST as part of the fiscal 2024 funding legislation. But it is not clear where that funding request stands.

In addition, in mid-December House Science Committee lawmakers from both parties sent a letter to NIST that Politico reported “chastised the agency for a lack of transparency and for failing to announce a competitive process for planned research grants related to the new U.S. AI Safety Institute.”

The lawmakers said they were particularly concerned over a planned AI research partnership between NIST and the RAND Corporation — an influential think tank tied to tech billionaires, the AI industry and the controversial “effective altruism” movement (VentureBeat has also reported about a “widening web” of effective altruism (EA) in AI ‘safety’ and security, including within RAND and leading LLM model company Anthropic).

In the letter, the lawmakers wrote: “Unfortunately, the current state of the AI safety research field creates challenges for NIST as it navigates its leadership role on the issue,” adding that “findings within the community are often self-referential and lack the quality that comes from revision in response to critiques by subject matter experts. There is also significant disagreement within the AI safety field of scope, taxonomies, and definitions.”

NIST is a nonpartisan body and ‘the adults in the room,’ says expert

In an interview with VentureBeat about the USAISI leadership appointments, Rumman Chowdhury, who formerly led AI efforts at Accenture and also served as head of Twitter (now X)’s META team (Machine Learning Ethics, Transparency and Accountability) from 2021-2011, said that while people have been “chomping at the bit” for announcements about NIST and the USAISI, the pace is not unusual with all the vetting required.

In addition, Chowdhury, who currently runs the nonprofit Humane Intelligence and helped run the first public generative AI red team event at DEF CON last summer, said she “loves” that the USAISI is part of NIST, which she described as “sort of the adults in the room” when it comes to developing standards for AI safety and security.

“They’ve been doing this kind of work for a very long time,” she said, adding that NIST is a nonpartisan body. “They’re not appointed, so this is not at the whims of whoever is in power — these are employees of the federal government and that’s one distinction that people who are new to DC, especially from the tech world, don’t realize. Because appointments can be a little bit more political.”

The AI Safety Institute was an ‘unfunded mandate’

Still, Chowdhury did point out that funding is an issue for the USAISI. “One of the frankly under-discussed things is this is an unfunded mandate via the executive order,” she said. “I understand the politics of why, given the current US polarization, it’s really hard to get any sort of bill through…I understand why it came through an executive order. The problem is there’s no funding for it — while the UK is sitting on $100 million, to the tune of $10 million a year.”

When asked about the Senate request for funding, she said that she knew about it but as far as she knew the agency had not received anything.

Chowdhury also pointed to the AI Safety Institute Consortium, which was announced at the same time as the USAISI, which invited organizations to contribute technical expertise in areas such as AI governance, AI safety, red-teaming, responsible AI and data documentation.

According to the consortium agreement, which is open to anyone to join, membership fees to join the consortium are $1000 per year. Chowdhury said she did not know anything about NIST/USAISI plans regarding research grants.

Anthropic called for more NIST funding in April 2023

The calls for NIST funding have been ongoing — in April 2023, VentureBeat reported on Anthropic’s call for $15 million in funding for NIST to support the agency’s AI measurement and standards efforts.

The company released a call-to-action memo, two days after a budget hearing about 2024 funding of the U.S. Department of Commerce in which there was bipartisan support for maintaining American leadership in the development of critical technologies.

Anthropic’s memo said the call for NIST funding is a simpler, “shovel-ready” idea available to policymakers.

AI Safety Institute within NIST called a ‘positive signal’

Chowdhury said she remains optimistic about the prospects for the USAISI and its role within NIST.

“I don’t know Elizabeth personally, but I know Elham and she’s one of the most level-headed people I know, which is great because I think with a role like CTO you need a scientist…somebody who just wants to get the work done,” she said.

In addition, she added that “putting my political science hat back on, where an organization sits in the larger institution matters a lot, especially politically.” With the USAISI placed within NIST, “my opinion is it is a pretty positive signal that they want to be science-forward versus politics forward.”