The National Institute of Standards and Technology (NIST), a U.S. Commerce Department agency known for developing and testing technology for the government, companies, and the public, has updated its testbed, Dioptra. This tool measures how malicious attacks, especially those that "poison" AI training data, can degrade AI system performance.
Dioptra, named after a classical astronomical and surveying instrument, is an open-source, modular web-based tool initially released in 2022. It aims to help companies and AI users assess, analyze, and track AI risks. According to NIST, Dioptra can benchmark and research AI models, providing a common platform for exposing models to simulated threats in a "red-teaming" environment.
"Testing the effects of adversarial attacks on machine learning models is one of Dioptra's goals," NIST stated in a press release. "This open-source software, available for free download, could assist government agencies and small to medium-sized businesses in evaluating AI developers' performance claims."
Dioptra's release is accompanied by documents from NIST and the newly established AI Safety Institute, detailing ways to mitigate AI dangers, such as the creation of nonconsensual pornography. This update follows the U.K. AI Safety Institute’s launch of Inspect, a similar toolset for assessing model capabilities and safety. Both the U.S. and U.K. are collaborating on advanced AI model testing, a partnership announced at the U.K.’s AI Safety Summit at Bletchley Park last November.
The development of Dioptra also stems from President Joe Biden’s executive order on AI, which mandates NIST's involvement in AI system testing. The order also sets standards for AI safety and security, requiring companies like Apple to notify the federal government and share safety test results before public deployment.
AI benchmarks remain challenging due to the opaque nature of sophisticated AI models, whose infrastructure and training data are often kept secret by their creators. A recent report by the Ada Lovelace Institute highlights that current evaluation policies allow AI vendors to selectively choose which tests to conduct, complicating the assessment of real-world safety.
While NIST acknowledges that Dioptra cannot completely eliminate risks, it suggests the tool can identify which attacks degrade AI performance and quantify this impact.
However, Dioptra currently works only on models that can be downloaded and used locally, such as Meta’s expanding Llama family. Models accessible only through an API, like OpenAI’s GPT-4, are not yet supported.
