New Guidance for Federal AI Procurement Embraces Red Teaming and Other HackerOne Suggestions – Source:www.hackerone.com

The U.S. government’s approach to evaluating and adopting new technology for its own use often impacts private sector adoption. That’s why it’s significant that, while AI is already having a transformative effect on productivity across industries, the U.S. government is also seeking to harness the benefits of this emerging technology for federal agencies and has now developed criteria to guide its decision making while evaluating AI. As the U.S. government works to apply AI to critical U.S. government operations, it is vital that AI’s power is harnessed safely and responsibly—not only to ensure that the government’s own deployment of AI is secure and effective, but also because of the government’s ripple effect on standards and adoption of AI across all sectors.

Earlier this year, the Office of Management and Budget (OMB), which establishes budget rules for federal agencies, issued a memorandum on Advancing the Responsible Acquisition of Artificial Intelligence in Government which outlines for both agencies and the public significant aspects of responsible AI procurement and deployment. In particular, OMB’s memo embraced AI red teaming as a critical element of the acquisition of AI for U.S. government agencies.

Rules for U.S. Federal Agency AI Procurement

Last October, the Biden-Harris Administration published an Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence (AI EO). That expansive action set the tone for the US government’s approach to utilizing AI in a safe and secure manner and required OMB to provide guidance to US government agencies on how to manage risks when acquiring AI products and services.  

Consistent with HackerOne’s long-standing policy advocacy in favor of responsible AI deployment, we provided OMB with comments on how the security and safety best practices championed by HackerOne aligned with the AI EO and should be leveraged in OMB’s development of that guidance. Specifically, HackerOne cited the benefits of conducting AI red teaming, ensuring the transparency of AI red teaming methodology, and of documenting the specific harms and bias federal agencies are seeking to avoid. These suggestions drew on our extensive experience working with government agencies and companies to enhance cybersecurity and our use of similar best practices in testing AI models.

We were pleased to see that the memo reflects our core recommendations:

• Embracing AI Red Teaming: OMB has made it a requirement that agencies procuring general use enterprise-wide generative AI include contractual requirements ensuring that vendors provide documentation of AI red teaming results.

• Identifying Specific Harms: In addition to the categories of risk that vendors include, OMB has encouraged agencies to require documentation to cover AI red teaming related to nine specific categories of risk.  

The inclusion of these elements within the memo will help protect the security and effectiveness of the U.S. federal government by requiring that the AI products and services that undergird critical operations be proactively tested to identify potential risks and harms. It also further underscores the role of AI red teaming as a best practice that all companies should adopt to help ensure the safety and security of their AI products and services and to build the trust of their customers.

Learn more about AI red teaming with HackerOne.