OpenAI Tightens AI Safeguards Following Hugging Face Incident

Following the recent Hugging Face incident, in which an OpenAI model went rogue and targeted the open-source platform while attempting to complete a task, the AI firm said it is adding greater urgency to its efforts to strengthen AI safeguards.

In an update published on August 18, the AI firm said that as models become more capable, the risks associated with developing and testing them internally also grow.

Following Hugging Face, the company paused certain frontier AI workloads that could execute code or access the internet, and has since introduced stricter controls including workload sandboxing, network isolation and continuous security testing.

The pace of testing has been temporarily slowed and its largest planned frontier reinforcement learning (RL) run remains on hold.

The company is looking to evolve its Preparedness Framework, first published in December 2023. The framework is the process of tracking and preparing for advanced AI capabilities that could introduce new risks of severe harm.

The company has found that following internal evaluations of Astra, one of its upcoming models, it may meet the critical level of cyber capability threshold under the Preparedness Framework.

On August 7, OpenAI said it was pausing internal activities relating to Astra until it meets stricter security controls. In its latest update, it revealed that Astra workloads remain paused until they are fully migrated and enhanced to meet the new security bar.