Anthropic's New Tech: Isolate Parameters, Remove AI Risks
Anthropic's SGTM isolates dangerous AI knowledge into removable parameters—no data deletion needed.
“AI Disruption” Publication 8400 Subscriptions 20% Discount Offer Link.
In recent years, the capabilities of large language models have advanced rapidly, but this has been accompanied by increasingly intractable dual-use risks. When models learn from massive amounts of publicly available internet data, they not only master language and reasoning abilities but also inevitably encounter highly sensitive and potentially dangerous knowledge domains such as CBRN (chemical, biological, radiological, nuclear) hazard creation and software vulnerability exploitation.
To address this, researchers typically incorporate safety measures such as refusal mechanisms during post-training, hoping to block the misuse of these capabilities. However, the evidence shows that these defenses are not robust against deliberate adversaries seeking to circumvent them. The model’s power places it in a delicate and fragile balance between being protected and being bypassed.
This has prompted researchers to begin exploring interventions during the pre-training phase to fundamentally prevent models from acquiring dangerous capabilities.
The current standard approach is data filtering: identifying and removing harmful content before training. However, this method faces multiple challenges:
High annotation costs and imperfection: Accurately identifying all CBRN-related content among billions of documents is both expensive and error-prone.
Harmful content is often mixed within benign documents: For example, a chemistry textbook may be mostly beneficial educational content, but it may also contain knowledge that could be misused.
Dual-use knowledge is highly entangled: Many concepts inherently possess both beneficial and risky characteristics, making completely clean separation impossible.
Improved sample efficiency of models: Recent research indicates that as model scale increases, even minimal amounts of dangerous data can significantly enhance model capabilities on related dangerous tasks.
