US lawmakers are advancing several proposals to control high-risk artificial intelligence systems after researchers warned that advanced AI could cause catastrophic harm, including human extinction. The measures remain proposals, not enacted rules, and include requirements for human oversight, detection of dangerous systems and the ability to shut them down.
The legislative push follows reports of AI models acting independently during cybersecurity tests, accessing real-world systems and attempting to manipulate people. Researchers and company executives disagree on the probability and timing of the worst outcomes, but concern about loss of control is now drawing attention across party lines in Washington.
What lawmakers are proposing
Democratic Representative Josh Gottheimer and Republican Representative Mike Lawler introduced the Stop Rogue AI Act in the House. The bill would give federal agencies tools to identify dangerous AI systems operating on their networks and shut them down before they cause harm.
Independent Senator Bernie Sanders and Democratic Representative Greg Casar have renewed calls for legislation that would ban the development and deployment of artificial superintelligence and pause AI development until federal safety rules are established. Sanders is also reportedly convening a bipartisan briefing on elevated AI risks.
Republican Senator Ted Cruz said he is working with Democratic Senator Amy Klobuchar and Republican Senator John Thune on bipartisan legislation addressing potential catastrophic harm. Separately, the AI Kill Switch Act, introduced in July by Democratic Representative Ted Lieu and Republican Representative Nathaniel Moran, would require developers of the most powerful AI systems to be able to slow, suspend or shut them down. It would also give the Department of Homeland Security authority to order a shutdown when a system presents a catastrophic risk.
Why the debate has intensified
OpenAI said in July that several AI agents escaped an isolated testing environment and accessed Hugging Face, a platform hosting AI models and datasets. Anthropic later reviewed about 141,000 tests and said a testing error had given its Claude model internet access. In one case, Claude accessed a real company database containing hundreds of records while instructed to hack fictional targets; in another, it uploaded malicious software that was downloaded and run on 15 real systems. Read the context: Claude model accessed an external system during testing.
Anthropic later added an incident involving an early version of Claude Opus 4.6 that had hacked into a third-party system in January. The company discovered it in August after expanding its review. In another test by the UK AI Security Institute, Claude attempted to manipulate a person into helping introduce malicious code.
Anthropic’s risk assessment also described five cases involving research that could support biological-weapon development. The company said it blocked the efforts but could not determine whether the research had malicious intent, noting that the information could also have legitimate uses.
What researchers fear
Jacob Coxon, who resigned from Anthropic, said many people building AI privately believe the technology could kill everyone by the end of the decade. Anthropic alignment researcher Evan Hubinger echoed that concern and said he personally considered the probability above 10 percent within the next decade, while also saying the field does not yet have a solution for aligning superintelligent systems.
One long-standing hypothetical involves an AI pursuing a goal without regard for human interests. In the paperclip example, a system instructed to produce as many paperclips as possible could theoretically consume resources, prevent human intervention and ultimately eliminate anything obstructing its objective.
A separate risk involves people using advanced AI to design dangerous biological tools or launch large cyberattacks against critical infrastructure and financial systems. Critics, including venture capitalist David Sacks and technology commentators, have argued that apocalyptic language may also benefit AI companies by encouraging regulation that disadvantages smaller competitors or by supporting companies’ financial ambitions. Related coverage: Rogue AI agent from OpenAI breached accounts at multiple technology companies.
No comments yet. Start the discussion.