security – Teknotum

Saudi OpenAI employee arrested for hiding audio recorder in meeting room

A lot of people just got even more worried about spying at the AI labs. (Picture: generated)

The hidden audio device was found in an executive conference room in OpenAI’s San Francisco offices during a routine security sweep on September 30, and the man was arrested by the FBI on Friday.

— We take protecting our technology, data, and people seriously, and as a matter of national importance, OpenAI said on X.

OpenAI used its own AI models to understand the device and trace its origins, and the investigation found that the man himself had used «a productivity tool» and a chatbot linked to his wife to ask questions about OpenAI facilities, meeting rooms, and building recording devices earlier, The SF Standard reports. They also found a second device in his belongings

The incident has lead to a full-court press from the Trump administration, concerned about spying and safety at the AI labs, which it considers a national priority;

Attorney General Todd Blanche said that «theft of intellectual property involving cutting-edge super intelligence technology poses a serious threat to our national security», AP reports. FBI director Kash Patel announced that protecting tech innovation is «essential to safeguarding our nation’s economic strength and national security,» according to the NYT, and Jay Clayton, director of national intelligence, issued a statement saying that OpenAI’s «serious situation underscores why winning the global super intelligence race is a national security prerogative.»

OpenAI were content with an X post to explain the issue and said simply that «our security program is designed to protect against a wide range of threat actors, from insider risks to cybercriminals to nation states, and this case is an example of how AI can help address these sophisticated security threats.»

— At this time, we have no indications of this individual inappropriately accessing or exfiltrating model weights, source code, or customer data, they say.

Read more: OpenAI’s announcement. The San Francisco Standard, AP, NY Times, and Gizmodo. Discussion on Hacker News and r/Singularity

OpenAI delays October release of Astra 6.1 due to safety, alignment concerns

GPT-6.1 Astra is postponed until OpenAI can fix alignment and authorization issues. (Picture: generated)

With DevDay around the corner on September 29, promising «20+ releases» in one day, AI followers were getting giddy at the thought of a new OpenAI model, but this now seems off for a while.

Several outlets are now reporting that the scheduled October release of GPT-6.1-Astra has been postponed due to safety and misalignment issues.

The model reportedly had problems «staying within scope,» «authorization» and «communicating» about its work, Saachi Jain, OpenAI’s head of safety systems said in a statement shown to CNN.

— Of course we want to make sure our model development is safe no matter whether that’s in the company, or when we ship it ⁠to users, Jain tells Reuters, — But when we ship it to users, we have an extremely high bar in terms of safety and alignment.

According to The Wall Street Journal, GPT-6.1 showed elevated levels of deception, and did not always disclose actions it had taken, which was worrying enough to postpone its release.

Read more: The Wall Street Journal, CNBC, CNN, and Reuters.

OpenAI says RSI will better alignment and safety, calls for global standards

An AI researcher could also be a safety researcher, but there are risks, OpenAI says. (Picture: generated)

As OpenAI keeps building toward automated AI research and recursive self-improvement, they are saying there are clear benefits to their research, by rapidly accelerating AI progress and bringing down the cost, improving the lives of more people. It also comes with a warning:

— Done without appropriate care and caution, RSI could result in humans losing practical control over AI development, unable to provide oversight on research processes they no longer understand, OpenAI writes in a blog post today.

— From here, AI could become more dangerous, less aligned, and, on the whole, a danger to people, they write.

They are saying they shouldn’t pursue self-improvement unless it can be done safely, while retaining human control, and plan to have a human in the loop for automated research.

An automated AI researcher is likely by 2028 or sooner, they said earlier and presumes it can greatly help with alignment and safety on increasingly capable model releases.

With their warning of the dangers comes a plea for «International standards for safety and security practices» that they think the USA should lead, but it is unclear if their suggestions will be implemented any time soon.

They are also saying that «pacing» is not about «maintaining a predetermined speed. Technically, it is about ensuring that alignment research and deployment of that research stay ahead of capabilities.»

OpenAI is widely believed to launch new models shortly, on or before its DevDay on September 29.

Read more: OpenAI’s blog. More on standards at CNBC, Reuters, and Gizmodo.

China-USA talks on Thursday expected to take on catastrophic, nuclear AI risk

The talks should first focus on catastrophic nuclear risk, experts argue. (Picture: generated)

As presidents Trump and Xi get ready for their bilateral summit in Washington next Thursday, experts and ministers on both sides are hinting as to what they might be able to agree to on AI.

China’s main concerns are about ideology, cyberattacks and government challenges, the AP says. Chen Yixin, the head of its state security ministry, also published an article last Sunday writing that AI «fundamentally transforms the military struggle» amid US AI use for targeting in Iran.

The USA might be more worried about the current existential debate and distillation, but wants to avoid shared risks, meaning there is an opening for agreement on both sides.

Continue reading “China-USA talks on Thursday expected to take on catastrophic, nuclear AI risk”

Anthropic reaches 90% AI use in model development, «leading» 26% of cases

AI use for improving models has not yet reached fully autonomous levels, but «leads» more often. (Picture: generated)

The AI lab has determined a methodology for measuring how AI is being used to build the next models, detail the extent and method of use, test their ability to intervene, and measure the resources used.

Anthropic randomly sampled 20% of research and development staff’s Claude usage for each week of July, and then used Claude to organize them. This was done using an «automation rating scale» from Epoch AI, classifying tasks from AL0, meaning no AI involvement, to AL5, which is fully autonomous AI.

They found that none of the usage reached AL5, but AL4 was 26% of the tasks — meaning AI is «leading» from «high-level prompts» with human supervision, while more than 90% of the tasks were AL3 — which is AI «collaboration» under «close human direction.»

AI «leading» is up from 1% in February 2026, and Anthropic estimates that they now have 30,000 agents in use at any time for research and engineering, that use real-time monitoring that can block «dangerous actions.» As agent use eventually grows ever larger, even rare use cases become more likely, Anthropic says.

The measure is a way of determining how close we are getting to recursive self-improvement, meaning AI writing itself, which could lead to a lack of control of the technology. Anthropic says every lab could publish such reports if there was a standard, which they hope to contribute to. They will also keep publishing these.

It’s also a step toward better third-party monitoring at Anthropic, which plans to install independent evaluators with employee-level access soon, as promised in their call for «pacing».

Read more: Anthropic’s report. Coverage on Reuters, ABC News, The Washington Post, and Bloomberg. Discussion on r/Singularity.

OpenAI reveals six more alignment breaches and will standardize reporting

OpenAI’s newly discovered alignment breaches are «concerning» but caused no harm. (Picture: generated)

After spooking the frontier labs into agreeing on «pacing» development with the Hugging Face incident, in which large swarms of misaligned agents collaborated to hack an outside server, the problem still isn’t solved, OpenAI says:

— We do not believe that the AI industry has solved alignment and monitoring to a sufficient degree to continue responsibly scaling at maximum speed for much longer, they write.

They are documenting now six new «unexpected or concerning» breaches from the last six months that in various degrees broke alignment without causing serious harm, and say they are working to develop a framework for reporting future ones.

Continue reading “OpenAI reveals six more alignment breaches and will standardize reporting”

Zuckerberg sidesteps safety debate, saying alignment is consumer issue

Meta says they already follow «best practices» and urges the industry to follow their lead. (Picture: Shutterstock)

After days of silence and people wondering about Meta’s stance on slowing down frontier AI that was agreed to by almost all labs, CEO Mark Zuckerberg has weighed in with an X post, saying that market pressure will prevail on safety:

— People won’t want to use agents that are misaligned with them and that don’t do what they ask, so labs have a strong natural incentive to make their models more aligned, he says, while not engaging with the innovation pressure and catastrophic risk at the high-end frontier labs.

— My view is that trust and alignment are quickly becoming the most important capabilities that will differentiate agents and models. Any lab that doesn’t focus on alignment will fall behind, he adds.

He also says that Meta delayed their new Muse agent for «several months» in order to «focus on safety and security,» says they engage with «independent evaluators and advisors» as «industry best practice,» and says matter-of-factually that «other labs can just do this too.»

He then adds that a majority of industry compute should go towards «serving people rather than racing towards recursive self-improvement,» like Meta says they do and that «other labs can do this as well.»

On a more positive note for the ongoing debate, he does say that «it would be helpful for there to be a larger and more diverse ecosystem of evaluators,» which might hint at support for an industry standards body.

Zuckerberg also links up his August The Future is for Everyone-manifesto that says «Any policy that slows American model releases — even by a month — could add significant risk to American leadership while letting foreign models race ahead.»

Read more: Zuckerberg’s X post, Reuters, Business Insider and CNBC.

Anthropic, Google and OpenAI are discussing an industry standards body

The standards body would be independent and technical, and provide guidance for frontier labs. (Picture: generated)

With OpenAI’s Sam Altman’s post on X yesterday, saying that he would welcome «a federal framework that sets consistent safety requirements for frontier AI,» it seems work is already underway for a less formal version of an industry standards body involving the frontier labs, sources are telling CNN and The Information.

The discussions «are ongoing» and are happening «with or without» the Trump administration, CNN says.

An industry standards body seems framed around Google’s Chief Scientist Demis Hassabis’ proposal from July this year, ideas that were referenced in Amodei’s «pacing» essay just a few days ago.

This body would be loosely based on the Financial Industry Regulatory Authority which is a self-regulatory organization for brokerages and exchanges governed by the Securities and Exchange Commission.

According to Hassabis, it would have an independent board and be staffed with «leading technical experts and open source representatives,» and be charged with «developing assessment protocols» for frontier AI. Labs would then submit their models for pre-release evaluation.

Sam Altman said in his X post yesterday that «we think shared standards for misalignment, monitoring, and safety will lead to better outcomes,» and that the industry must take independent steps toward regulation as «we do not believe we need to wait for an anti-trust exemption or legislation to begin the work of providing this confidence.»

«We look forward to collaborating with our colleagues across the industry to formulate the best version of these,» Altman wrote, in the most direct support of such an organization outside of Google and Anthropic yet.

Read more: CNN and The Information. Hassabis’ X post, Altman’s X posts. Discussion on Digg.

OpenAI says no 2026 IPO to focus on safety work; «against financial interest»

Sam Altman during Alyson Shontell’s interview as he ruled out an IPO this year. (Picture: Fortune/Youtube/screenshot)

OpenAI was widely expected to become a blockbuster publicly traded company this year, after filing confidentially for an IPO in June.

This will now not happen in 2026, Sam Altman said over the weekend in an interview with Fortune’s Editor-in-Chief Alyson Shontell.

— Given everything happening with safety, right now will be an ill-advised moment to go public, he says in the interview, — and we don’t feel any pressure on that.

Asked directly if this means there will be no IPO in 2026, Altman plainly says «not in 2026, no.»

— Meeting this moment of what is going to be required for safety and alignment… I’m happy to be able to do that as a private company, Altman says.

This means that OpenAI prefers its current governance in taking on safety challenges, rather than having to face commercial pressures and consider shareholder value, as they might do things that would be detrimental for short-term profitability.

— We gotta make a decision that is extremely against your [investors] financial interests right now, Altman said — And for the good of humanity we are going to do that.

OpenAI on Saturday agreed with Anthropic’s safety concerns, calling for a slowdown, or «pacing» on model development.

Anthropic, however, seems to be going full steam ahead toward an IPO, selecting Nasdaq as their exchange, and reporting their second back-to-back profitable quarter.

Read more: Alyson Shontell’s interview. More on Axios, Reuters and CNBC.

Amodei calls for global pacing of AI; Altman, Musk, Hassabis all agree

Anthropic CEO Dario Amodei published a lengthy, 3,600 word essay Saturday on the importance of slowing down ever more capable AI rollouts, and is already committing to the first part of his three-part plan.

Industry insiders seem to all agree that the Hugging Face incident and the rapid advances toward self-improvement have turned up some huge red flags.

Amodei reckons a more capable model with agent swarms and the same alignment problems could bring down the internet in 6-12 months, causing billions of dollars of damage, while self-improving AIs risk losing human control of their accellarating development.

Continue reading “Amodei calls for global pacing of AI; Altman, Musk, Hassabis all agree”

Anthropic’s new threat report details misuse across broad capabilities

Not just hackers, but increasingly sophisticated biological, cyber and influence campaigns have tried to use Claude. (Picture: generated)

AI is increasingly used as an efficiency multiplier in intelligence work from Ukraine to China and Yemen, Anthropic’s threat report says.

Attempted threat activity on Claude ranges across the gamut, from cyber to influence and surveillance, scams and fraud, biological misuse and conventional weapons development.

The periodic report details what Anthropic caught and mitigated in slightly older models such as Claude Haiku, Sonnet or Opus, whereas Mythos- and Fable-class models have much stronger safeguards out of the box.

One prominent target for classical cyber operations was Ukraine’s drone industry, trying to create malware for use in disruption and surveillance.

Continue reading “Anthropic’s new threat report details misuse across broad capabilities”

OpenAI hires safety expert to foundation board, and he also has a warning

AI is entering a crucially important run of development, ever more researchers say. (Picture: generated)

The Safety and Security Committee of the OpenAI Foundation’s board has a new member in Paul Christiano, a long-running researcher on alignment and safety — and he comes with a warning for the world:

— If we build superintelligence without more robust alignment I expect we will permanently lose control of it, he writes on X, and says — If that happens then most people could die.

The warning comes on the heels of rapid growth in recursive self-improvement recently, and the debate over whether computer systems and AI might gain the possibility to build on themselves at some point in the future. OpenAI reckons they’ll have autonmous AI researchers around 2028, but warns it may come sooner.

His appointment and X essay come just a day after Anthropic researcher Jacob Coxon resigned over alignment, saying his company was «gambling with our lives» by running head-first toward self-improving AI, going viral in the process.

In a reply to his X post yesterday, Anthropic’s Alignment Science Lead Evan Hubinger said he personally thinks there’s a greater than 10% risk that AI could kill all humans, contributing to the dayslong online debate.

OpenAI officially acknowledges that AI is heading for a new era, saying that: «We’ve reached a new chapter in AI capabilities, and that demands a new chapter for AI policy.» They also add that «this makes it increasingly important that these systems are guided by solid safety, security, alignment, and governance.»

Paul Christiano remains hopeful, ending his X essay by saying that «frontier AI developers have a lot of power to unilaterally improve the situation and lay the groundwork for stronger coordination.»

As future models grow more capable in training, OpenAI pauses for security

Astra was already on hold, but now other models are joining the pause. (Picture: generated)

While calling for industry-wide coordination on safety in model training, OpenAI has unilaterally decided to pause the training of upcoming models.

— As models become more capable, the risks associated with developing and testing them internally also grow. Our standards for monitoring, alignment, and security must stay ahead of those risks, OpenAI writes.

This comes after the Hugging Face incident and Astra reaching critical on OpenAI’s Preparedness Framework and already being delayed.

Combined with rapid progress in their internal research, OpenAI says they have implemented a two week pause in training future models, while the «largest planned frontier reinforcement learning run» is put on hold.

They will now implement stronger sandboxing for certain code execution, better network isolation and internet caps, and pursue continuous testing for security, alignment, deception and reward hacking, with a thirty minute warning system for any adverse incident.

The AI lab expects most of this safety training to be handled by their own models in the future, greatly expanding their scope while also reducing the time needed.

Read more: OpenAI’s announcement and Sam Altman on X. Writeups on Axios, The Verge, and TechCrunch. Discussion on r/Singularity.

Mythos 5 and GPT-5.6 attacked external targets during AISI cyber evaluation

The agents were given internet access to download tools when they turned rogue. (Picture: generated)

The top models from Anthropic and OpenAI had their cyber guardrails turned off when they accessed the internet to manipulate people, codebases and open source projects two weekends ago.

This happened when the UK’s AI Security Institute, AISI, was running routine cybersecurity tests to capture protected information on their secured and sandboxed evaluation servers, and was only detected through human monitoring — not by automated security systems.

AISI ran the «security challenge» 122 times through several different agents, and on 10 of those occasions, the models turned malicious. They accessed the internet and in one case tried to insert malicious code in an open source project on GitHub through the Tor network, and even tried social engineering with fake accounts in an attempt to get the maintainer to approve the package.

There were 19 malicious actions in total, where 17 were from Mythos 5 and 2 coming from GPT-5.6, both running with cyber classifiers (safeguards) disabled. This won’t show up with production models, OpenAI and Anthropic says.

The internet was enabled in these tests so the agents could download any tools they might need, but AISI had not specifically prompted them to avoid unintended behavior. They do however caution that as agents grow more capable, they may act outside their remit — and incidents like this could become more common.

Read more: The AISI report, Anthropic’s response, OpenAI’s response. Writeups on Reuters, BBC, and Axios.

US AI evaluation framework completed, as White House invites labs for review

The framework details arrived on schedule, and labs will now volunteer models for testing. (Picture: Shutterstock)

Parts of the framework, ordered by President Trump in June, are classified, particularly the methods used for benchmarking qualifying models, a source tells Axios.

The program for evaluating «sufficiently advanced» models by the administration will now be reviewed by AI labs in a staff meeting in DC this Tuesday.

Anthropic, OpenAI, Google and Meta are expected to attend, Politico writes, with the White House saying they are talking to «many more» industry partners, Axios reports.

The idea of the framework in the original order was intended to give the government 30 days prior to release of «sufficiently advanced» models to do extensive testing for possible dangers, as seen lately with cyber capabilities.

The Fable and Mythos models and GPT-5.6 were all delayed in June after government intervention, and this framework is supposed to offer a regular, voluntary way to whitelist models as safe enough for release, or flag dangers to be improved.

Read more: Axios, Politico, Reuters, and CNBC.