— Open weights expand access to the AI economy, they strengthen competition at least for some use cases, and they give customers greater control, CEO Dario Amodei writes in a policy document.
He does, however, highlight a couple of «nightmare scenarios» that he hopes to avoid. The first is that authoritarian governments could get access to more powerful models than the USA and use those to further oppress their own people or reach military superiority — and he says it doesn’t matter if these models are open or closed.
The second is that powerful models can be misused in cyber or biological attacks due to «serious alignment problems.» In this case open weight models do carry increased risk «because it is very difficult to apply guardrails,» Amodei says.
Chips ban and expanded testing
In order to fix or mitigate these issues, it would be helpful to keep up the US chips ban on China, as he thinks scaling laws hinder more powerful models from being developed.
He also again calls for mandatory safety testing of «all sufficiently capable models,» open or closed, for «cyber, biological and alignment risk.» He says this testing should be global, and that China should join in.
Nvidia has been busy with open letters lately, sharing one with industry-wide signups in support of open source models, where Anthropic was conspicuously absent, and another launching the Open Secure AI Alliance. None of them appear to have a single Chinese signatory.
Read more: Amodei’s essay. Writeup on Axios. Discussion on r/Singularity.