返回资讯中心
外部精选
软件工程
#开发者实时流

Regulate the deployment, not just the model: a who-does-what plan for AI safety

AI has reached a new level of public interest. That may seem like a silly thing to say, as it’s been of interest for a while. But prior interest was about it as a tool, or about the money to be made or lost, or about an…

Hacker News 最新nedruod2026/10/24 · 13 天前30 分钟阅读

以下正文同步自 Hacker News 最新,版权归原站所有,已转换为易读排版。

Ryan Baker

Oct 20, 2026

3

1

Share

AI has reached a new level of public interest. That may seem like a silly thing to say, as it’s been of interest for a while. But prior interest was about it as a tool, or about the money to be made or lost, or about an effect on jobs.

The new interest demands more focused attention. It demands policy. It demands action. Those other types of interest were optional or illusory. The new interest is connected with risks I consider real. Being real doesn’t make these risks certain. But being real, even with uncertainty, is reason enough to act.

I’ve written a few pieces lately about risks, cybersecurity risks, and loss-of-control risks. These tried to explain complex topics, starting from the initial premise. Recommendations follow the explanation. This piece will be a bit different. Its focus is upon what I think we should be doing. You might consider it a policy piece. Policy in a completed form has to be a bit broad. There’s always more than one thing going on. As such, I’ll lay out my outline first, and then offer below the extra detail, background and context that I think is important to understanding each point in that outline.

If you’re scanning the outline and see something you disagree with, I’d suggest you continue on, because the outline is, by design, abbreviated, and not fully descriptive.

This isn’t the only effort of this type. Institute for Progress (IFP) and AI Futures are two well known efforts. What distinguishes this most of all is thinking more deeply about how models are deployed. This is a background element that’s somewhat ignored, because it’s not as novel and is not as visible to users. But it’s absolutely critical. I don’t expect this plan would replace plans from well known organizations, but by illustrating a gap, I hope to inspire improvement amongst those plans.

This outline is categorized by who would be expected to do what. In some cases, someone is expected to do it, but there’s also another role, of providing the incentives that improve the expectation they will.

Enforcement is deliberately vague. It depends on jurisdiction, has to fit existing legal frameworks, and will need to be worked out, and negotiated, through the political and legal systems. It would feel like I was putting the cart before the horse to even guess at how that would take shape at national and international levels.

Who does what. Model weights above the threshold reach users only through certified trusted deployers. Independent evaluators, certified by a regulator with legal authority, test the models and audit the deployers.

Layer 1: Infrastructure and Distribution

Trusted deployers

Who are they? A new designation that existing and new organizations can be certified as. Expect major cloud providers (AWS, Azure, Google), AI labs (for self-owned) and neoclouds to join immediately. The plumbing largely already exists, but needs standards and certification.

Revenue model? Charging for compute and operational services as a utility function. Or, an internal unit (for a lab, government, or large organization)

Develop secure environments to host models that receive an independent AI Deployment Safety Level (AIDSL) rating. Example, but not comprehensive list of requirements:

Comprehensive monitoring, telemetry, and auditing (including multi-turn agent pattern detection).

Resistant to external cyberattacks and insider threats.

Resistant to internal cyberattacks. At minimum red-teamed by deployed model and a second highly capable model.

Guardrails applied by default to all deployments which prevent access of dangerous capabilities.

Strict controls for any guardrail exceptions as used for “limited” access deployments. For example, accessing users must be known directly, or an evidence chain validated using only trusted organizations.

Understand the sensitivity of any model deployed. Never deploy a model in an environment with an AIDSL rating insufficient to the combination of deployment type and model sensitivity.

Use internal qualification for small custom models and Independent Evaluator data for larger/frontier models.

Models under training assumed to be at least X% more capable than the best model trained to date at the given scale. If the training scale is unprecedented, increase X by a size factor. (X should be established by independent evaluators)

Perform or validate Know Your Customer (KYC) verification for all model providers, for Independent Evaluators and anyone with access to deployments with guardrail exceptions.

Provide access to Independent Evaluators to confirm secure environments meet standards

Needs: Clear deployment standards from the regulator; model sensitivity data from evaluators.

Incentives: Compute and compliance service revenue

Who are they? None exist today. Hugging Face is a candidate to adopt this role or a government-supported entity could be formed if no private examples emerge.

Serve as the intermediary for labs that do not want to negotiate directly with each individual Trusted Deployer.

Escrow raw model weights for models that cross dangerous capability thresholds, holding them under maximum-security, air-gapped/high-AIDSL storage standards.

Distribute restricted models exclusively to certified Trusted Deployers whose environments meet the required AIDSL rating.

Needs: Standardized AIDSL verification APIs; cold-storage security standards.

Incentives: Service and distribution fees; maintaining platform centrality for the open-science ecosystem.

Independent Evaluators

Who are they? Organizations like METR, Apollo Research, Redwood Research, UK AISI, and US CAISI for model evaluations, alongside specialized cloud/security auditing firms (similar to FedRAMP/SOC2 assessors) and new dedicated organizations in the future. As an example, see Anthropic’s initial list.

Evaluate models for safety and capability thresholds using lab-provided resources. During training, before limited release, and before public release.

Audit and evaluate the operations of Trusted Deployers (and Model Clearinghouses) to assign and verify AIDSL ratings across inference and training environments.

Engage with and help create an Independent Regulator. Contribute technical expertise to improve safety evaluation methodologies and trusted deployment standards.

Educate government bodies. Especially bodies that would provide oversight to an independent regulator.

Needs: Deep model and infrastructure access; funding for salaries and operating costs.

Funding is a special concern here. You want independent evaluators to not be beholden to labs. Having their salaries dependent on labs would present a conflict of interest. Having labs provide necessary compute resources does not have the same influence, but salaries and operating costs are still a concern. Today these are funded philanthropically, which may remain practical for some time, but shouldn’t be taken for granted. Some evaluators may be able to create a revenue stream by selling safety expertise to users of AI. Governments could provide grants. A diversity of low bias sources is more stable than one single source.

Who are they? A new FINRA-like entity (as explored in Lawfare’s Designing a FINRA for Frontier AI) . Supported by frontier labs, trusted deployers, and the overall tech industry. Board membership mixed between industry and government.

Finalize guardrail standards and facilitate sharing guardrail threat intelligence and technical knowledge.

Certify Independent Evaluators. Track evaluations of Trusted Deployers and certify results.

Needs: Statutory authorization from US law. Participation of frontier labs.

Frontier AI Labs

Who are they? Anthropic, OpenAI, and Google today.

Support creation of a US-based independent regulator.

Contribute to guardrail development for frontier deployments.

Contribute to trusted deployment standards.

Utilize Trusted Deployers for inference deployments (public and limited).

Utilize Trusted Deployers for training deployments (first and third party).

Provide deep access to Independent Evaluators.

Provide cybersecurity model and tool access to cybersecurity teams.

Devote at least 30% of research budgets and compute capacity to alignment safety.

True alignment work, not cybersecurity work.

Accounting rule: Operating cybersecurity is a foundational component, like compute. If an alignment experiment needs compute and operational cyber costs, those count toward the 30%. Compute and operational cyber for capabilities research and inference do not.

Examples of alignment research: J-Space understanding, corrigibility, reward hacking avoidance, honesty research, observability evasion avoidance, model welfare research, ethics and constitutional effects.

Devote at least 10% of research budgets and compute capacity to cybersecurity improvements

Exclusive of operational cybersecurity. Operational cybersecurity is an operational cost built into all budgets.

Dedicated research into internal and external cybersecurity.

Internal Examples: Better classifiers to enable diffusing cybersecurity capabilities safely, hardening models against abliteration, new sandbox techniques.

External Examples: Device management to isolate or patch old vulnerable devices.

Offer 10% of research budgets and compute capacity to Independent Evaluators.

Needs: Access to compute for training and inference; clear deployment standards.

Who are they? Leaders are mostly Chinese today (DeepSeek, Alibaba, etc.), alongside Mistral and Meta. Likely to continue evolving.

Understand capability thresholds of their own models before release.

Below dangerous thresholds: Models may have their weight files publicly shared and freely downloaded.

Open access enabled by Model Clearinghouses and/or Trusted Deployers.

Needs: Low-friction clearinghouse distribution channels; standardized capability evaluation suites.

Tech Industry

Who are they? Major software vendors, cloud platforms, open-source foundations, and security providers.

Support the creation of the independent regulator

Develop the Trusted Deployer model technically

Develop security tools and processes (such as IFP’s The Great Refactor for automated vulnerability patching) to accelerate broader security progress across all commercial and internal software.

Needs: Early access to defensive cybersecurity models via Limited Inference Deployments.

Incentives: Protecting core platform ecosystems from systemic AI-driven cyber disruption; commercial demand for resilient software.

Who are they? Non-tech enterprises, critical infrastructure operators, and commercial AI adopters.

Actively adopt tech industry security assistance and defensive AI tooling.

Deliberately prioritize cybersecurity and resilience for all tech-related functions—moving beyond routine patching to systematic hardening and continuous monitoring.

Incentives: Avoiding operational breaches; qualifying for cyber insurance and liability protections.

The US Government needs to develop capability before it can be a successful regulator. In the immediate sense, it needs to support efforts to build dialogues, to establish recognized expertise and standards. It should pursue structures that are flexible. This is both because they can be adoptable quickly and can improve with experience.

Provide statutory backing for the FINRA-like Independent Regulator to oversee standards for public inference, limited inference, and training deployments.

Conduct bilateral dialogue with the Chinese Government:

Immediate priority: Establish mutual adoption of the Trusted Deployer model for high-capability models (e.g., GLM-5.3 or higher) so open-weight proliferation does not undermine global guardrails.

Longer-term priority: Work toward verifiable international pacing agreements

International agreements are critical, but also difficult. They are difficult because of current relationship strains, but also because verification is complex. If

正文由 探How 从来源站点 RSS 同步,内容未经改写;遇到排版缺失或需要图片、视频时请以原文为准。

打开原文
去社区讨论 →