Securely Deploying Open-Weight Models

The adoption of open-weight models is accelerating. Many enterprises are excited to embed their specific edge and know-how into custom models that they own, control, and can continuously improve. This is the future we’re building around at Applied Compute: we believe that every enterprise should own its specific intelligence and that open-weight models will ultimately strengthen the American enterprise more than closed models will [1].

However, as the model moves from an outsourced commodity to a proprietary advantage owned and developed by each enterprise, new questions come up: What risks are involved? Do additional risks come with using Chinese-origin open-weight models? How do we deploy these models safely?

At Applied Compute, we’ve seen open-weight model deployments across more than twenty enterprise applications and five model families move from early ideation to production. We believe that both U.S.- and Chinese-origin models can be deployed safely in high-stakes enterprise applications when properly sandboxed and adequately evaluated.

Drawing on our experience, we present a framework for assessing and mitigating risks when building with and deploying open-weight models.

Defining open-weight models

Once a language model has been trained, it is distributable via a collection of files containing numerical parameters called weights. These weights are effectively the knobs and dials that control how the model behaves. They are not a database of facts or source code that can be read and audited directly.

Closed model providers keep their weights proprietary and offer access through products and APIs. They operate the model and can monitor usage, restrict access, and, without appropriate agreements, train on user conversations.

Open-weight providers publish downloadable weights that enterprises can run on their own hardware or through U.S. cloud providers, including within customer-controlled Virtual Private Clouds (VPCs). An enterprise can operate the model without a connection to its original developer, keeping inputs, outputs, and usage private. Open weights can also be modified through post-training to develop capabilities specific to an organization’s domain. The enterprise retains control over that proprietary augmentation.

Assessing risk factors

We tend to think about deployment risk in two broad categories: (1) risks associated with where a model runs and who can access its data; (2) risks associated with how the model behaves once it is deployed.

Data residency and access risks. Enterprise models will often process proprietary research, protected customer information, credentials, and other sensitive material. Where that information is processed, who can access it, and whether it is retained, shared, or trained upon are foundational security questions.

Agentic risks. Even when an enterprise controls where its models run and who can access their data, the models themselves can still behave unreliably or inappropriately. These risks become especially consequential when models have the ability to take actions and modify state on the enterprise’s behalf. In our work with customers, we’ve found the following agentic risks most common:

• Data exfiltration. An agent with access to both sensitive information and external networks creates a path by which confidential data can leave the environment through agentic actions.
• Unauthorized or irreversible actions. Agents may be given permission to modify records, delete data, send communications, or initiate financial actions. The severity of a possible model error depends heavily on these available privileges.
• Degraded work artifacts. Agents can produce plausible but incorrect analysis, code, or decisions. These errors can inform business-critical decision-making and may propagate through many subsequent steps before a human sees the result.
• Prompt injection, compromised context, and supply-chain risk. Agents consume information from documents, databases, websites, tools, and other agents. Malicious instructions embedded in those sources can redirect their behavior and trigger other failure modes, such as data exfiltration.
• Weak observability and accountability. Long sequences of model decisions and tool calls can be difficult to reconstruct, monitor, and understand.

These agentic risks apply to both open- and closed-weight models. A more specific concern with externally developed open-weight models, especially those of Chinese origin, is whether malicious or undesirable behaviors could have been intentionally or unintentionally embedded in the model weights. For example, this might look like models working on sensitive code for U.S. national security complying with requests but subtly introducing mistakes that are hard to detect.

We have never observed any such backdoors despite evaluating Chinese models across dozens of critical applications for American companies. That said, research literature has established that such backdoors are technically feasible and can survive subsequent training. In one study, researchers deliberately constructed backdoored models that generated vulnerable code at up to five times the rate observed without the trigger [2]. Published evaluations have produced different findings: CrowdStrike observed DeepSeek-R1’s vulnerable-code rate rise from 19% to 27.2% when told the user is working on industrial control systems in Tibet, while Cognition found no clear affiliation effect across four models and six user personas in the agentic deployment setting it tested [3][4].

Mitigating risk factors

We believe that data residency and access risks are more easily addressed with open models than with closed; these concerns are one of the reasons many of our customers are excited to use open-weight models in the first place. At Applied Compute, we deploy all models on Western-domiciled infrastructure, either under the customer’s direct control or through an approved U.S. cloud provider. This keeps model outputs and traces private from the model developer; the model developer maintains no network access or observability into the model’s usage.

Mitigation of agentic risks is somewhat more nuanced. Our approach at Applied Compute is two-fold: (1) Clearly define a contract for agentic privileges and implement guardrails in support of that contract; (2) Rigorously evaluate intended tasks in production-realistic environments to identify any potential failure modes prior to promoting a workload to production. Our recommendations generally align with guidance from National Institute of Standards and Technology (NIST) and U.S. and allied cybersecurity agencies on access controls, data protection, evaluation, monitoring, and human oversight [5][6][7].

Guardrails should establish clear boundaries around what an agent can access, change, and transmit:

• Data modification. Define which files, records, and systems the agent can modify. Maintain protected version history or backups outside its access, and require approval for changes with material or irreversible consequences.
• Internet access. Decide whether the agent can download information, upload data, or both. Specify the domains and services it can reach and what information it may send. Where connectivity is unnecessary, disable it.
• User permissions. The agent should operate within the invoking user’s permissions, narrowed further to what the task requires. Connecting additional tools or delegating work to other agents should never grant it authority the user does not have.
• Audit logs. Record model inputs, tool calls, changes to application state, and outputs. These records should be stored in a location where the agent cannot modify or delete them.

Within these boundaries, we begin every engagement with rigorous evaluations intended to establish baseline performance in a realistic production environment. This means using production data, even if it is a sandboxed copy, along with the intended tools and permissions, while removing obvious testing cues such as training references or evaluation-specific names. Research suggests that models behave differently when they believe they are being evaluated [8].

We often run extensive ablations across model families and harness components, classifying failure modes using taxonomies tailored to the deployment. We employ monitoring agents to inspect execution traces, flag unexpected actions, and surface recurring failure patterns for review. The findings help us determine whether to change the model, adjust the harness, or strengthen a guardrail.

These comparisons provide evidence about behavior under the conditions tested. They help us assess expected performance and probe for any contextual bias. But evaluations can never be completely exhaustive, and they are limited in their ability to surface backdoors tied to narrow, unknown trigger phrases. We therefore rely on clearly defined guardrails and permissions to limit the consequences of failures those evaluations may miss.

With careful attention to risk mitigation and monitoring, we believe both U.S.- and Chinese-origin open-weight models can support high-stakes enterprise work. Responsible deployment requires evidence from the actual workflow, clear limits on what agents can do, and continued evaluation as models and applications evolve.

References

[1] Microsoft, et al. (2026, July 24). Open Weights and American AI Leadership⌝. Joint open letter.

[2] Hubinger, E., et al. (2024). Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training⌝. arXiv:2401.05566.

[3] Stein, S. (2025, November 20). CrowdStrike Research: Security Flaws in DeepSeek-Generated Code Linked to Political Triggers⌝. CrowdStrike.

[4] Cognition. (2026, July 8). Measuring the Trustworthiness of Open-Source-Derived Models⌝.
[5] Australian Signals Directorate’s Australian Cyber Security Centre, et al. (2026, May 1). Careful Adoption of Agentic AI Services⌝.
[6] Autio, C., et al. (2024, July 26). Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile⌝. NIST AI 600-1. National Institute of Standards and Technology.
[7] National Security Agency, et al. (2024, April 15). Deploying AI Systems Securely: Best Practices for Deploying Secure and Resilient AI Systems⌝. Joint Cybersecurity Information; National Security Agency, et al. (2025, May 22). AI Data Security: Best Practices for Securing Data Used to Train & Operate AI Systems⌝. Joint Cybersecurity Information.

[8] Lynch, A., et al. (2025, June 20). Agentic Misalignment: How LLMs Could Be Insider Threats⌝. Anthropic Research.