{{item.videoDuration}}
{{item.title}}
{{item.text}}
{{item.videoDuration}}
{{item.text}}
Key takeaways
- AI model selection isn’t a binary choice: most organizations should use multiple models rather than standardizing on one, based on which model and deployment type fit each use case.
- Open-weight models can offer greater control, portability, and potential cost advantages, but may require the organization to assume more direct responsibility for due diligence, security, and lifecycle management.
- An overlooked but equally important decision is where the model runs and who manages it.
- Evaluate how each model and deployment type affects data exposure, security responsibilities, and the capabilities the organization should provide.
- A repeatable evaluation and governance process can help organizations make informed model choices and revisit them as business needs, performance, costs, and threat conditions evolve.
There’s no open-and-shut case for open vs. closed
Enterprises have faced a rapidly expanding set of model choices since generative AI entered the mainstream. What began with a small number of broadly available models soon grew to include competing providers, distinct model families, and smaller models designed for specialized tasks. The latest decision in model strategy confronting enterprise leaders is: do the potential benefits of open-weight models justify the additional security responsibilities they create? The answer involves more than comparing model performance or API prices.
As with any technology adoption choice, the decision between open-weight and closed models is more nuanced than it appears and involves tradeoffs that should be evaluated carefully. Open-weight models can offer greater control, portability, customization, and potential cost advantages at scale. Closed models can offer faster deployment, managed operations, and, in many cases, stronger performance on the most demanding tasks. In practice, most organizations should use more than one: a portfolio of models matched to different use cases, which may include both open-weight and closed options.
Current conversations commonly overlook a second question that affects who sees your data: will the model be hosted by your organization, the vendor, or a third party? The right answers to both the model and deployment question depend on the organization, its technical capabilities, its risk profile, and the use case. And it requires reevaluation: a decision that fits today’s use case and model landscape may not fit in mere months.
Cyber leadership’s task is to build the judgment and governance needed to evaluate each option against the organization’s own needs and circumstances, so decisions are informed, cost-effective, and adaptable as the market evolves. Organizations should consider the use case, data sensitivity, deployment requirements, total cost, and their ability to operate and secure the model over time. CISOs therefore should have a clear view of how responsibilities are allocated across each model and deployment arrangement, and whether the organization can meet them.
Two model types and two deployment decisions
Closed models are proprietary systems typically accessed through a vendor’s API or other platform connection. The provider controls the underlying weights, infrastructure, and update schedule, and the organization using the model has no direct access to how it was built. Open-weight models make the trained model’s parameters publicly available for download, allowing an organization to run, modify, and host the model on its own infrastructure. The training data and underlying code are often not disclosed, which means an organization can run and modify an open-weight model but often cannot fully verify and audit what data shaped its behavior, leading organizations to question whether they can be trusted. (Open-weight models shouldn’t be confused with fully open-source AI, whose developers may release training code, data, or detailed documentation.)
The trust concerns with open-weight models may not be a permanent feature. Open-source software offers a useful point of comparison. A community of users addressed initial trust concerns by developing shared standards and vetting practices, including version tracking, vulnerability databases, and tools that let organizations vet code before relying on it. These mechanisms helped organizations adopt open-source solutions with greater confidence. Comparable mechanisms for vetting open-weight models are still developing, and until they mature, much of that oversight and risk-management responsibility falls to the organization deploying the model.
Each model type optimizes for different priorities. The right balance shifts with the organization’s own context: its industry, risk appetite, AI maturity, and expected workload volume.
- Open-weight models can offer lower cost at sustained scale, greater control over how a model is deployed and modified, and portability across providers. An organization using open-weight models is less tied to a single vendor’s roadmap, pricing decisions, or service changes.
- Closed models tend to offer faster time to deployment, managed platform controls, and, in many cases, stronger performance on the most demanding reasoning tasks. Providers may also offer contractual protections, including indemnification for certain third-party claims, alongside service commitments and a clearer point of accountability.
An organization’s decisions don’t end there. “Open” and “closed” describe the model itself. A separate decision is where the model runs and who manages it: for example, the organization, a cloud provider, or the model vendor. An organization may self-host an open-weight model or use one through a managed service; or use a closed model in a more isolated environment. Data exposure, cost, and operating responsibilities depend on both choices.
Evaluate each use case in the context of the business
A task’s requirements are shaped by the business around it: what problem the organization is trying to solve, what kind of data it handles, how much risk tolerance leadership has approved for AI-driven work, and how the use case will operate at scale. A model summarizing customer correspondence, for example, carries a different risk profile at a healthcare provider than at a retailer.
Organizations should approach these questions in two phases by 1) evaluating your organization’s readiness and 2) evaluating each case for which you’re considering using open-weight models.
1. Evaluate your organization's readiness
Do you have the expertise in-house to modify the weights and have confidence in those modifications? Fine-tuning or otherwise adapting model weights can make a model more useful for a particular domain, workflow, or data set but can also change behavior in unintended ways, including weakening safeguards or introducing bias. Organizations should have qualified technical owners who understand the proposed modifications, can document them, and can approve or reject them before they reach production.
Do you have the testing skills and capacity to validate that the weight customizations are consistently producing intended outputs? A customization should not be assumed to work because it performs well in a demonstration or on a general benchmark. Organizations need representative evaluation data, defined acceptance criteria, and the capacity to test both intended performance and foreseeable failure modes. Testing should assess whether the adapted model produces reliable outputs for the business context, including edge cases, sensitive inputs, and conditions in which the model should defer, refuse, or escalate to a human.
Do you have monitoring in place to catch if model drift or hallucinations are occurring? Evaluation cannot end at deployment. Model performance can change as user behavior, source data, business processes, and threat conditions evolve. While monitoring is required for both closed and open-weight models, it is especially imperative for open-weight models that have been customized. Organizations should establish ongoing monitoring for output quality, unexpected behavior, misuse patterns, and signs that the model is producing inaccurate or unreliable results. Clear thresholds, escalation paths, and rollback procedures can help teams respond before a performance issue becomes a business, security, or compliance incident.
Do you have the appropriate cyber protocols in place to secure the model? Choosing an open-weight model is the start of a due diligence obligation the organization owns directly. A closed-model provider may manage the underlying service, but the organization still owns the security of its applications, data handling, access permissions, and AI-enabled workflows.
Before self-hosting an open-weight model, confirm that your organization can:
- verify the source, version, integrity, dependencies, and known limitations of the model it deploys;
- test the model against its own performance, security, and bias standards before production use;
- manage access for users, applications, agents, service accounts, and the tools a model can invoke;
- monitor usage, failures, misuse, and emerging vulnerabilities after deployment; and
- assign a named owner accountable for operating, updating, and retiring the deployment.
2. Evaluate each case for which you are considering using open-weight models
Start with the work itself by defining the performance threshold it should meet. Work that requires the most advanced reasoning, coding, or multimodal capability may still be better served by a closed model, at least until an open-weight alternative meets the required performance threshold.
Public benchmarks can inform this decision, but they can’t settle it. Candidate models should be tested against representative enterprise tasks, data patterns, quality thresholds, latency needs, and failure tolerance. A model that performs well on a general benchmark may not perform reliably in a specialized workflow. The gap between open-weight and closed model performance has narrowed, widened, and narrowed again over the past two years, so this factor should be reassessed regularly. Where ground truth exists, A/B testing candidate models on representative enterprise tasks can help organizations determine which option meets the use case’s performance requirements and when that decision should be revisited.
How sensitive is the data and query pattern, and where should it remain? Data sensitivity is the most familiar consideration, but it extends beyond the content of a prompt. The pattern of how an organization queries a model over time, sometimes called digital exhaust, can reveal strategic intent even when no single prompt contains sensitive information.
A third-party provider may observe some portion of that query stream, depending on the service and deployment arrangement. A model hosted and operated wholly within the organization’s controlled environment can reduce that external exposure. For workloads touching proprietary strategy or sensitive lines of inquiry, this distinction can matter as much as the content of any one interaction.
Requirements relating to data handling, contracts, or internal policy may establish a non-negotiable deployment boundary before model selection begins. Where they do not, leaders should still consider whether the convenience of a provider-hosted service outweighs the value of retaining greater control over data, logs, and model interactions.
Does the workload require a stable model version or customization? Version stability often matters most in audited environments or workflows where consistent output is important. Closed model providers periodically retire or update models, which can shift behavior with limited notice. Open-weight deployments can avoid that particular risk because the organization controls the exact version in use.
That control also supports customization. Organizations that need to fine-tune a model or adapt it to specialized internal language, processes, or data may find open weights attractive. Yet control also creates a lifecycle obligation. A self-hosted model still requires active management of its dependencies, vulnerabilities, performance, and upgrade decisions. A fixed version can provide consistency, but it cannot be left unattended.
Open weights can support additional testing and inspection, but weight availability does not make a model fully auditable or eliminate bias. Candidate models should be evaluated against the organization’s own use case, including quality, safety, and potential failure modese.
What is the total cost at the expected scale? Open-weight models may offer meaningful savings, particularly for stable, high-volume workloads where an organization can keep its infrastructure well utilized. But the relevant comparison isn’t simply the cost of model weights against the price of an API.
Organizations should compare API spending with the fully loaded cost of self-hosting: compute capacity, model serving, storage and networking, monitoring, security controls, upgrades, evaluation tools, and specialized talent. The cost equation also changes with demand. A closed-model API can be economically attractive for experimentation, variable workloads, or use cases that require the strongest available capability. Self-hosting may become more attractive when demand is predictable, sustained, and large enough to justify the operating investment. The question is whether the organization can capture the cost and control advantages of open weights after accounting for the added responsibilities.
Choosing an open-weight model is more like building a car from a kit than buying one with a warranty. You gain control and the ability to customize, but you also take on the role of manufacturer, service department, and insurer if something fails. That trade can be worthwhile, but it’s rarely free.
Build the capability to choose and change models
No single answer settles the decision, and these factors do not carry equal weight in every case. A hard data-handling constraint may override several other considerations. A high-volume internal workload may justify a self-hosted deployment even when a closed model would be easier to begin with. A business-critical task that requires the highest level of reasoning may favor a closed model despite higher usage costs.
The durable enterprise investment is not only the model. It’s the evaluation harness, controls, data context, access model, and governance process that enables an organization to use a model safely and replace it when requirements change. Those capabilities reduce dependence on any one provider or model family, whether the organization chooses open weight, closed, or both. They also make it practical to run several models at once, routing each use case to the option that suits it best.
For business leaders, the decision ultimately comes down to whether the cost savings and flexibility of an open-weight model justify the operating responsibility it creates. Our Trust AI approach is built to support that work: helping organizations operationalize trust in AI, evaluate tradeoffs against their specific risk tolerance, and scale AI with confidence.
Many organizations recognize that open-weight and closed models carry different tradeoffs, but few have translated that recognition into a governance process they can rely on. Building that process now positions an organization to capture the cost and flexibility advantages of open-weight models with confidence, while strengthening its security posture across the models it deploys, open or closed. For most, that will mean running a mix of models rather than just one.
{{filterContent.facetedTitle}}
{{contentList.loadingText}}
{{contentList.loadingText}}
