IBM CEO Arvind Krishna says open weight AI models have become good enough, and cheap enough, to run on a company's own hardware. For hospitals, clinics, and health tech teams weighing AI budgets, that claim gives you a useful way to frame the decision: do you need the best possible model, or a very good one you control?
By the end of this piece, you'll have a short checklist for judging whether a "good enough" on-prem model fits a healthcare workflow, and a sense of what it would take to run one.
What IBM's CEO actually said
Per The Register, Krishna argues that open weight models now deliver roughly 90% of the performance of frontier models at a fraction of the token cost. His point is aimed at enterprises generally, not healthcare specifically. The Register also ties the argument to IBM's own business: infrastructure, mainframes, Red Hat, and cloud.
The source doesn't detail benchmarks or name specific models, so treat the 90% figure as Krishna's characterization, not a measured result. Any health organization would want to test on its own tasks.
Why the 90% at a fraction of the cost math matters
AI projects often stall on economics, not capability. Enterprises are under pressure to make AI financially viable, and per-token pricing adds up quickly when a tool touches thousands of documents or conversations a day.
If a cheaper model gets you most of the way there, the question changes. You stop asking "which model scores highest?" and start asking "where does the last 10% actually matter?"
In healthcare, that is a real question, not a rhetorical one. Some tasks tolerate a slightly weaker model. Others don't.
Where 'good enough' could fit in a health setting
This section is our analysis, not IBM's. Lower-stakes, high-volume work is the natural candidate for a cheaper model:
- Summarizing internal policy documents or training materials
- Drafting routine patient-facing communications for staff review
- Sorting and tagging administrative paperwork
- Helping staff search internal knowledge bases
In each case, a human reviews the output and an occasional miss is recoverable. That's the profile where paying for the top-tier model is hardest to justify.
Where you may still want the best model
Clinical decision support, anything touching diagnosis or treatment recommendations, and tasks where an error could harm a patient deserve a more careful bar. Nothing in IBM's argument says a 90% model is acceptable there. The gap between 90% and 100% is exactly where safety-critical work lives, so test rigorously, keep clinicians in the loop, and follow your regulatory requirements.
Why on-prem appeals to data-sensitive organizations
The Register's framing notes that the open-versus-closed gap is narrowing just as companies reassess cloud spending and data control. Healthcare is a sector where data control is a daily concern. Running a model on your own infrastructure means patient-related information doesn't have to leave your environment to be processed, which can simplify some privacy conversations.
It doesn't eliminate them. You still need access controls, auditing, and compliance review. But keeping data in-house is a major reason organizations consider on-prem in the first place.
What running AI on-prem demands
Cheaper tokens don't mean free. Moving in-house shifts costs and responsibilities onto you:
- Infrastructure: you need hardware and people to run and maintain it.
- Complexity: updates, monitoring, and security become your job.
- Skills: someone has to evaluate, tune, and support the model.
That is part of IBM's pitch. Per The Register, the company is positioning itself to benefit from a move toward smaller, cheaper, on-prem deployments, through its infrastructure, mainframe, and Red Hat offerings. Read the CEO's enthusiasm with that in mind: he sells the picks and shovels.
A quick checklist before you decide
- List the workflow. What exactly would the model do, and who reviews it?
- Rate the cost of an error. If a mistake is minor and caught, a cheaper model may do. If it could affect care, raise the bar.
- Test on your own data. A general claim of 90% performance tells you little about your documents and your language.
- Price the full stack. Compare token savings against hardware, staffing, and maintenance.
- Check lock-in. Open weight models can reduce dependence on a single vendor, which is part of the appeal.
- Loop in compliance early. Privacy and security review shouldn't be an afterthought.
The takeaway
Don't ask whether open weight models are as good as the frontier. Ask which of your tasks can live with 90%, and pilot a cheaper on-prem model on exactly one of them, with a human reviewing every output.
