There is a persistent trade-off in cloud AI: to get an answer, you have to hand over the question. Send a medical record, a contract or a customer database to a model, and that data exists in readable form on someone else's servers, at least momentarily. Google has released an open-source project aimed squarely at that problem.

The tool is HEIR — Homomorphic Encryption Intermediate Representation — a compiler toolchain that converts trained models so they can run inference on encrypted inputs. The server computes an answer without ever seeing the underlying data.

What homomorphic encryption actually does

Normal encryption protects data in two states: while it is stored, and while it travels. To actually compute on it, something has to decrypt it first. That decryption moment is the gap.

Homomorphic encryption closes the gap by allowing mathematical operations to be performed directly on encrypted values. The server manipulates ciphertext, produces an encrypted result, and returns it. Only the person holding the key can decrypt the answer. The server has done useful work while remaining, in a meaningful sense, blind to what it processed.

A rough analogy: imagine handing someone a locked box with gloves built into the sides. They can rearrange the contents without ever opening the box or seeing what is inside.

Why this has stayed theoretical for so long

Homomorphic encryption is not new — it has been an active research field for well over a decade. The obstacle has always been cost. Computing on encrypted data has historically been orders of magnitude slower than computing on plain data, which made it impractical for anything as demanding as running a neural network.

The second obstacle is expertise. Writing homomorphic encryption code correctly has required specialist cryptographic knowledge that few engineering teams have. A compiler toolchain attacks exactly this problem: it lets developers work with models in familiar terms and handles the translation into encrypted operations.

That is the significance of HEIR being a compiler rather than a library. It is infrastructure aimed at making the technique usable by ordinary engineers.

Where it would matter most

The use cases are the ones currently blocked by data-sensitivity rules:

  • Healthcare — running diagnostic models on patient data without exposing records to a cloud provider.
  • Financial services — fraud analysis on transaction data under strict confidentiality rules.
  • Legal work — analysing privileged documents without waiving confidentiality.
  • Cross-organisation analysis — parties collaborating on combined data without revealing their own inputs.

These are precisely the sectors that currently choose self-hosting for regulatory reasons — one of the main arguments in favour of open-weight models running on your own infrastructure. If encrypted inference becomes practical, that calculation changes.

The honest caveats

Several things are worth stating clearly:

  • Performance remains the open question. A toolchain that makes encrypted inference possible is not the same as making it fast enough. Real-world overhead will determine whether this is adopted or admired.
  • Not every model will convert well. Some operations are far more expensive under encryption than others.
  • It protects inputs, not everything. Homomorphic encryption addresses data confidentiality; it does not fix model bias, hallucination, or misuse.
  • Early days. This is a released toolchain, not a proven production pattern with years of deployment behind it.

Why it matters

Most privacy improvements in AI have been policy-based: promises about retention, contractual commitments, access controls. Those depend on the provider behaving well and being competent — an assumption that fails regularly, as this week's exposure of 181,874 meeting records illustrates.

Cryptographic guarantees are a different category. If a server mathematically cannot read your data, no policy failure, rogue employee or breach on their side exposes it. That is a structurally stronger promise than any privacy statement, which is why the approach is worth taking seriously even while the performance question is unresolved.

Key takeaways

  • Google released HEIR, an open-source compiler toolchain for running AI inference on encrypted data.
  • Homomorphic encryption allows computation on ciphertext, so servers never see the underlying inputs.
  • The historic barriers have been severe performance cost and the need for specialist cryptographic expertise.
  • Healthcare, finance, legal and cross-organisation analysis are the clearest beneficiaries.
  • Whether encrypted inference is fast enough for production use remains the decisive unanswered question.

The bottom line

Encrypted inference would replace "we promise not to look at your data" with "we are unable to." That is a meaningfully better guarantee. Whether it arrives in practice depends on performance numbers we do not have yet — but making the technique accessible to ordinary developers is the necessary first step.