Google launches HEIR, a compiler that runs AI on encrypted data

Google has introduced HEIR (Homomorphic Encryption Intermediate Representation), an open-source compiler and development toolchain designed to make the deployment of encrypted computation easier. The announced differentiator is the ability to take already-trained AI models, built to operate on conventional plaintext inputs, and recompile them so they operate directly on encrypted data, according to a report by InfoQ.

What homomorphic encryption is, and why it matters

Homomorphic encryption (HE) allows computations to be performed directly on encrypted data. In practice, a server can process ciphertexts and return a result that is also encrypted, without ever having access to the original information. This breaks the classic trade-off of end-to-end encryption, in which data must be decrypted to be processed.

According to Google, HE addresses several limitations of end-to-end encryption: it allows services such as spam and virus detection to operate on users' encrypted data, and it reduces the risk of exposing proprietary AI models when they are deployed on users' devices, among other scenarios.

Where Google has already applied HEIR

The company says it has used HEIR for concrete use cases:

• Private content recommendation, without exposing user data;
• Credit card fraud detection, protecting sensitive financial information;
• Network intrusion detection without revealing packet contents to the service provider;
• Hotword recognition in audio streams without exposing the recordings.

Technically, HEIR uses an intermediate representation as an abstraction layer to, in Google's words, "represent and scale complex models across diverse dialects." The promise for developers: write a Python program, annotate which data types should be encrypted, and use HEIR to compile the code for homomorphic execution.

How it works in practice (not yet a single click)

Despite the "one-click" appeal, the current process is not fully automated. Google provides demos showing how pre-trained PyTorch models can be compiled to FHE (Fully Homomorphic Encryption). The flow involves several steps, starting with using torch_mlir to export the PyTorch model to MLIR, before the compilation stages for encrypted execution.

In other words: HEIR lowers the barrier, but truly encrypted inference still requires developers to understand the compilation pipeline. The vision of "one-click capability" is the destination, not the current state.

The elephant in the room: performance

The most debated point, raised by several readers on Hacker News, is overhead. User Sabretooth1405 summed up the concern: as they understand it, HE and similar techniques have very high overheads (on the order of ~10³) for inference tasks, which makes them commercially unviable.

User dhx brought concrete numbers: a 64-bit equality operation takes 80ms, addition and subtraction take 100ms, and division takes as long as 8 seconds. These are orders of magnitude removed from conventional computing.

There is, however, a specific counterpoint for LLMs. User odo1242 pointed out that the scenario may be more favorable for language models, since their calculations rely heavily on addition and multiplication, while branching, one of the operations FHE handles worst, is not very relevant in this context. patters, meanwhile, offered another perspective: a 1000x overhead would take some image classification tasks from 1ms to 1s, which "is viable for some applications."

The discussion also touched on the broader privacy trade-off. User meindnoch argued that "the most private AI is the one running on my own hardware, not in some giant data center," and Chris2048 echoed the sentiment, suggesting that when you want to keep a query or piece of data private, the extra energy cost of running it locally might be worth it.

One important detail: although HEIR includes benchmarking code in its repository, Google has not disclosed figures on the tool's relative speed when applied to LLMs, which leaves open the practical viability for the most anticipated use case.

What changes for developers in Brazil

For Brazilian teams, the most direct angle is regulatory. The LGPD (Brazil's data protection law) imposes restrictions on the processing of personal data, and sectors such as finance and healthcare live with confidentiality requirements that often rule out the use of third-party cloud-hosted models. Homomorphic encryption is, in theory, a way to outsource AI processing without handing over data in the clear, an appealing scenario for fintechs, healthtechs, and any application that needs to process sensitive data outside its own perimeter.

In practice, however, two factors weigh against immediate adoption in the local context:

1. Computational cost. The order-of-magnitude overhead discussed above translates into more CPU time and, therefore, more infrastructure cost, something sensitive for teams already operating with tight cloud budgets.
2. Tooling maturity. Being an evolving open-source project with a still semi-manual pipeline, HEIR today serves more for experimentation and proof of concept than for critical production.

The takeaway is to keep watching closely: HEIR signals the direction of making inference over encrypted data accessible to application developers, not just cryptographers. For those dealing with regulated data, it's worth running the demos in the repository to understand the real integration effort before making any production bet.