By Luigi Caramico, Founder and CTO of DataKrypto
Enterprises increasingly rely on large-scale cloud and accelerator infrastructure to run AI workloads. While encryption at rest and in transit are well understood, one part of the AI lifecycle remains comparatively underprotected: the moment data and model parameters are processed inside the accelerator. During inference and fine-tuning, sensitive inputs, intermediate activations, and proprietary weights appear in plaintext within GPU memory and orchestration layers – an exposure that existing infrastructure-level confidentiality tools do not eliminate.
This gap contributes to confusion between two related but distinct concepts: confidential computing and confidential AI. Confidential computing focuses on protecting the infrastructure boundary, such as isolating CPU enclaves and verifying platform integrity. Confidential AI, by contrast, concerns the confidentiality of the data and the model throughout the full computation pipeline, including portions executed on GPUs.
Understanding this distinction is essential for organizations that must secure both their infrastructure and their AI workloads. This blog examines how continuous encryption, implemented through FHEnom for AI, addresses this unprotected stage of computation while preserving performance and compatibility with existing AI architectures.
Summary
Confidential computing protects the infrastructure; confidential AI protects the AI itself by eliminating plaintext exposure throughout inference and training.
Threats Confidential Computing Addresses
- Cloud operator access
- Hypervisor compromise
- VM-level isolation
Threats Confidential AI Addresses
- Memory scraping inside the guest
- Rootkits in OS or drivers
- GPU VRAM exfiltration
- PCIe/interconnect snooping
- Model/weight theft
- Multi-tenant leakage
What Is Confidential Computing?
Confidential computing is a broad category of technologies designed to protect data while it is being processed. Traditionally, data security focuses on encryption at rest (on disk) and in motion (across networks). Confidential computing extends that protection to data in use through technologies such as Trusted Execution Environments (TEEs), ensuring data confidentiality and integrity even during computation, especially in potentially untrusted environments like public clouds.
Essentially, TEEs, such as Intel TDX, AMD SEV, or NVIDIA’s Confidential GPUs, create hardware-isolated enclaves where applications can run shielded from the cloud host, hypervisor, or other tenants to enable the following:
- protect workloads from cloud operators and external snooping
- keep VM memory encrypted from the outside
- provide attestation (proof that the code inside hasn’t been tampered with)
TEEs are limited in that if your own VM operating system or drivers are compromised, TEEs cannot stop attackers from accessing plaintext data in RAM or GPU VRAM.
Where Plaintext Appears in Today’s AI Systems
Even with TEEs in place, conventional AI pipelines expose sensitive information at multiple stages of computation. Once a prompt enters the application, it exists in plaintext within the guest operating system and application memory. After tokenization, inputs, embeddings, and intermediate tensors flow through GPU VRAM as unencrypted data, including attention buffers, key-value caches, activations, and other intermediate representations. Model weights, often representing valuable intellectual property, are also loaded into memory in plaintext. Any compromise of the OS, drivers, or privileged processes can result in access to this material. This unprotected path represents the portion of the AI lifecycle that confidential computing alone does not secure.
What Is Confidential AI?
Confidential AI builds on the foundation of confidential computing and takes it a few steps further. It is not just about isolating workloads—it is about cryptographically guaranteeing that AI models and data remain private, even during live inference or training.
Datakrypto’s FHEnom for AI is a confidential AI solution, combining a TEE with advanced FHE to ensure:
- Always-on encryption: Models, embeddings, and activations remain encrypted in GPU memory – no plaintext ever exists, even if the OS is breached.
- Session key isolation: Each user session has unique keys, keeping any breach contained to a single transaction.
- End-to-end lifecycle protection: From data ingestion to inference and fine-tuning, no unencrypted weights or prompts are exposed.
- Performance at scale: Encryption is GPU-native with negligible latency, enabling real-world workloads without slowdown.
Whereas confidential computing protects infrastructure, confidential AI protects the AI itself, its weights, data, prompts, and outputs.
The table below summarizes the complementary roles these two technologies play and highlights the specific gaps each one covers.
How Confidential Computing and Confidential AI Differ
| Dimension | Confidential Computing | Confidential AI |
| Primary Focus | Secures the infrastructure environment (VMs, enclaves, host isolation). | Secures the AI workload itself (models, data, tensors, GPU memory). |
| Plaintext Exposure | Plaintext still appears inside the guest OS and GPU VRAM once computation begins. | Eliminates plaintext exposure by encrypting model weights, embeddings, activations, and outputs during computation. |
| Attack Surface Addressed | Protects against cloud operators, hypervisors, and cross-tenant VM attacks. | Protects against OS compromises, driver-level attacks, memory scraping, and model-extraction vectors. |
| Trust Assumptions | Trust CPU vendor + guest OS + runtime. | Trust CPU enclave only; GPU, OS, and drivers remain untrusted. |
| Scope of Protection | Environment-level isolation and attestation. | End-to-end protection of the AI lifecycle—ingestion, inference, training. |
| Regulatory Alignment | Meets baseline “data at rest/in transit” requirements but leaves data-in-use exposure. | Aligns with GDPR/HIPAA/CCPA expectations for protecting data during processing. |
| Business Impact | Reduces risk from cloud infrastructure breaches. | Prevents model/IP theft, prompt leakage, and data exposure—direct enterprise value risks. |
| Where Protection Stops | Stops at VM boundary; GPU workloads run in plaintext | Protects through GPU computation; no plaintext inside guest memory or VRAM |
Why You Need Both
Confidential computing and confidential AI solve different parts of the confidentiality problem, and most enterprise environments require both to achieve complete protection.
Confidential computing secures the infrastructure layer. It ensures that workloads running in outsourced or multi-tenant clouds remain isolated from the host, hypervisor, and other tenants. These protections help organizations meet regulatory obligations, such as HIPAA, CCPA, and GLBA, that require demonstrating control over where sensitive data resides and who can access it at the infrastructure level.
Confidential AI secures the AI workload itself. It protects model weights, prompts, embeddings, activations, and outputs throughout the full inference and training lifecycle, whether the model is open source or proprietary. By eliminating plaintext exposure inside the guest OS and GPU memory, confidential AI closes the remaining gap left by confidential computing and reduces the risks of model theft, data leakage, and cross-tenant exposure.
Together, these technologies provide end-to-end confidentiality: the infrastructure is protected from the outside in, and the AI workflow is protected from the inside out. This combination allows enterprises to deploy advanced AI systems with stronger assurance of privacy, compliance, and resilience, without limiting performance or architectural flexibility.
`