user_x_sk, decrypts plaintext results
Outsource a confidential knowledge base to an untrusted cloud, embed inside an attested SEV-SNP CVM with an information-theoretically blind GPU, store as distance-preserving ciphertext, retrieve at near-plaintext cost.
A working retrieval-only private RAG system: a regulated enterprise outsources a document corpus to a cloud provider, queries it interactively, and gets ranked results back — without the cloud ever seeing plaintext text, plaintext embeddings, or which document matched. The prototype combines a thin client, an attested embedding TEE, an untrusted-but-blind GPU, and distance-preserving ciphertext at rest.
The hard question this is built around: can openweight transformer embedding inference be served by an untrusted cloud GPU at near-plaintext cost, without the GPU learning the input? The literature offers three families of answer — full FHE (orders of magnitude too slow), confidential-compute GPUs (10× cost premium, vendor lock-in), and TEE-on-CPU paired with masked GPU offload. This codebase ships the third.
The chosen storage path is CAPRISE-at-rest: the cloud operator sees only distance-preserving ciphertext, and the CAPRISE seed is derived inside the SEV-SNP CVM per request from a two-party HKDF (storage page). A second strategy — RemoteRAG, where the doc index is plaintext but the query gets formal (n, ε)-DistanceDP via planar-Laplace noise + Paillier rerank — was prototyped end-to-end and parked. The two are mutually exclusive at the type level. See storage · RemoteRAG for the decision trail.
| Term | Definition |
|---|---|
| TEE / CVM | Trusted Execution Environment. This codebase uses an AMD SEV-SNP Confidential VM as the trust boundary — encrypted memory plus a hardware-signed attestation report. |
| RATLS | Remote-Attested TLS. Client verifies the SEV-SNP attestation report inside the TLS handshake before any plaintext is sent. |
| CAPRISE | Distance-preserving encryption (Ye et al. 2026). Server can run cosine over ciphertext directly; the production storage path uses CAPRISE with a two-party HKDF-derived seed. |
| GELO | The per-forward orthogonal-mask split-inference protocol used by embedding / reranking / generation. Hidden states cross PCIe as U = A·H with A a fresh Haar mask sampled inside the TEE. |
| RemoteRAG | An alternative storage protocol (Cheng et al., ACL Findings 2025). Plaintext doc index plus planar-Laplace DP on the query plus Paillier rerank. Prototyped and parked — mutually exclusive with CAPRISE. |
| Compass | Zhu, Patel, Zaharia, Popa (OSDI 2025). HNSW-over-Ring-ORAM private graph retrieval — the GraphRAG layer wraps it. |
| AloePri | A static weight-obfuscation alternative to GELO (matrix-Γ obfuscated weights served via unmodified vLLM). Evaluated as a comparison generation path; no TEE in the loop. |
| HNSW | Hierarchical Navigable Small World graph — the graph-based ANN index used by every retrieval layer in this stack. |
| Vec2Text | Morris et al., EMNLP 2023. Inverts embedding vectors back to near-exact source text — the canonical motivation for keeping embeddings off the storage server. |
| T1 / T2 / T3 | Hardware deployment tiers used in §05 hardware table. T1 = in-process mock, T2 = VM-simulated CVM, T3 = real SEV-SNP silicon with VFIO GPU passthrough. |
A request traverses four trust zones, in three stages. Stage 1 · Ingest takes raw chunks from the client, embeds them inside the attested TEE (offloading the matmuls to a blind GPU under the GELO mask), derives the CAPRISE + AES keys via two-party HKDF, and writes the resulting ciphertext records to the untrusted vector DB. Stage 2 · Retrieval embeds the query through the same shape, encrypts it under the query CAPRISE coefficient, runs cosine ANN over the ciphertext on the server, then decrypts the top-k′ hits back inside the TEE. Stage 3 · Rerank derives a per-session SessionKey via HKDF, scores each (query, chunk) pair under GELO+mask, sorts in the TEE, and AES-GCM-seals the top-k final list (padded with decoys to k_max, shuffled) under a per-query key — scores never leave the CVM.
user_x_sk, decrypts plaintext results
U=A·H + public weights only
| Stage | # | Step | Who runs it | What crosses the boundary |
|---|---|---|---|---|
| 1 · Ingest | ① | Client opens RATLS, verifies attestation, posts {tenant_id, user_x_sk, chunks: [{id, text}, …]} | Client → TEE | plaintext text + 32-byte secret over RATLS |
| ② | TEE tokenizes + GELO-embeds each chunk · GPU runs the masked matmuls (Q/K/V/O + FfnUp/FfnDown) under the mask · TEE re-masks per layer · pools + L2-normalizes | TEE + GPU | masked U on PCIe outbound · U·W back | |
| ③ | TEE looks up (or creates on first contact) tee_user_x_sk[tenant]; runs HkdfPolicy::derive(user_x_sk, tee_user_x_sk, tenant_id) → (caprise_seed, aes_chunk_key) | TEE | — | |
| ④ | TEE CAPRISE-encrypts each pooled vector with the doc coefficient (3/8) + a fresh 16-byte nonce; AES-256-GCM encrypts each chunk text | TEE | — | |
| ⑤ | Storage server inserts × N records into InMemoryEncryptedIndex; the server only ever sees the ciphertext pair (EncryptedEmbedding, ChunkCiphertext) + access patterns | Storage server | encrypted records over HTTP | |
| ⑥ | TEE zeroizes caprise_seed, aes_chunk_key, and the incoming user_x_sk at end of request | TEE | — | |
| ⑦ | TEE returns 200 OK { ingested: N } over the same RATLS session | TEE → Client | response over RATLS | |
| 2 · Retrieval | ① | Client opens RATLS, verifies attestation (fresh nonce), posts {tenant_id, user_x_sk, text, top_k′, k_final, k_max} — a single request that drives Stages 2 + 3 together | Client → TEE | plaintext query + 32-byte secret over RATLS |
| ② | Same embed pipeline as Stage 1 — GELO-masked matmuls offload to GPU for one input | TEE + GPU | masked U on PCIe outbound · U·W back | |
| ③ | TEE re-derives the same (caprise_seed, aes_chunk_key) deterministically from tee_user_x_sk[tenant] ‖ user_x_sk — that's the property that lets queries decrypt what ingest encrypted | TEE | — | |
| ④ | TEE CAPRISE-encrypts the query with the query coefficient (1/8) and a fresh nonce | TEE | — | |
| ⑤ | Storage server runs cosine ANN over the ciphertext — CAPRISE is distance-preserving, so no decryption is needed. Returns top-k′ pairs (EncryptedEmbedding, ChunkCiphertext) | Storage server | encrypted query in · encrypted hits out | |
| ⑥ | Encrypted top-k′ candidates flow from the storage response back into the CVM and feed Stage 3 in the same in-CVM session; the CVM does not decrypt yet, and nothing crosses RATLS at this point | TEE (internal) | — (no boundary crossed) | |
| 3 · Rerank | ① | TEE CAPRISE-decrypts each candidate embedding and AES-decrypts its chunk text inside the CVM; decrypted bytes never leave the trust boundary | TEE | — |
| ② | For each of the k′ candidates the reranker (cross-encoder or causal-LM discriminator) runs the GELO-masked forward pass; matmuls offload to GPU under the same mask as Stage 1/2 | TEE + GPU | masked U on PCIe outbound · U·W back | |
| ③ | Scores stay inside the CVM. In-TEE sort + tie-shuffle picks top-k_final; scores then zeroized | TEE | — | |
| ④ | TEE derives QueryKey via HKDF (info "gelo-rerank.query.v1") and AES-GCM-encrypts each ranked chunk under it; the list is padded with decoys to k_max and shuffled so emission order conveys no rank | TEE | — | |
| ⑤ | Client receives the EncryptedRerankBundle as the response to the original Stage 2 request, decrypts each item with the locally-re-derived QueryKey, drops decoys, sorts by the embedded rank, and forwards the ordered plaintext to its generator | TEE → Client | k_max AES-GCM ciphertexts over RATLS |
| # | Approach-4 step | Current implementation | Status |
|---|---|---|---|
| 1 | Ingestion / chunking | Client passes raw text to TEE over RATLS; TEE chunk-encrypts AES-256-GCM with the derived aes_chunk_key | green |
| 2 | Session setup / attestation | SEV-SNP attestation via /attest; SnpAttestationVerifier chain validates ARK→ASK→VCEK; REPORT_DATA binds (model_id, scheme_id, nonce) | green |
| 3 | Embedding (TEE-side) | Primary variant chosen: full embedding model in TEE with GELO mask offloading linears to GPU. ObfuscaTune-encoder + GELO-encoder variants not built. | green |
| 4 | Storage — CAPRISE key management | Option 3 chosen: two-party HKDF (user_x_sk client + tee_user_x_sk TEE → caprise_seed_key + aes_chunk_key). In-memory persistence today; KMS-released variant deferred. | green |
| 5 | Query submission | Client POSTs (tenant_id, user_x_sk, text, top_k) over RATLS; TEE re-derives the same keys deterministically, embeds via GELO, CAPRISE-encrypts with the query coefficient (1/8) | green |
| 6 | Retrieval | Cosine over CAPRISE ciphertext (in-memory linear ANN today; HNSW swap planned at >100k docs). Distance-preserving — no server-side decryption ever. | green |
| 7 | Decryption / handoff | TEE CAPRISE-decrypts the candidate embeddings + AES-decrypts the chunk texts inside the CVM, zeroizes the derived keys, returns plaintext top-k over the same RATLS session. | green |
| 8 | Reranking / filtering | Not implemented in-TEE; client-side cross-encoder rerank is the design intent (TEE returns plaintext top-k that the client reranks locally). The TEE re-ranks Stage-1 candidates against the clean embedding only for accuracy verification in the bench. | deferred |
| 9 | Answer generation | Out of scope for the prototype — client-side LLM is the assumed architecture. ObfuscaTune / GELO+OSNIP variants for hosted generation are designed in the spec, not built. | deferred |
Each entry below is a box in the diagram. Detail pages cover the per-component method, threat model, source paper, and measured overhead.
The production storage path. Doc embeddings live on the untrusted server as distance-preserving CAPRISE ciphertext; cosine ANN runs over the ciphertext directly. The CAPRISE seed is HKDF-derived inside the SEV-SNP CVM per request from a two-party secret (Option 3) and zeroized at return.
core::caprise · distance-preserving encryptioncore::keying · HkdfPolicy · two-party HKDFgelo-rag · GeloRagTwoPartyService (runner-bound)core::storage · InMemoryEncryptedIndex (cosine over ciphertext)core::content · AES-256-GCM chunk-text cipherA second storage strategy prototyped during the work and parked. Keeps the index plaintext on the server; the query gets formal (n, ε)-DistanceDP via planar-Laplace noise, with a Paillier homomorphic dot-product reranking against the clean query. Mutually exclusive with CAPRISE — the project chose CAPRISE.
remote-rag::planar_laplace · Stage-1 query DPremote-rag::paillier · Stage-2 PHE rerank · 256-bit · CRTremote-rag::service · RemoteRagService (parallel, not wired in runner)An attested SEV-SNP CVM runs a transformer encoder; every Q/K/V/O + FFN GEMM is masked with a Haar-uniform orthogonal A per forward pass and offloaded to a commodity Vulkan GPU. The GPU sees only U=A·H plus public weights.
gelo-protocol · mask · shield · U-Verify · OutAttnMult · permuted attentiongelo-embedder · GeloBertEmbedder · GeloQwenEmbeddergelo-gpu-wgpu · WgpuVulkanEnginegelo-tee-sev-snp · SnpTrustedExecutor · SnpAttestationVerifierPost-retrieval rerank inside the same SEV-SNP CVM. Two architecture-typed services — Qwen3-Reranker-0.6B (causal-LM, primary) and bge-reranker-v2-m3 (XLM-RoBERTa cross-encoder, parity bench) — both ride the embedder's GELO mask + TwinShield primitives. Scores never leave the TEE; the wire emission is a fixed-shape, shuffled, AES-GCM bundle with rank sealed inside each ciphertext.
gelo-reranker · causal_discriminator · cross_encoder · score · sessionAutoregressive answer generation inside the same SEV-SNP CVM. Gemma 4 E2B and E4B run under the embedder's GELO mask + TwinShield primitives, with hybrid attention keeping cheap sliding-window layers fully in-TEE and only the long-range global layers crossing PCIe under mask. The Per-Layer Embedding table lives in encrypted CVM DRAM so token-id gather addresses never reach the GPU; the KV cache stays in-TEE and tokens stream back to the client under per-query AES-GCM.
gelo-embedder::decoder · gemma4 · generation · kv_cache · samplergelo-protocol::ple · PLE table + gather inside the trusted executorForward-looking. Second private-inference path for the generative LLM step, complementary to the GELO TEE-GPU stack. The Qwen3 1.7B weights are rewritten once offline into an obfuscated GGUF (covariant obfuscation, arXiv 2603.01499); stock llama-server hosts it without a fork; the trusted client wrapper holds only a secret token permutation τ and the tokenizer. Prompts cross the wire as an integer-array of obfuscated IDs — the server never reads any plaintext. Trade vs GELO: no TEE required, in exchange for static obfuscation (TTRSR ≤ 15% empirical, not information-theoretic per-batch).
obfuscate_qwen3_gguf · offline weight rewriter (Algorithm 1 keymat · §5.2.5 norm fusion · Π · αe/αh noise · inter-head shuffle)AloePriClient · trusted-side τ-map over native /completion int-array endpointShipped · benched. Private LightRAG retrieval inside the same SEV-SNP CVM. Three Compass-encrypted HNSW indexes (entities · relations · chunks) plus two XorMM volume-hiding multimaps (adjacency · src_chunks) plus an AES-GCM chunk store — every server-visible op reduces to a Ring-ORAM read, an EMM lookup, or a blob fetch. Local + Hybrid modes wired; 10–58 ms p50 in-memory at N=100–2000.
ring-oram · semi-honest baseline + lazy eviction + treetop cache · async BlockBackendcompass-index · layered HNSW · Directional Filter (~50 % read reduction)xormm-emm · volume-hiding adjacency & src_chunks · cuckoo placementcompass-rest-backend · axum + sled storage server + reqwest clientlight-kg-store + lightrag-private · LightRAG kg_query orchestratorAMD EPYC SEV-SNP for the TEE, commodity Vulkan GPU via VFIO passthrough for offload. No confidential-compute GPU needed — GELO masks make the GPU information-theoretically blind regardless.
The prototype is built for commodity hardware on both ends — EPYC SEV-SNP for the trusted side, any Vulkan-capable accelerator on the untrusted side.
Encrypted-RAM VM with hardware attestation by the AMD Secure Processor.
/dev/sev-guest · SNP_GET_EXT_REPORTvirtee/sev 7 (Apache-2.0) · DCAP byte format · ECDSA-P-384 sigCommodity Vulkan device — sees only masked activations + public weights.
wgpu 26 · Linux RADV / NVIDIA / Intel Mesaburn-cubecl 0.20.1 over cubecl-wgpu 0.9.0 · SPIR-V codegencubecl-runtime::TuneCache · disk-persistent · configured by workspace cubecl.tomlnew_fp16()| Tier | Host | Binary mode | Status | Validates |
|---|---|---|---|---|
| T1 — in-process | any x86_64 Linux | cargo test --features mock | green | protocol math · report format · parser/verifier round-trip · tamper rejection |
| T2 — VM-sim CVM | regular QEMU/KVM | SNP_MODE=mock | green | OS boundary · systemd lifecycle · weight loading · full HTTP service |
| T3 — real silicon | Hetzner EPYC + VFIO GPU | SNP_MODE=production | deferred | real /dev/sev-guest · ARK chain · RMP · SWIOTLB · GPU passthrough |