An attested SEV-SNP CVM runs the embedding model. Every Q/K/V/O + FFN matmul rides through a Haar-uniform mask before crossing PCIe to a commodity Vulkan GPU. The GPU is information-theoretically blind.
protocol GELO + TwinShield
TEE AMD SEV-SNP CVM
GPU commodity Vulkan via VFIO
overhead ~1.27× plain · Qwen3-Emb-0.6B
read across the matmul seam
§01
Role in the project
where this fits
The embedding layer is the most computationally heavy step in private RAG, and historically the hardest to keep confidential. Full-FHE inference is 1000× too slow; confidential-compute GPUs (H100-CC) carry a 10× cost premium and lock the deployment to one vendor. GELO + SEV-SNP hits a third point on the curve: commodity Vulkan GPU, ~6% protocol overhead, hardware trust anchored in AMD's Secure Processor.
The embedder turns user text into a unit-norm vector. The vector then feeds the storage layer (CAPRISE-at-rest or RemoteRAG, see Storage). At no point does the GPU, the cloud operator, or the storage server see plaintext text or plaintext activations — they see only attested code identity, masked activations U=A·H, and (publicly known) model weights.
What makes this tractable is the openweight assumption: the user pulls a specific public embedding model (e.g. Qwen/Qwen3-Embedding-0.6B) by SHA-pinned revision, and the attestation report binds the running CVM to those specific bytes. There is no weight secret to protect — only activation confidentiality flowing through public weights. That single assumption is what lets the GPU stay commodity.
§02
Definitions & glossary
protocol-side terms used on this page
Term
Definition
GELO
The per-forward orthogonal-mask split-inference protocol used by this codebase. Hidden states H cross PCIe as U = A·H where A = Σ·Q is a fresh Haar-uniform orthogonal matrix (with a positive-diagonal sign-shield Σ) sampled inside the TEE. Mask material exists only for one forward pass.
Mask A
The per-forward orthogonal matrix that crosses with each offloaded activation. Cancels at the next non-linearity by re-mixing inside the TEE (A⁻¹ · masked_result).
Shield Σ
Sign-shield: a diagonal of ±1 entries with magnitude tuned so the mask preserves the activation's typical norm. The shield(8, 4.0) default uses 8-block scale with 4.0 σ.
TwinShield
Xue et al. 2025. Pair-of-projections defence the embedder validated alongside GELO; cited as the security baseline the protocol composes with.
U-Verify
Freivalds-style randomised matmul integrity check. Issues k probes against the GPU's masked output to detect malicious matmul tampering.
HD₃ mask
Optional Hadamard-cascade mask A = D₃·H·D₂·H·D₁·H (QuIP# / QuaRot family). Opt-in via with_hd3_mask(); reduces CPU mask-GEMM cost at power-of-two dimensions.
CAPRISE
Distance-preserving encryption for the at-rest path — see storage. The embedder hands off the pooled vector to CAPRISE before it leaves the TEE.
SEV-SNP / CVM
AMD Secure Encrypted Virtualization · Secure Nested Paging. The confidential VM technology this codebase targets for the TEE half.
RATLS
Remote-Attested TLS. The TLS handshake validates the SEV-SNP attestation report before any plaintext is sent.
BEIR
Heterogeneous IR benchmark (Thakur et al. 2021). Used here for end-to-end retrieval-accuracy parity checks against unmasked baselines.
§03
Threat model
what each component sees · what it cannot
The trust posture is: hardware-attested TEE on CPU, information-theoretically blind GPU, everything else hostile.
Component
Trust
What it sees
What it does NOT see
User-side text
confidential
—
—
TEE (SEV-SNP CVM)
trusted
text · activations · mask state · model weights · pooled embeddings
—
GPU + driver + PCIe
untrusted
public model weights · per-batch masked activations U=A·H · integrity-probed matmul results
clean activations H · mask A · user text · pooled embedding
Two recent attacks (2025–2026) target schemes structurally related to GELO. Neither applies here — GELO's per-batch full-rank Haar sampling is exactly the architectural choice that closes both.
Attack
Reference
Why GELO survives
Precomputed-basis recovery
arXiv 2602.11088 (Wang et al.)
Recovers LLaMA-3 8B layer secrets from SOTER / TSQP / TransLinkGuard in ~6 min by exploiting a static low-dimensional basis. GELO's A is sampled per batch via Householder QR from a fresh ChaCha20-seeded Gaussian — Haar-uniform over the full orthogonal group O(n). No static basis exists.
Sequential vocab matching against fixed permutations
arXiv 2505.18332 "Hidden No More" (Wang et al., ICML 2025)
99%+ recovery from PermLLM / STIP / Centaur by exploiting fixed permutations. GELO's A is not a permutation (it's a full-rank rotation) and is fresh per batch. Two independent reasons the attack misses by construction.
ArrowMatch (weight-mixed masks)
Xue et al. 2025 background
Targets schemes that mix the mask into W (STIP, ObfuscaTune). GELO masks the activation axis only; A never touches W. Inapplicable by construction.
Not covered. Side channels in the TEE itself (cache, timing, power), one-time weight upload to GPU (cleartext at startup), workload-shape inference from dispatch pattern, simultaneous SEV-SNP hardware vulnerability — all out of scope for the protocol layer.
§04
Supported model architectures
two encoder shells under one TrustedExecutor
Every architecture below loads as Arc<Weights> from safetensors via the HuggingFace Hub. The SHA-256 of the manifest bytes is the model_identity that rides through every attestation report — relying parties pin specific weights, not just the architecture.
Family
Crate path
Reference model
Layers · Hidden · Inter
Distinguishing ops
Pooling
BERT encoder
gelo_embedder::bert
BAAI/bge-base-en-v1.5
12 · 768 · 3072
LayerNorm · GELU FFN · multi-head attn (no causal mask)
Offloaded (masked): Q · K · V · O · FfnGate · FfnUp · FfnDown — 7 GEMMs per layer × 28 layers = 196 offloaded linears per text. Gate+Up bundled via offload_linear_many behind one mask apply.
Long-context lever:OutAttnMult moves Q·Kᵀ to GPU under TwinShield's 4-partition embedding; auto-switches off at embedding shape (n ≪ hidden) per measured FLOP balance
§05
Components
GELO-side · 11 implementation units
Each card pairs [method ↦ source ↦ what it secures ↦ threat ↦ overhead]. Ordered by request path: weight load → mask → offload → attention → hardening → integrity → pooling → attestation.
protocol kernel
GELO maskgelo_protocol :: mask
Per-batch Haar-uniform orthogonal A, applied on the token axis. U=A·H ships to the GPU, H·W = Aᵀ·(U·W) recovers on return — bit-exact in IEEE-754 modulo cancellation.
method
Householder QR over a fresh ChaCha20Rng Gaussian seed. AOCL-BLIS-backed cblas_sgemm in the apply/unapply hot path under the blas feature.
source
Belikov & Fedotov · arXiv 2603.05035
secures
activations crossing PCIe to untrusted GPU
threat
GPU is Haar-uniform-blind; survives 2026 precomputed-basis and Hidden-No-More attacks by construction
cost
≈ 100 ms apply + 173 ms unapply per Qwen3 forward across 112/196 calls; AOCL-BLIS saves 5× vs vanilla BLIS
hardening
Shield rowsgelo_protocol :: shield
k random high-energy rows spliced into H before masking and stripped after recovery — turns the multi-batch Gram leak U⊤U = H⊤H into H⊤H + S⊤S.
method
k=8 rows at energy ≈ 4·mean‖h‖ (production paper-parity). Stack into scratch · mask · recover · slice off.
source
TwinShield — Xue et al. 2025 §V-B
secures
cross-batch Gram leak that would otherwise enable a FastICA recovery
threat
persistent GPU-side observer collecting many U_i. tests/bss_recovery.rs asserts recovery succeeds without shield + fails with.
cost
+28 ms stack + 14 ms strip / forward, amortized across 112 calls
integrity
U-Verifygelo_protocol :: integrity
Freivalds-style integrity probe: for asserted Z = A·B, check A·(B·r) ≈ Z·r against random r. Undetected tamper rate ≤ (2L)⁻ᵏ.
method
r ∈ {-L..L}ᵖ, k probes. Ships L=3; deployment picks k. Weight cache Arc-shared with embedder (saves 2.4 GB on Qwen3).
integrity of every offloaded GEMM — actively-malicious GPU returning crafted garbage is rejected
threat
actively-malicious untrusted GPU
cost
k=2 → ≈ 2.8% miss · k=8 → 2.4·10⁻⁷ miss. Disabled by default in throughput bench (k=0).
attention lever
OutAttnMultgelo_protocol :: out_attn_mult
4-partition embedding for Q·Kᵀ where both operands are runtime values. GPU sees a (2n,2n) masked permuted matmul without the recovery scalars.
method
Mask Q + R_Q, Kᵀ + R_Kt with fresh matrices · scale by random scalars a, b · stack into (2n, d)·(d, 2n) · permute · TEE recovers four partitions
source
TwinShield — Xue et al. 2025 §V-A
secures
the one matmul GELO doesn't cover (no public weight)
threat
same as GELO: GPU sees Q/K only in the 4-partition form without the secret scalars
cost
+24% wall forced on at embedding shape (regime mismatch); net win at long context. Auto-switches off when n < hidden.
research lever
Permuted attentiongelo_protocol :: attention
Amulet-inspired softmax-equivariance: softmax(πQKᵀπᵀ/√d)·πV = π·softmax(QKᵀ/√d)·V. Fresh row-permutation lets softmax + both attn matmuls run on the GPU under obfuscation.
method
Per-batch π ∈ S_n · transformed causal mask · optional σ ≈ 0.01 Gaussian noise (Hidden No More mitigation)
source
Amulet — Wang et al. arXiv 2512.07495
secures
the in-TEE-attention bottleneck (28% of Qwen3 per-text wall at n≈400)
threat
survives Hidden-No-More only when paired with σ-noise + shield rows
cost
regresses at embedding shape (84 extra dispatches/text on iGPU). Defaults off; opt-in via BEIR_PERM_ATTN=1. Breakeven n ≈ 1000–2000.
offload engine
WgpuVulkanEnginegelo_gpu_wgpu
The untrusted side of the GELO protocol. burn-cubecl over wgpu → Vulkan on Linux, Metal on macOS, DX12 on Windows. Vendor-agnostic; consumer-GPU-friendly.
heavy GEMMs (Q/K/V/O/Up/Gate/Down) — sees masked activations only
threat
information-theoretically blind. ArrowMatch-class attacks don't apply: A is on the activation axis, never mixed with W
cost
≈ 247 ms / text of GPU GEMM on Qwen3 at n≈400 · AMD Strix Halo iGPU (RADV)
BERT-class embedder
GeloBertEmbeddergelo_embedder :: bert
12-layer BERT encoder driven through a TrustedExecutor. Q/K/V/O + FFN GEMMs route through the protocol; embedding lookup, LayerNorm, GELU, residual, mean-L2 pooling stay TEE-side.
method
tokenizers 0.21 · weights as Arc<BertWeights> · provision_weight_shared Arc-shares with U-Verify cache
source
in-house · defaults to BAAI/bge-base-en-v1.5 via hf-hub
secures
protocol fidelity vs the plain reference — bit-exact in fp32 modulo float associativity
threat
see GELO / Shield / U-Verify cards
cost
~6× faster per text than Qwen3 (12 vs 28 layers, 768 vs 1024 hidden)
decoder-LLM embedder
GeloQwenEmbeddergelo_embedder :: decoder
Decoder-LLM-as-embedder driven through a TrustedExecutor. 28-layer transformer with RoPE, GQA, SwiGLU FFN, RMSNorm, last-token + L2 pooling.
method
Same offload schedule as BERT plus SwiGLU's gate/up/down. offload_linear_many bundles (gate, up) behind one mask apply. Causal mask handled TEE-side; in-TEE GQA attention by default at embedding shape.
source
in-house · defaults to Qwen/Qwen3-Embedding-0.6B
secures
protocol fidelity · attestation binding
threat
see GELO / Shield / U-Verify cards
cost
153 ms / text masked · parallel-rayon · paper-parity · AOCL-BLIS · iGPU · 1.27× plain
attestation
SEV-SNP attestationgelo_tee_sev_snp
Production CVM evidence path. 1,184-byte SEV-SNP attestation report with REPORT_DATA binding (model_identity, scheme_identity, nonce), signed by the per-chip VCEK in the AMD-SP.
method
HardwareReportIssuer opens /dev/sev-guest · SNP_GET_EXT_REPORT. MockReportIssuer signs against a bundled ARK→ASK→VCEK PKI. Verifier validates ECDSA-P-384 chain + recomputes REPORT_DATA.
source
virtee/sev 7 (AMD-blessed)
secures
relying-party trust — CVM runs the expected weights with the expected protocol scheme
threat
cloud operator running a different binary, swapping weights, or attempting downgrade
cost
0.39 ms issue · 2.70 ms verify (mock chain) · once per session · never on per-text path
test substrate
In-process executorsgelo_protocol :: sim
Reference executors: InProcessTrustedExecutor drives the full protocol on the same machine; PlaintextExecutor is the parity baseline; RayonCpuEngine is the CPU offload backend.
method
Identical protocol math to the SEV-SNP executor — the SEV-SNP wrapper forwards every method. Used by the BEIR bench and unit tests.
source
in-house
secures
protocol math correctness; throughput floor without TEE memory encryption overhead
threat
n/a — test substrate
cost
see §06 numbers
§06
Compute flow & trust boundaries
per-layer · TEE ⇄ GPU mask round-trip
Inside one decoder block, the trusted side runs everything except 7 linear projections. Each projection is masked, shipped, multiplied, and unmasked. Below: one Qwen3 block in detail.
Why this split is fast
GELO is a FLOP-asymmetric split: the heavy compute goes to the GPU, the cheap compute stays in the TEE — and at embedding shape the gap is roughly two orders of magnitude per layer.
Per layer, attention runs as O(n²·d) (heads × sequence-pairs × head_dim) and one linear projection runs as O(n·d²) (tokens × hidden × hidden). One linear projection's work matches all of attention only when n ≈ d; for any n < d, linears dominate. Embedding inputs sit at n ≈ 50–400 tokens against d = 768 (BGE) or d = 1024 (Qwen3) — well into the regime where the projection bucket is ~10–20× attention.
How GELO exploits it
The cheap O(n²·d) op (attention, runtime-runtime · no public weight) stays in the TEE — small absolute cost at embedding n.
The expensive O(n·d²) ops (Q/K/V/O/Gate/Up/Down — 7 GEMMs per layer with public weights) go to the GPU under the GELO mask.
Mask apply / unapply is O(n²·d) too, the same complexity class as in-TEE attention — bounded by the cheap side, never by the expensive side.
Net result on Qwen3 at n ≈ 400, d = 1024: the GPU absorbs ~247 ms of GEMM that would dominate any pure-TEE path, while the in-TEE attention bucket is only ~238 ms across all 28 layers (~0.3 ms / call amortised). The mask machinery rides along at the cheap-side complexity, never the expensive one. This is the structural reason GELO + commodity GPU lands at 1.27× plain instead of 5–10×.
What the TEE actually does
More operation types than the GPU, but each individually cheap (FLOP-asymmetric, see left). Every non-linear op lives here — the GPU only ever holds the obfuscated state and cannot run an op that needs the cleartext.
Norms — LayerNorm (BERT) or RMSNorm (Qwen3). Need cleartext, run per-row.
Attention — Q·Kᵀ, softmax, ·V. Runtime-runtime · no public weight to mask against · stays in TEE at embedding shape.
Positional — RoPE on Q, K (Qwen3 only).
Activations — GELU (BERT) or SwiGLU = silu(gate)⊙up (Qwen3).
Residuals + biases — element-wise adds on cleartext.
Mask machinery — sample A · apply U = A·H · unapply Aᵀ·(U·W) · shield-row pack/strip. BLIS-direct CBLAS at the GEMM hot path.
Pool + L2 normalize — mean (BERT) or last-token (Qwen3), then L2.
What the GPU does
A small, sharp set of GEMMs against publicly-known weights. Sees only masked activations + the model bytes everyone already has.
matmul on a registered weight — Q · K · V · O · FfnUp · FfnDown (BERT, 6/layer) or +FfnGate (Qwen3, 7/layer).
matmul_many — bundles GEMMs that share an input (QKV; gate+up for Qwen3) so one upload + one sync covers several products.
matmul_dynamic + softmax_batched — only when OutAttnMult / permuted-attention engages; at embedding shape, off.
Per Qwen3 forward at n ≈ 400: ~247 ms GPU GEMM (the heavy bucket) vs ~238 ms in-TEE attention spread across 28 layers (~0.3 ms / call). Op-count and wall-clock pull in opposite directions — that's the whole point of the split.
Two figures below trace one layer-block. FIG. 02a shows a BERT-class encoder block (BGE-base; 12 layers); FIG. 02b shows a decoder-LLM block (Qwen3-Embedding; 28 layers). Same 4-offload-group rhythm in both — the differences live in norm placement, activation choice, and whether attention is causal + RoPE'd. Solid red = data on cleartext; dashed amber = masked transit across PCIe.
FIG. 02b — One Qwen3 decoder block · pre-LN · SwiGLU · causal GQA + RoPE · 28 layersnorm moves inside the residual loop (pre-LN) · FFN forks into parallel gate/up branches that merge through SiLU ⊙
What to look for between the two figures:
① norm placement — BERT puts LayerNorm after the residual add (post-LN); Qwen3 puts RMSNorm before each sub-block, inside the residual loop (pre-LN).
② positional — Qwen3 has a RoPE step between QKV and attention; BERT has none.
③ attention — Qwen3 is causal + GQA (16 Q-heads, 4 KV-heads); BERT is bidirectional multi-head.
④ FFN topology — BERT is a linear chain (Up → GELU → Down); Qwen3 is a forked SwiGLU (Gate ∥ Up → SiLU ⊙ → Down) where the two GEMMs share one masked input and travel to the GPU as a single matmul_many dispatch.
Op
Where
Sees
Qwen3 fwd
BERT fwd
LayerNorm / RMSNorm
TEE
cleartext H
~10.9 ms (57 calls)
~25 calls
QKV matmul (bundled)
GPU under mask
U · public W
~142 ms (matmul_many)
12 dispatches
Attention (Q·Kᵀ · softmax · ·V)
TEE · cleartext both sides
cleartext Q, K, V
~238 ms (28 layers)
~5× less
RoPE
TEE
cleartext Q, K
~2.1 ms (28 calls)
—
O matmul
GPU under mask
U · W_O
part of ~105 ms single-matmul bucket
12 dispatches
Activation (SwiGLU / GELU)
TEE
cleartext gate, up
~25.5 ms (SwiGLU)
smaller (GELU is 1-arg)
FFN matmuls
GPU under mask
U · public W
part of GPU bucket
12 × 2 dispatches
Mask apply / unapply
TEE · BLIS-direct
cleartext both sides
~100 + 173 ms (308 GEMMs)
~5× less (132 GEMMs)
Mask sample (paper-parity)
TEE
fresh Haar A
~8.8 ms (once / fwd)
~8.8 ms
Pool + L2 normalize
TEE
cleartext final H
µs (last-token + L2)
µs (mean + L2)
§08
Performance & correctness
measured · AMD Ryzen AI Max+ 395 · 2026-05-14
All numbers from crates/gelo-rag/tests/beir_accuracy.rs on AMD Ryzen AI Max+ 395 with the integrated Vulkan GPU. Paper-parity mode (one Haar A per forward + 8 shield rows), --features blas via AOCL-BLIS, rayon parallel fan-out across texts.