generation
— AloePri

Generative inference without a TEE in the loop. Qwen3 1.7B is rewritten once, offline, into an obfuscated GGUF that an unmodified llama-server hosts. The trusted endpoint holds only a secret token permutation τ and the tokenizer. Every prompt crosses the wire as a sequence of obfuscated integer IDs whose detokenisation is gibberish in the server's vocabulary; the server runs a normal forward pass over the rewritten weights and returns obfuscated output IDs.

protocol  AloePri covariant obfuscation (arXiv 2603.01499)
trust anchor  client process (no TEE required)
serving stack  llama.cpp / llama-server — stock, no fork
current artifacts  Qwen3 4B + 8B (bf16; 9.1 + 17.4 GB)
best HumanEval  8B αe=1.0 — 8/20 = 40 %
IO token permutation · embedding/attention noise
paper TTRSR ceiling
<15%
VMA · IA · IMA · ISA on Qwen2.5-14B at αe=1.0, αh=0.2 (paper Table 2; not yet measured on our Qwen3 1.7B artifact)
observed Δ accuracy
+0.5 / −4 / −6pp
MMLU / PIQA / HumanEval at n=200 / 200 / 50; SE 3–7pp; IFEval deferred
internal-dim expansion
d→ d+2h
d=2048 → 2304 (h=128, +12.5%) · stock llama-server reads the new qwen3.embedding_length from GGUF metadata, no fork required
production format
bf16default
bf16 since 2026-05-20 — matches fp32 accuracy at ½ size + 2× tps. fp16 unsafe (denormal flush); Q8_0/Q6_K/Q5_K_M still collapse (heavy-tailed AloePri weights)
§01

Role in the project

Qwen3 1.7B + stock llama-server · where this fits

Generation is the post-retrieval LLM step in private RAG. After the reranker emits its k_max-item AES-GCM bundle, the client decrypts the top-k_final chunks and assembles a prompt. AloePri is the route that lets that prompt be processed by an LLM running on commodity hardware without a TEE.

What this complements. Embedding, storage, and reranking all run inside a SEV-SNP CVM with a GELO-masked GPU offload (embedding, storage, reranking). Generation has a different cost shape: the model is 5–50× larger than the embedder, and decode is autoregressive, so per-token mask resampling and per-offload PCIe round-trips dominate wall-clock. AloePri sidesteps the cost curve by moving the entire forward pass to an untrusted server and shipping a one-time obfuscated artifact instead.

Why Qwen3 1.7B as the demonstrator model. Dense transformer with GQA(16, 8), 28 layers, hidden 2048, head_dim 128, RoPE base 1M, vocab 151 936. Open weights, small enough to iterate quickly, large enough to give meaningful accuracy numbers on MMLU / PIQA / HumanEval. It is also the same backbone used as the GELO-LLM demonstrator on the GELO route, so accuracy deltas between the two private-inference routes (GELO vs AloePri) are directly comparable.

Why llama.cpp as the serving stack. Pure C++ binary; no Python serving stack; CPU and Vulkan-iGPU paths both supported; OpenAI-compatible HTTP API plus a native /completion endpoint that accepts integer token-ID arrays directly. llama-server reads qwen3.embedding_length from the GGUF metadata, so AloePri's expanded internal dim d + 2h is transparent. No fork required for the obfuscation steps currently deployed (token-level permutation Π, embedding/head Gaussian noise, and the inter-head attention shuffle from paper §5.2.3).

What's traded. AloePri does not give the per-batch information-theoretic argument GELO provides. Its security is empirical: a static obfuscation, sized so published attacks (VMA, IA, ISA, IMA, NN, TFMA, SDA) recover under 15% of tokens at the paper's recommended hyperparameters. The deliberate trade is: accept a weaker but still-meaningful guarantee in exchange for compatibility with unmodified llama-server, fp32-only weight storage (8.6 GB), and zero protocol-aware code on the server side.

What's TODO. The downstream integration story — which retrieval shape, which prompt template, which token budget, which deployment topology the generation step sits in — is left as a stub. The protocol surface this page documents is the same regardless of how the surrounding RAG pipeline calls into it.

§02

Definitions & glossary

terms specific to this page
Term Meaning
AloePri Lin et al., Towards Privacy-Preserving LLM Inference via Covariant Obfuscation, arXiv 2603.01499 (ByteDance / Nanjing University). The protocol this page implements.
Covariant obfuscation Joint transform on input data and model weights such that the obfuscated model on obfuscated input produces obfuscated output equivalent to the plaintext pipeline. The "covariant" property is what makes the rewrite lossless: model accuracy is preserved exactly (up to noise terms αe·ε, αh·ε added for VMA defense).
τ (tau) Secret token-level permutation sampled from Sn, where n is the tokenizer's active vocabulary size (151 669 for Qwen3 1.7B; the GGUF's full 151 936-slot table has 267 padding slots that stay identity). Drawn once per deployment; held by the client; never leaves the trusted zone.
Π (Pi) Permutation matrix corresponding to τ. Applied to the embedding (Π·Wembed*) and model head (Whead*·ΠT) during offline rewrite.
Z Vocabulary mapping Z = { V[i] ↦ V[τ(i)] }. Used online to obfuscate input token IDs and de-obfuscate response token IDs.
P̂, Q̂ (key matrices) Generated by paper Algorithm 1 from secret base matrices (B, B-1, E, F, Z). Satisfy P̂ · Q̂ = Id. P̂ shape is d × (d+2h); Q̂ is (d+2h) × d. Adjacent layers' weights get right-multiplied by some P̂ and left-multiplied by the matching Q̂ so the cancellation runs through every forward pass.
Expansion size h Algorithm 1 hyperparameter. Internal hidden dim grows from d to d+2h inside each transformer block; residual-stream dim stays at d. Default h=128. Paper Figure 5 shows TPOT impact <10% out to h=512 on R1-Distill-14B.
R̂qk, Ĥqk, Ẑblock Paper Algorithm 2 intra-head transforms applied to Q/K weights: 2D rotary obfuscation, 2-block scaling, and RoPE-block-wise permutation respectively. Deployed on Qwen3 since 2026-05-19 via the matrix-Γ kernel extension — instead of folding γqk backward via §5.2.5 (which the §05 deployment couldn't because Qwen3 trained Q/K is non-Gaussian), we replace the per-element γqk tensor with a per-layer 2D Γ = MqT·Diag(γqk)·Mq and let the kernel apply it via ggml_mul_mat at the q_norm site. Storage cost +42 MB / fp32 model; compute cost <1 % at decode. R̂qk is the RoPE-aware rotation per NEOX pair; Ĥqk is ±1 Walsh-Hadamard (paired across the NEOX halves so the rotation block-form is preserved); Ẑblock is a β-wide locality-preserving permutation within RoPE-frequency bands. See §08 / §09 for measured impact.
Ûvo Random invertible matrix applied to V (left of Wv) and inverted on O (right of Wo). Stops V activations from being a clean intermediate the server can attack.
τkv, τgroup Inter-head permutations from Algorithm 2 §5.2.3. τkv shuffles K/V heads; τgroup shuffles Q/O heads within each GQA group. Both are part of the same offline rewrite, with no per-request cost.
αe, αh Gaussian noise coefficients applied to embedding and head weights: W̃embed = Π·(We + αe·ε)·P̂embed. Default αe=1.0, αh=0.2. Paper §7.3: at αe=0.5, VMA jumps to >30% recovery — these knobs matter.
QK-norm Per-head RMSNorm applied to Q and K vectors between the QKV projection and RoPE in Qwen3 (attn_q_norm.weight / attn_k_norm.weight, shape [head_dim=128], broadcast across heads). A training-stability fix borrowed from work like Chameleon / OLMo-2. Architecturally new in Qwen3 relative to the AloePri paper's evaluated baselines (Qwen2.5, Llama3, DeepSeek have no QK-norm), and the cause of the item-7 restriction documented in §09.
§5.2.5 fusion Paper §5.2.5 layer-normalisation transformation. Replaces RMSNorm(γ_vec) with RMSNorm(κ·I) after folding diag(γ_vec) into the adjacent linear weight. Exact in expectation under i.i.d. Gaussian-input assumption; per-input bias is bounded by the paper's eCnorm. Used for residual-stream norms (attn_norm, ffn_norm, output_norm). Does not work for Qwen3's QK-norm — see §09.
head-shuffle The subset of paper Algorithm 2 that survives Qwen3 QK-norm: per-layer τkv + τgroup applied as a feature-axis permutation on Wq / Wk / Wv rows and Wo columns. Permutes whole heads, leaving each head's internal layout (and the per-head_dim γ_qk broadcast) unchanged. Mathematically clean; no commutativity argument needed.
Offline rewrite One-shot client-side transform of plaintext weights into the obfuscated GGUF. Reads the source GGUF (Q8_0 quantised), dequantises to fp32, samples τ + key matrices + noise from seeds, applies §5.2.5 fusion of residual norms + Algorithm 1 keymat + Π + head-shuffle, writes fp32 GGUF + a .key.npz sidecar holding τ. Per-model, not per-request.
TTRSR Text Token Recovery Success Ratio. Fraction of original tokens an attacker recovers under a given attack. Lower = better privacy. Paper headline: TTRSR <15% on the broadest attack suite at recommended hyperparameters.
VMA / IA / ISA / IMA / NN / TFMA / SDA The seven attacks evaluated against AloePri. VMA / IA exploit plaintext-vs-obfuscated weight relationships; ISA / IMA / NN train inverters from observed internal states; TFMA / SDA exploit token-frequency patterns. The reference codebase ships them at src/security_qwen/.
RmDP Rényi-metric Differential Privacy. Paper §6 definition: combines α-Rényi divergence with a metric on the token-sequence space. The framework used to prove AloePri's noise budget is strictly smaller than a data-only mechanism's at equal accuracy.
§03

Threat model

what each component sees · what it cannot

The trust split here is fundamentally different from the GELO embedder / reranker. There is no hardware-rooted trust anchor in the loop: protection is structural (obfuscation hides the prompt) plus empirical (noise + key matrices defeat published recovery attacks). The trusted endpoint is just the client process that holds τ.

Component Trust What it sees What it does NOT see
User-side text confidential — —
Client wrapper
(trusted zone)		 trusted plaintext prompt · plaintext response · τ · tokenizer · seeds used at offline rewrite —
Network (TLS) untrusted HTTP request body: {"prompt": [obf_id_1, obf_id_2, …], …} · HTTP response body: {"content": "…", "tokens": [obf_resp_id_1, …]} · request timing · request count τ · plaintext prompt · plaintext response
llama-server untrusted obfuscated GGUF weights θ̃ · obfuscated input token IDs (as received from client; no server-side re-tokenisation needed) · all internal activations · KV cache · obfuscated logits · sampled obfuscated output IDs τ · plaintext prompt or response · plaintext weights · the noise matrices ε used at offline rewrite
GPU / host operator untrusted same as the server process (no further visibility differential — there is no TEE separating them) same as the server

Three empirical leak surfaces the protocol forces a published attack to overcome. The headline change from the paper is the partial defense against runtime-state-inversion attacks (ISA / IMA / NN): on Qwen3 we ship only the inter-head shuffle portion of paper Algorithm 2 §5.2.3, not the full intra-head construction.

Leak channel Attack class Current mitigation Status vs paper
Plaintext-vs-obfuscated weight pair VMA · IA Embedding / head Gaussian noise (αe·ε, αh·ε; αe=1.0, αh=0.2; paper §5.2.2) + Algorithm 1 key matrices on every residual-stream-touching weight. Sort-then-match by row similarity no longer recovers τ; the linear system for P̂_R is ill-conditioned by λ > 0. Parity. The Π token permutation and §5.2.2 Gaussian noise implement the paper construction verbatim. TTRSR not yet measured on this artifact (attack-resistance benchmark pending).
Internal-state inversion
paper §5.2.3 attack surface 		ISA · IMA · NN Two-side key matrices Q̂·W·P̂ on every weight scramble residual-stream activations. Inter-head τkv + τgroup shuffle which Q/K/V head produced each attention score. Intra-head transforms (R̂qk, Ĥqk, Ẑblock) deployed since 2026-05-19 via the matrix-Γ kernel extension (MT·Diag(γqk)·M replaces 1D γqk at the q_norm site). Scrambles per-head_dim coordinate identity of Q/K within each head. Closer to paper-full after the 2026-05-19 ramp. Paper applies 5 attention-side transforms; we now apply 4 (R̂qk, Ĥqk±1, Ẑblock, Πhead). The fifth, distinct Ẑblock_η on the K side, is not deployed — alg2.py uses the same Z for both Q and K (with repaired Mk construction so Mq·MkT = I holds). ISA HS at attn_norm-23 now passes the 15 % gate. IMA-L0-activation (AloePri-specific attack at attn_norm-0) does not move under Algorithm 2 because that surface is pre-Wq. See §08 ledger.
Token-frequency statistics TFMA · SDA τ obfuscates token identity at the index level; the server's empirical unigram distribution over obf_id is a τ-shuffling of the plaintext unigram, recoverable only by cross-corpus frequency matching. Parity. Same construction as paper. Not yet measured.
Per-request entropy any future attack exploiting key reuse None at the protocol level — τ, key matrices, and noise ε are sampled once at offline rewrite and reused across every request. The mitigation is the noise + key-matrix construction itself, not freshness. Structural difference from GELO's per-batch fresh Haar mask. Accepted trade.

Internal-state inversion (paper §5.2.3) — what's defended, what isn't

What ISA / IMA threaten. A server operator reads intermediate hidden states or attention scores from GPU memory during inference, then runs them through an inverter trained on plaintext Qwen3 1.7B to recover the user's tokens. Algorithm 1 keymat alone preserves Q, K, V byte-for-byte (by design, so the model still works) — so a pre-trained inverter applies unchanged. Algorithm 2 closes that gap.

Algorithm 2's five intra-head + inter-head transforms. R̂qk (RoPE-aware 2D rotation per pair), Ĥqk (per-pair scaling diagonal, paired across NEOX halves), Ẑblock (RoPE-pair permutation), Ûvo (V/O random invertible — paper-specified, reference omits), plus the inter-head shuffles τkv + τgroup. All are chosen so Q·KT end-to-end equals plaintext (model still works) but internal per-(head, pair-position) values are scrambled.

The Qwen3-specific gap. The intra-head transforms all need to commute with the per-element γ at Qwen3's QK-norm site (which sits between Wq/Wk and RoPE — Qwen2.5 / Llama-3 / DeepSeek don't have this norm). Paper §5.2.5's fold-γ-into-the-adjacent-linear-weight trick is exact only under i.i.d.-Gaussian-Q assumption; trained Qwen3 Q/K vectors correlate with γ and the per-input κ approximation breaks attention. §05 (2026-05-18) therefore deployed only τkv + τgroup, leaving R̂qk / Ĥqk / Ẑblock dormant.

Our fix (2026-05-19): matrix-Γ kernel extension. Patched llama.cpp: at the QK-norm site, consume a 2D tensor Γ = MT·Diag(γqk)·M in place of the 1D γqk. Algebraically (Q / RMS(Q)) · Γ = Qplain_normed · M when M is orthogonal (exact to fp32 noise). Obfuscator bakes M into Wq / Wk output axis; kernel detects the 2D form via the aloepri.qk_norm_matrix metadata flag. No protocol change. Deploys R̂qk + Ẑblock + ±1 Ĥqk on top of §05's head shuffles. Measured: ISA HiddenState at attn_norm-23 drops 16.3 % → 11.5 % (passes §6.3 gate). See §08.

New attack surface introduced by matrix-Γ: eigendecomposition leak. The 2D Γ = MT·Diag(γ)·M is a similarity transform of a diagonal matrix. An adversary with code execution on the server can run numpy.linalg.eig(Γ) and recover γ (eigenvalues) and M (eigenvectors, up to ordering) in milliseconds — defeating the intra-head obfuscation entirely. This is a static-weight τ-recovery attack against the QK-norm tensors; the corresponding row in §08's attack table is highlighted, and the threat-model doc lives at docs/research/aloepri-qk-norm-matrix-gamma-threat-model.md. Defending it would require per-prompt fresh M (GELO-style masking), which breaks AloePri's "static obfuscation, no infra change" thesis — not measured + not defended in the current deployment.

Not covered. Model architecture (the server can fingerprint Qwen3 from layer count and tensor shapes — only the weights and token identities are obscured). Query rate and timing. Tokenizer roundtrip identity — the client uses the native /completion endpoint with integer arrays so the wire payload doesn't depend on tokenizer detok→retok, but cross-version tokenizer drift between client and server is still a deployment hazard. Model-weight privacy — the client needs plaintext weights to perform the offline rewrite; AloePri does not protect the weights themselves from the client.

§04

Supported model architectures

Qwen3 1.7B is the current target · Gemma 4 deferred

The covariant-obfuscation construction is architecture-generic per paper §8, but every new architectural feature needs its own composition-theorem application. The list below is what's actually validated end-to-end, what's deferred, and why.

Model Status Layers · Hidden · Head_dim GQA Notes
Qwen3 1.7B (current target) landed (Π token permutation · embedding/head noise · inter-head attention shuffle) 28 · 2048 · 128 (16, 8) Source GGUF bartowski/Qwen_Qwen3-1.7B-GGUF Q8_0; obfuscated artifact keymat-h128-pi-noise-alg2-FULL-zfix-hadamard-fp32.gguf at 9.1 GB (post-2026-05-19 ramp; §05 baseline keymat-h128-pi-noise-alg2-fp32.gguf superseded). RoPE base 1M, no p-RoPE. Single residual norm class per block (attn_norm, ffn_norm) + 1 global output_norm = 57 residual norm sites total. Per-head attn_q_norm, attn_k_norm = 56 additional sites, deployed as 2D Γ matrices via the matrix-Γ kernel extension (see §09).
Qwen3 4B / 8B trivial scaling 36 · 2560 · 128 (4B) (32, 8) Same architecture family; offline rewriter is parameter-agnostic. Validation effort = re-run the determinism + mini-accuracy harnesses at the new scale.
Gemma 4 E2B / E4B deferred 35 / 42 · 1536 / 2560 · 256 — Blocker: 5 residual norm sites per block. Gemma 4 inserts post-norms (post-attention, post-FFN, per-layer-post-norm) and the PLE-output norm in addition to the standard pre-norms, giving roughly 5 norms per block × 35–42 blocks = 175–210 residual norm sites. Each is a §5.2.5 fusion point whose per-input κ bias accumulates multiplicatively through layer Lipschitz factors. The paper's accuracy bound is measured at 2 norms / block (Qwen2-class); at 5 norms / block the same eCnorm compounding plausibly pushes accuracy loss to 7–12 % (or worse) on a model that's also asked to swallow PLE + K=V untie + p-RoPE on top.

Other Gemma-4-specific deltas — PLE table vocab-axis permutation, K=V untying in global layers, p-RoPE-restricted R̂qk — are individually tractable but interact with the κ-compounding ceiling. Re-attempt after the Qwen3 attack-resistance benchmark is in hand and the per-norm-site κ tuning work (see §09) has been spiked.
Qwen3-MoE / Gemma 4 26B A4B research stream — MoE Require CryptoMoE balanced-dispatch defense composed on top of AloePri. Separate work item.
Multimodal (audio / vision encoders) out of scope — — Open question on per-modality attack classes; see round-2 §D.9.

Why Qwen3 1.7B. Small enough to iterate quickly (the offline rewrite plus a smoke test cycles in ~15 s on this hardware), large enough to give meaningful accuracy numbers on MMLU / PIQA / HumanEval, and the same backbone used by the GELO-LLM demonstrator on the GELO route — so accuracy deltas between the two routes are directly comparable. Stock llama-server handles the obfuscated GGUF without a fork.

The Qwen3-specific architectural delta. The AloePri paper §7.1 explicitly lists Qwen3 in its evaluated models, but the public reference (sheng1feng/Aloepri @ 60e8ea3) imports transformers.models.qwen2.modeling_qwen2 throughout — there is no Qwen3 path in the released code. The one architectural feature that distinguishes Qwen3 from Qwen2.5 at the attention block is QK-norm: a per-head RMSNorm on Q and K, sitting between the QKV projection and RoPE. The paper's Algorithm 2 (§5.2.3) has no construction for it. ByteDance's industrial deployment presumably solved this internally; the academic artifact didn't ship that solution. §09 documents the resulting gap and our partial workaround.

§05

Components

six implementation units · Qwen3 1.7B + llama.cpp

Each card pairs [component ↦ source ↦ what it secures ↦ trust side]. Ordered by the lifecycle of a deployment: rewrite the model once → run the server → serve every request through the client wrapper.

offline

obfuscate_qwen3_ggufpython/aloepri :: rewriter

One-shot CLI that consumes a plaintext Qwen3 GGUF and emits an obfuscated GGUF plus a .key.npz file. Reads the source via gguf-py, dequantises every weight to fp32, applies the full transform stack (noise → Π → §5.2.5 residual-norm fusion → Algorithm 1 keymat → head-shuffle), writes fp32 GGUF. Runs in ~15 s on Qwen3 1.7B.

inputs
plaintext GGUF · seeds (for keymat, Π, noise, alg2) · hyperparameters (λ, h, αe, αh, β, γ, qk_scale_range)
outputs
obfuscated GGUF with qwen3.embedding_length = d + 2h and aloepri.* metadata · <out>.key.npz (τ + per-layer head-shuffle keys)
key file
POSIX 0600, never logged, never transmitted. The pi_seed / alg2_seed are not stored in the GGUF — only the client-side .key.npz holds them, so the server cannot reconstruct τ from the artifact.
parity test
gamma-only mode produces bit-identical output to plaintext under the same prompt (verifies §5.2.5 fusion correctness); keymat mode produces coherent on-topic continuations (verifies covariant chain holds end-to-end).
server

llama-serverghcr.io/ggml-org/llama.cpp:server-vulkan

Stock llama.cpp HTTP server. Loads the obfuscated GGUF, reads qwen3.embedding_length = 2304 from metadata, builds the standard Qwen3 forward graph. No protocol-aware code: every Algorithm-1 / head-shuffle transform is already baked into the weight tensors offline. The server cannot distinguish the obfuscated artifact from a normal Qwen3 of unusual hidden dim.

endpoint
POST /completion (native) — accepts "prompt": [int, …] directly, returns "tokens": [int, …] with return_tokens: true. Bypasses tokenizer detok/retok on the wire.
delta-LoC vs stock
0. No fork. Qwen3 has been in llama.cpp mainline since well before this project; AloePri's only requirement is "qwen3.embedding_length is read from metadata, not hardcoded" — which is already true.
integration risk
None at the present scale. Future Qwen3 architecture variants (Qwen3-Next state-space hybrid, Qwen35 IMRoPE) may diverge; pin the artifact to a specific Qwen3 dense subarch.
client

AloePriClientpython/aloepri :: aloepri_client

The trusted-side wrapper. Holds τ and the tokenizer. Every complete(prompt, …) call: tokenize plaintext → map IDs through τ → POST as an integer array to llama-server's native /completion → receive tokens array → map through τ-1 → detokenize → return plaintext.

endpoint
POST {LLAMA_URL}/completion with {"prompt": [obf_ids], "return_tokens": true, …} — native llama.cpp API.
integer wire
The wire payload is an int array, not text. No tokenizer encode↔decode roundtrip on the network path; no BPE-edge-case risk. The client's tokenizer is the only one in the loop.
active-range guard
τ permutes only the tokenizer's used vocab range [0, 151 669). The GGUF's full table size 151 936 includes 267 padding slots that stay identity. Client refuses to run if tokenizer vocab_size ≠ key.active_size.
crypto

KeyMaterialaloepri_client :: KeyMaterial

The full set of secrets generated at offline rewrite. τ is the only element the client wrapper actually consumes (token-level I/O). The per-layer head-shuffle keys (τkv, τgroup) are saved for reproducibility and for the future attack benchmark but never touch the request path.

contents
seed (32 B) · pi_seed · τ (length 151 936, ~1.2 MB raw / ~422 KB compressed) · active_size · per-layer τkv, τgroup permutations · arch tag · format version
scope
Lives in the client wrapper's process memory; persisted as <artifact>.gguf.key.npz in 0600 mode; never crosses the trust boundary.
rotation
Compromise of the key file = compromise of every past and future prompt under this artifact. Rotation requires re-running the offline rewrite end-to-end and re-uploading the obfuscated GGUF.
eval

attack-harnessevals/aloepri-attacks/

12-attack prompt-inversion benchmark adapted from the AloePri reference (vendor/aloepri-py/src/security_qwen/) plus AloePri surface variants. Three surface families: static obfuscated weights (VMA, IA, IMA-EmbedRow-ridge, IMA-EmbedRow-transformer, QK-norm Γ eigendecomposition); forward-pass activations (NN, IMA-L0-activation across 3 surfaces, IMA-L0-transformer, ISA HiddenState across 3 surfaces, ISA AttnScore); wire-side response stream (TFMA, SDA). Targets the obfuscated Qwen3 1.7B artifact running on llama-server; reports per-attack TTRSR against paper §6.3's ≤ 15 % acceptance threshold. See §08 for the full per-attack ledger.

release-gate threshold
TTRSR ≤ 15 % per attack, at paper-recommended hyperparameters (αe=1.0, αh=0.2, h=128, λ=0.3, β=8, γ=1e3).
specific watchpoint
ISA on the partial-Algorithm-2 artifact tells us whether head-shuffle alone is enough or whether per-norm-site κ tuning is needed (§09).
eval

accuracy-harnesspython/aloepri/evals/

Mini accuracy benchmark: MMLU 0-shot (200 prompts), PIQA 0-shot (200), HumanEval pass@1 (50), IFEval verifier subset (50). Drives both the plaintext baseline and the obfuscated artifact through identical sampling configs and reports the per-task accuracy delta.

scope
"In-budget signal, not publishable numbers" — 30–45 min runtime, single Vulkan iGPU container per endpoint. Reusable for hyperparameter sweeps and post-noise / post-Algorithm-2 regressions.
current results
MMLU Δ +0.5 pp · PIQA Δ −4.0 pp · HumanEval Δ −6.0 pp (n=50, SE ~7 pp) · IFEval deferred (timeout debugging).
determinism
Vulkan iGPU + flash-attn + -np 1 + temperature 0 + seed 0 is fully byte-deterministic in practice (verified across 5 prompts × 3 replicates each).
§06

Compute flow & trust boundaries

per-request — what crosses the wire · per-block — what the server actually computes

Two figures and two algorithm specs. FIG. 03a shows the full deployment in two stages — Stage 1 is the one-time setup where the client samples the keymat / τ and builds the obfuscated GGUF; Stage 2 is the per-request flow where plaintext stays inside the trusted zone and only an obfuscated integer-ID array crosses the wire. FIG. 03b shows the per-attention-block Algorithm 2 deployment, now fully landed (matrix-Γ kernel, 2026-05-19) including the intra-head transforms (R̂qk, Ĥqk, Ẑblock). The two callouts below summarise each algorithm's parameters and — importantly — which attack surfaces it is and is not designed to defend.

Algorithm 1 — Key Matrix (paper §5.2.1) two-side linear obfuscation of every residual-stream weight · P̂·Q̂=I_d guarantees forward-pass correctness

For every residual-stream weight W ∈ ℝd×d, the obfuscator emits W̃ = Q̂·W·P̂ (read-side) or W·P̂ (write-side, e.g. embedding) where P̂ ∈ ℝd×(d+2h) and Q̂ ∈ ℝ(d+2h)×d. The expanded d+2h internal dimension carries decoy directions sampled from null(FT) and null(E); only the d-dimensional projected subspace carries real signal. P̂ is sampled as [B | C | E]·Z with B = U + λV, U orthogonal, V Gaussian, Z orthogonal — so the noise scale of P̂'s direction structure is controlled by λ.

Parameter Default Effect
d 2048 (Qwen3-1.7B) model hidden dim · fixed by architecture
h 128 expansion size · inflates residual dim to d+2h=2304 · larger = harder ridge inversion · paper Figure 5 shows <10 % TPOT regression up to h=512
λ 0.3 coefficient on the Gaussian B = U + λV · larger λ = noisier P̂ direction structure · paper Figure 4: FP32-safe up to λ=3.0; BF16 breaks at 3.0
αe, αh 1.0, 0.2 (paper) · 0.1, 0.033 (aloepri default for d=2048) §5.2.2 Gaussian noise scale on embedding / LM head · larger = harder direct row recovery · at our aloepri default the noise is <¼ of paper's per row (see §09 dimensional-headroom note)

Defends (with paper's d=5120 / α_e=1.0): VMA (per-column sort-and-match defeated by P̂'s mixing) · IA (gate-IA / attn-IA invariants scrambled by Q̂) · partially: IMA-EmbedRow-ridge at large d (noise scatter dilutes into garbage directions).
Does NOT defend by itself: IMA-EmbedRow-ridge against a paired-data attacker when α_e and d-headroom are small — the obfuscation is a static linear map and ridge learns P̂⁻¹ in closed form. See §09 for our 99.22 % gap vs paper's 0 % and why.

Algorithm 2 — Attention obfuscation (paper §5.2.3) intra-head + inter-head transforms of Wq,k,v,o · static per-deployment orthogonal mixing of Q/K vectors

For each layer's attention block, the obfuscator composes an intra-head matrix Mq = R̂qk · Ĥqk · Ẑblock (orthogonal) into Wq's output axis, an analogous Mk into Wk, and an inter-head permutation (τkv, τgroup) across heads. Per-projection keymat (Q̂q, Q̂k, Q̂v, P̂o from Algorithm 1) wraps the d-axis. Together with the matrix-Γ kernel extension that consumes Γ = MT·Diag(γqk)·M in place of the 1D γ at attn_q_norm / attn_k_norm, the forward-pass identity holds: (Q̃ / RMS(Q̃)) · Γq = Qplain_normed · Mq.

Component Param Default Role
R̂qk ρ_i ∈ (0, 2π] random per pair 2×2 rotation per RoPE pair · commutes with RoPE because it rotates within the pair
Ĥqk signs Walsh-Hadamard ±1 involutive sign matrix · keeps Mq orthogonal so matrix-Γ algebra is exact
Ẑblock β 8 max window for the dynamic-window block permutation across RoPE 2×2 pairs
Ẑblock γ 1e3 softmax temperature for window-size sampling in BlockPerm
τkv S(n_kv_heads) random permutes kv heads · stays orthogonal to RoPE structure
τgroup S(n_groups) random permutes query head groups · GQA structure-preserving
Ûvo — random invertible cancels through Wv·Wo path · preserves attention output identity

Defends (paper's threat model): blind-signal-separation attacks on Q/K activations (ICA-style attackers with no paired data) · cross-batch statistical attacks on attention scores · per-head linear inverters that don't account for the head-axis permutation.
Does NOT defend (structural): anything pre-Wq · IMA-EmbedRow-ridge (Alg 2 doesn't touch We at all) · IMA-L0-activation at attn_norm-0 (pre-Wq residual) · paired-data ridge attacks on Qcur_normed / Kcur_normed because Mq is static-orthogonal (learnable in closed form given paired training data; see §09 measured ledger).

FIG. 03a — Two stages · Stage 1 setup builds the obfuscated GGUF on the client (Algorithm 1 + §5.2.2 noise + Π + optional Algorithm 2) · Stage 2 per-request flow keeps plaintext inside the trusted zone, only an obfuscated int array crosses the wire navy = client (trusted) · gpu-tone = llama-server (untrusted) · dashed amber = obfuscated wire payload · solid grey arrows = one-time setup transfers
Trusted zone · client wrapper holds τ · keymat · tokenizer · plain Qwen3 source weights · no GPU Untrusted zone · llama-server (matrix-Γ patched llama.cpp) receives obfuscated GGUF θ̃ once · serves /completion · no τ, no keymat setup: HTTPS GGUF upload · request: HTTPS obf int array STAGE 1 · SETUP (one-time, per deployment) client samples keymat + τ · applies Algorithm 1 + noise + Π + Algorithm 2 to the source GGUF · uploads obfuscated θ̃ to server ① source plain GGUF Qwen_Qwen3-1.7B-Q8_0.gguf · obtained from huggingface.co/bartowski ② Algorithm 1 — sample keymat P̂, Q̂ with P̂·Q̂ = I_d params: d=2048 (Qwen3-1.7B), h=128, λ=0.3 U ~ O_d · V ~ N(0,1/d)^{d×d} · B = U+λV · E,F via rank-h/2 Gaussian pairs · Z ~ O_{d+2h} P̂ = [B | C | E]·Z ; Q̂ = Zᵀ · [B⁻¹ | F | D]ᵀ C ⊂ null(Fᵀ), D ⊂ null(E) — decoy structure in the +2h expansion ③ sample τ ∈ S_151643 (vocab permutation, special tokens identity) permutable = {ids with token_type ∈ {NORMAL, BYTE}} · 151 643 of 151 669 active 26 special tokens (EOS, BOS, im_start, im_end, fim_*) kept identity ④ apply §5.2.5 fusion + Π + noise + Algorithm 1 + Algorithm 2 W̃_embed = Π · (W_e + α_e·σ_e·ε) · P̂_embed (α_e = 0.1 aloepri default; paper 1.0) W̃_q,k = Q̂_R · W_q,k · M_q,k where M_q = R̂_qk·Ĥ_qk·Ẑ_block (orthogonal, Walsh-Hadamard Ĥ) Γ_q = M_qᵀ·Diag(γ_q)·M_q → attn_q_norm becomes 2D (matrix-Γ) W̃_v = Q̂_v·W_v·Û_vo ; W̃_o = Û_vo⁻¹·W_o·P̂_o W̃_ffn covariant chain ; W̃_head = Q̂_head·(W_h+α_h·ε)·Πᵀ residual γ folded into adjacent linear, norm site replaced with scalar κ (§5.2.5) ⑤a obfuscated GGUF θ̃ keymat-h128-pi-noise-alg2.gguf + aloepri.qk_norm_matrix=true ⑤b .key.npz (stays on client, forever) {τ, τ⁻¹, vocab_size, active_size, seeds} never crosses the wire ⑥ llama-server (patched llama.cpp · option-c image) awaiting model upload one-time upload ~9 GB · HTTPS ⑦ load θ̃ · build compute graph detects aloepri.qk_norm_matrix → branches q_norm/k_norm to ggml_mul_mat consumes 2D Γ_q (128×128) instead of legacy 1D γ_q else (no flag) → bit-identical to upstream llama.cpp server has no awareness of τ or the obfuscation protocol ⑧ /health → ok · listening on :8061 forward graph identical to plaintext Qwen3 modulo matrix-Γ branch τ persists across all Stage 2 requests STAGE 2 · ONLINE REQUEST (per query) plaintext prompt → tokenize → τ-map → /completion (obf int array) → τ⁻¹ → detokenize → plaintext response ① plaintext prompt "What is the capital of France?" ② tokenize (Qwen3 BPE, no special tokens) plain_ids = [3838, 374, 279, 6722, 315, 9625, 30] ③ map each token ID through τ (from .key.npz, never crosses wire) obf_ids = [137 397, 44 230, 90 908, 60 247, 33 846, 135 351, 102 727] POST /completion {"prompt": [obf_ids], "return_tokens": true, "stream": true} attacker tokenising obf_ids back would see: "いるとupgrade Printf\tmode…" — gibberish ④ llama-server forward pass accepts int array directly — no server-side tokenisation needed embedding lookup: W̃_e[obf_ids] → (implicit) plain_embed in P̂_R space 28 decoder blocks · GQA(16,8) · RoPE 1M sample argmax over obfuscated logits forward graph identical to plaintext Qwen3 (see FIG. 03b for the attention-obfuscation detail) SSE stream {"content": "…(gibberish)", "tokens": [obf_resp_ids]} · client ignores content ⑤ accumulate obf_resp_ids from SSE chunks obf_resp_ids = […] · stream-mode dodges server-side PEG chat-parser ⑥ map response IDs through τ -1 plain_resp_ids = inv_tau[obf_resp_ids] ⑦ detokenize plaintext IDs "The capital of France is Paris.\n\nNo, that's not right…" ⑧ return plaintext response to caller

Why the integer-array wire matters. The canonical AloePri protocol in the paper passes text on the wire (client detokenises obf_ids → string, server re-tokenises → ids). That works only if the tokenizer is round-trip-lossless on every ID list — a property that fails on byte-fallback tokens, special tokens, and pathological BPE merges. llama-server's native /completion endpoint accepts integer arrays directly and can return them with return_tokens: true, removing the roundtrip entirely. The wire payload is an int array; if an observer detokenises it with the stock Qwen3 tokenizer they get UTF-8 gibberish like "いるとupgrade Printf\tmodeocyCIÓN芬", but that's their reconstruction — the protocol itself never serialises to text on the wire.

Boundary What crosses What does not
Stage 1 · setup · client ↔ server obfuscated GGUF θ̃ (~9 GB, one-time) · aloepri.qk_norm_matrix metadata flag · public Qwen3 architecture metadata τ · .key.npz · keymat (P̂, Q̂, M_q, Û_vo) · noise seeds · the plain source GGUF
Stage 2 · request · trusted zone ↔ network obfuscated prompt int array · obfuscated response int array · HTTPS metadata · request count · request timing τ · plaintext prompt · plaintext response · tokenizer config
Network ↔ llama-server same as above; server treats it as a normal int-array prompt for native /completion same as above
llama-server ↔ host / GPU obfuscated weights θ̃ · all internal activations · KV cache · sampler state nothing else — there is no further trust split inside the untrusted zone
Client wrapper ↔ rest of the trusted app plaintext prompt in · plaintext response out · usage / finish-reason metadata τ · key material — the wrapper is the τ-holding boundary even on the trusted side
FIG. 03b — Algorithm 2 (paper §5.2.3) full deployment at one Qwen3 attention block · matrix-Γ kernel (2026-05-19) carries the intra-head Mq=R̂qk·Ĥqk·Ẑblock through QK-norm via a 2D Γ = MT·Diag(γqk)·M tensor green = transform applied · the QK-norm op now consumes a 2D Γ tensor so the intra-head rotation cancels at the norm site · (per §08 attack ledger, Alg 2 currently buys no measurable defence gain on this Qwen3-1.7B / d=2048 / αe=0.1 config against the paired-data ridge attacker set; see §09)
Untrusted zone · llama-server forward pass · single attention block ℓ Qwen3 1.7B · GQA(16, 8) · head_dim=128 · RoPE base 1M · γ_q, γ_k shape [128] broadcast across heads H̃_in shape (n, d+2h=2304) obfuscated residual stream ① attn_norm — §5.2.5 fused γ_resid folded into W_q/k/v; norm site uses scalar κ ② Q̃, K̃, Ṽ projections — keymat · head-shuffle W̃_q = Q̂_R · W_q · rows permuted by feature_order(τ_kv, τ_group) W̃_k = Q̂_R · W_k · W̃_v = Q̂_R · W_v · K/V rows permuted by feature_order(τ_kv) whole-head granularity: head h of new W_q = head q_head_order[h] of old W_q ③ reshape Q̃ / K̃ / Ṽ to (n, n_head, head_dim=128) ④ QK-norm — matrix-Γ (2026-05-19 patched kernel) attn_q_norm.weight is now 2D Γ_q = M_qᵀ · Diag(γ_q) · M_q (head_dim × head_dim) kernel branches on aloepri.qk_norm_matrix → RMS-only + 128×128 ggml_mul_mat algebraic identity: (Q̃ / RMS(Q̃)) · Γ_q = Q_plain_normed · M_q (M_q orthogonal) ⑤ RoPE on Q̃, K̃ — Ẑ_block applied via M_q R̂_qk folded into W_q's output; Ẑ_block (β=8 window) permutes RoPE 2×2 pairs ⑥ attention — intra-head M_q + head shuffle τ_kv, τ_group Q̃_normed = Q_plain_normed · M_q ; K̃_normed = K_plain_normed · M_k with M_q·M_kᵀ = I (Ẑ-fix landed 2026-05-19) Q̃·K̃ᵀ ≡ Q_plain·K_plainᵀ exactly; head-axis index permuted by η(h) = h ÷ num_groups ⑦ output projection — Û_vo + keymat + head-axis perm W̃_v = Q̂_v · W_v · Û_vo ; W̃_o = Û_vo⁻¹ · W_o · P̂_o ; Û_vo cancels through columns permuted by feature_order(τ_kv, τ_group); attention output lands in plaintext W_o space mod head shuffle ⑧ residual + FFN (covariant; not shown) SwiGLU with Q̂·W·P̂ on every linear · ffn_norm §5.2.5-fused matrix-Γ kernel: how Algorithm 2 lands on Qwen3 paper §5.2.3 for Q projection: W̃_q = Q̂_q · W_q · R̂_qk · Ĥ_qk · Ẑ_block Qwen3 has QK-norm γ between W_q and RoPE — §5.2.5 fold of γ needs i.i.d. Gaussian q which empirically degrades the model. Solution (patched llama.cpp): attn_q_norm.weight is now 2D: Γ_q = M_qᵀ · Diag(γ_q) · M_q shape (head_dim, head_dim) = (128, 128) M_q orthogonal (Walsh-Hadamard ±1 Ĥ_qk + Ẑ_block + R̂_qk). (Q̃ / RMS(Q̃)) · Γ_q = Q_plain_normed · M_q forward-pass identity exact: intra-head M_q lands in Q̃_normed without Gaussian-q assumption. kernel gate: aloepri.qk_norm_matrix metadata flag · legacy 1D γ path is bit-identical fallback. what's now applied (full Algorithm 2) intra-head, per layer: R̂_qk · Ĥ_qk · Ẑ_block (orthogonal M) folded into W_q output axis inter-head, per layer: τ_kv ∈ S_8, τ_group ∈ S_2 η(h)-preserving on the GQA chain V/O leg: Û_vo random invertible cancels W_v·W_o composition residual-axis wrap: Q̂_q, Q̂_k, Q̂_v on input; P̂_o on output projection measured defence gain on this Qwen3-1.7B / d=2048 config: IMA-EmbedRow-ridge: 99.22 → 99.22% IMA-L0-act @attn_norm-0: 22.6 → 22.6% IMA @Qcur_normed-0: 35.5 → 38.7% → no measurable gain. Static orthogonal M_q is learnable by paired-data ridge. See §09. ■ green box: transform applied (now full Algorithm 2 + matrix-Γ) kernel branches on aloepri.qk_norm_matrix GGUF metadata flag · legacy 1D γ path bit-identical to upstream net result: keymat + intra-head M_q + head shuffle preserve Q·Kᵀ exactly (M_q·M_kᵀ = I via Ẑ-fix); attention pattern is intact attacker observing Q̃_normed sees Q_plain_normed · M_q — a static orthogonal rotation that a paired-data ridge inverter learns in closed form §08 ledger: ISA HiddenState @attn_norm-23 stays at 9.68 % ✓ · IMA @Qcur_normed-0 = 38.71 % (vs 35.48 % without Alg 2) — Alg 2 does not move the gate at d=2048 public reference at sheng1feng/Aloepri imports qwen2 modeling · has no QK-norm path · the matrix-Γ kernel extension bridges that gap on Qwen3

Where the obfuscation actually lives (Qwen3 1.7B)

Layer / op Plaintext form Obfuscated form Composition argument
Embedding We[x, :] (Π · (We + αe·ε) · P̂embed)[τ(x), :] Π row-permutes by τ⁻¹; P̂embed right-multiplies so output lands in P̂R space; noise defeats VMA
Residual norms (attn_norm, ffn_norm, output_norm) x · γ / ‖x‖RMS x · κ / ‖x‖RMS(d+2h) §5.2.5 fold: γ pre-baked into the adjacent linear weight; scalar κ at the norm site commutes with the residual-stream Q̂/P̂ chain. eCnorm bound per paper §5.2.4.
QKV projections x · Wq,k,v (x · Q̂R) · Wq,k,v · Mq/k · (rows permuted by τkv, τgroup) Keymat absorbs the residual-stream Q̂; the intra-head Mq = R̂qk·Ĥqk·Ẑblock (orthogonal) is folded into Wq's output axis with the matching Mk on Wk such that Mq·MkT = I; head permutation lands the per-head output at a shuffled index. See FIG. 03b.
QK-norm (Qwen3-specific) q · γq / RMShead_dim(q) (q̃ / RMS(q̃)) · Γq with Γq = MqT·Diag(γq)·Mq (matrix-Γ, 128×128) Patched llama.cpp kernel reads the 2D tensor when aloepri.qk_norm_matrix metadata flag is set, applies RMS-only normalisation followed by a 128×128 ggml_mul_mat. Algebraic identity (q̃ / RMS(q̃)) · Γq = qplain_normed · Mq holds for orthogonal Mq; no Gaussian-q assumption needed.
Output projection attn · Wo attn · Wo · P̂R (columns permuted by head shuffle) Right-multiplied keymat lands back in the next layer's residual space; head-axis permutation undoes the input-side shuffle of attn.
FFN (gate / up / down) standard SwiGLU Each weight wrapped Q̂ · W · P̂; ffn_norm §5.2.5-fused into gate + up Sequential composition theorem (paper §4.2.1) chains Q̂ at gate/up's input with P̂ at down's output; intermediate inflated dim absorbs the expansion h.
LM head Hfinal · Wh (H̃final · Q̂head) · (Wh + αh·ε) · ΠT Q̂head cancels the residual stream's P̂; ΠT obfuscates the output-vocabulary identity so the sampler picks τ(true_next_token); noise αh·ε defeats head-side VMA.

The composition theorems in paper §4.2 (sequential, parallel, summation) lift every per-component obfuscation into the full forward graph; this is what makes the keymat + head-shuffle composition still preserve model correctness despite skipping the intra-head transforms. The cost is a smaller obfuscation group at the attention layer — paper Algorithm 2 fully applied gives a much larger group of admissible (Q̃, K̃) pairs at each head, while head-shuffle alone only permutes among the 16!·8!·… cardinality of head orderings, which is recoverable in principle by an attacker who knows the GQA structure.

§07

Performance & correctness

measured on Qwen3 1.7B + 4B + 8B · Strix Halo Vulkan iGPU · bf16 default

Numbers below cover three deployments: the original Qwen3 1.7B obfuscated artifact (keymat-h128-pi-noise-alg2-fp32.gguf, 8.6 GB fp32), the 4B sibling (9.1 GB bf16), and the 8B sibling (17.4 GB bf16, also untied, paper-default config). All endpoints run inside the patched aloepri-llama-server:option-c container (matrix-Γ kernel) with -ngl 999 -np 1 --flash-attn on -c 4096 --ubatch-size 1024, on a Strix Halo Radeon 8060S iGPU. The container needs --group-add 992 (render gid) for Vulkan to find renderD128 — without it the backend silently falls back to CPU.

bf16 is the obfuscator default since 2026-05-20. obfuscate_qwen3_gguf.py --output-dtype bf16 (the new default) halves on-disk size and 2× decode throughput vs fp32, accuracy-equivalent (HumanEval pass@1 6/20 fp32 ↔ 6/20 bf16 on Qwen3-4B, byte-identical pass-id set). fp16 is explicitly unsafe — collapses the keymat cancellation; see "Production-precision validation" below for the denormal-flush mechanism. The 4B + 8B numbers in this section and §08 were measured against bf16-native. The 1.7B numbers were measured against the older fp32 artifact; TODO: re-measure 1.7B at bf16 to confirm accuracy equivalence holds at d=2048 too. 8B is the highest-accuracy obfuscated cell measured so far — HumanEval pass@1 8/20 = 40 % at paper-default αe=1.0, beating both 1.7B (35 %, αe=0.1) and 4B (30 %, αe=1.0).

Production-precision validation (sizes from Qwen3-4B obfuscated; 8B ratios identical at bf16 → halves fp32 size)

Format Size (4B) HumanEval pass@1 (n=20, αe=1.0) Decision
fp32 (reference) 18.5 GB 6/20 = 30 % reference, retained as --output-dtype fp32
bf16 (default) 9.1 GB 6/20 = 30 % production default — byte-identical pass-id set, 2× tps, ½ disk
fp16 9.3 GB 0/20 = 0 % UNSAFE — flushes ~1.15 % of attn_q to denormal-zero (obfuscated min_nz ≈ 3e-10 < fp16 floor 6e-5), breaks P̂·Q̂ = I_d cancellation
Q8_0 ~5.0 GB degenerate ("( ( ( ( ,chein,zech…" on 1.7B; not retested 4B) fails
Q6_K ~3.9 GB degenerate (1.7B) fails
Q5_K_M ~3.5 GB word-salad (1.7B) fails

Why obfuscated weights resist block-scale quantisation. AloePri-keymat weights are heavy-tailed per-row (max ≈ 55, std ≈ 4.7 on layer 27 of Qwen3 1.7B). Block-scale formats (Q8_0 stores one fp16 scale per 32 elements) lose small values to zero when a block also contains an outlier, breaking the covariant chain. K-quants are more flexible but still can't preserve the precision needed.

Why bf16 works where fp16 doesn't. The keymat construction B = U + λV with λ=0.3 produces a long lower tail in the transformed weight tensors: median row L2 norm ≈ 1.13 but min element magnitude reaches ~3e-10 in attn_q. fp16's smallest normal is ~6e-5, so ~1.15 % of attn_q entries flush to zero, breaking the exact P̂·Q̂ = I_d cancellation. bf16 keeps fp32's full 8-bit exponent (range down to ~1e-38) at the cost of 7-bit mantissa precision; cancellation survives, individual element values shift by ~1e-3 relative which is below the keymat's noise floor. Empirically: bf16 matches fp32 byte-for-byte on HumanEval, all attacks pass with negligible drift (IMA-EmbedRow-ridge: 57.81 % fp32 → 57.03 % bf16). The obfuscator default flipped from fp32 to bf16 in commit 2026-05-20.

Determinism verification

temperature=0.0, seed=0, -np=1, --flash-attn on produces byte-identical replicates on this hardware: 5 prompts × 3 replicates × 2 endpoints, all 30 calls produce the same output as their replicate group. The concern that Vulkan workgroup ordering or flash-attn tile scheduling would introduce per-run drift does not manifest at this scale. The accuracy deltas reported below are therefore real signal, not sampling noise within a single run.

Mini accuracy benchmark — plaintext vs obfuscated

Task n Plaintext Q8_0 (1.7B) Obfuscated fp32 (1.7B) Δ (pp) SE (pp)
MMLU 0-shot 200 54.5 % 55.0 % +0.5 ±3.5
PIQA 0-shot (multi-choice) 200 68.5 % 64.5 % −4.0 ±3.3
HumanEval pass@1 50 40.0 % 34.0 % −6.0 ±6.9
IFEval (subset) 50 deferred — initial run hit a 180 s read timeout under shared-iGPU contention; not re-run after container hygiene fix
Task n Plain / non-UVO ref Obfuscated 4B bf16 Obfuscated 8B bf16 Notes
HumanEval pass@1 20 6/20 = 30 % (non-UVO canonical) 6/20 = 30 % (beta2/no-H pow2 UVO) 8/20 = 40 % 4B beta2/no-H true-paper-K + pow2 UVO matches the non-UVO canonical n=20 gate and improves over dense/default UVO (3/20 = 15 %). 8B remains best on this small HumanEval sample.
5-prompt quality probe 5 5/5 readable 5/5 readable (beta2/no-H) 5/5 readable beta2/no-H passes readability; β=4/8 Q/K variants fail here, so HumanEval is the stronger utility signal
MMLU / PIQA / IFEval (4B + 8B) — deferred — 4B/8B accuracy sweep beyond HumanEval not yet wired through the patched server's /completion endpoint (TODO)

Pattern. Multi-choice / knowledge tasks (MMLU) absorb the κ-bias drift cleanly — a single-token decision is robust to small logit shifts. Generative tasks (PIQA solution selection on a base model, HumanEval code completion) drift further, plausibly because eCAloePri compounds multiplicatively across the generated token sequence. All three measured tasks are within paper-bound territory (≤ 3.5 pp) or in the marginal "proceed with flag" band (3.5 – 10 pp). None are in the "stop and tune" band (> 10 pp).

Latency

Hardware: AMD Strix Halo Ryzen AI Max+ 395 · Radeon 8060S iGPU · Vulkan backend · patched aloepri-llama-server:option-c container per endpoint with -ngl 999 -np 1 --flash-attn on -c 4096 --ubatch-size 1024 --user 1000:1000 --group-add 992 --device /dev/dri. The render-group membership is load-bearing — without gid 992 access to /dev/dri/renderD128, the Vulkan backend silently falls back to CPU (a ~4× speed loss). Plain Qwen3-4B Q8_0 on the same image establishes the upper-bound at 49 tps decode.

Component Plain Qwen3 4B Q8_0 (ref) Obfuscated 4B bf16 (default) Obfuscated 8B bf16 (default)
weights on disk 4.0 GB 9.1 GB 17.4 GB
decode (TPOT) 49.0 tok/s 22.1 tok/s 12.2 tok/s
prompt eval (prefill) 217 tok/s ~80 tok/s ~59 tok/s
HumanEval pass@1 (n=20) n/a 6/20 = 30 % 8/20 = 40 %
vs plain (decode ratio) 1.00× (ref) 0.45× 0.25×

Both obfuscated columns are bf16-native (the obfuscator default since 2026-05-20). 1.7B and fp32-reference columns are dropped from this table — see the "Production-precision validation" table above for the fp32-vs-bf16 ledger. The decode-ratio row treats plain Q8_0 4B as the upper bound. 8B is the highest-accuracy obfuscated deployment (40 % HumanEval, best of any cell) — the wider d=4096 absorbs paper-default αe=1.0 noise more gracefully than 4B at d=2560.

1.7B request breakdown (kept for historical context)

Component Plaintext Qwen3 1.7B Q8_0 Obfuscated Qwen3 1.7B fp32 Ratio / notes
weights on disk 2.2 GB 8.6 GB 4× — fp32 vs Q8_0 storage, no quantisation of obfuscated chain
weights in iGPU memory ~2.3 GB ~9.0 GB plus KV cache (grows with seq_len)
decode (TPOT) ~150 tok/s ~30 tok/s 5× slower — bottleneck is weight-bandwidth on the iGPU, not arithmetic
prefill (small prompt, 32 tok) < 100 ms ~250 ms dominated by initial weight load + KV cache warm-up
client: tokenise plaintext (HuggingFace tokenizers) — < 1 ms BPE on a 7-token prompt
client: τ-map IDs (numpy fancy-index) — < 10 μs length-151 669 lookup table; bounded by L2 cache
HTTP round-trip (localhost) — ~2 ms POST to /completion, JSON parse, response
client: τ⁻¹-map response (numpy fancy-index) — < 10 μs same lookup, reverse direction
client: detokenise response — < 1 ms BPE decode
request total (7-token prompt → 24 tokens out) ~0.4 s ~1.0 s obfuscated is server-decode-bound; client overhead < 5 ms (~0.5% of total)

Where the slowdown lives. The 4× decode penalty (4B bf16: 22.1 tps vs plain Q8_0 49.0 tps) is the bandwidth cost of bf16 vs Q8_0 (~2× bytes per weight) plus the +12.5 % d-expansion (h=128) plus a small matrix-Γ matmul per layer. Per-request client-side work (tokenise + τ-map + HTTP + τ⁻¹ + detokenise) totals well under 5 ms — the actual numpy fancy-indexing for a 32-token τ-mapping is microseconds, dominated by L2 cache. fp16 would double throughput again but collapses the obfuscation chain (denormal flush); Q8_0 would 2× more but also collapses (heavy-tailed per-row distribution). bf16 is the operational sweet spot.

What this hardware represents. Strix Halo is the upper end of consumer integrated graphics (256-bit LPDDR5x-8000, ~256 GB/s memory bandwidth). Discrete GPUs with HBM (H100 ≈ 3000 GB/s, RTX 4090 ≈ 1000 GB/s) would close the bf16-vs-Q8_0 gap further but at a different cost envelope. AloePri targets the iGPU / commodity-EPYC envelope deliberately — paper §7.7's claim that decode is "identical to plaintext" is exact only when the obfuscated artifact is in the same quantisation class as the plaintext baseline; bf16 narrows the gap to a single quantisation class jump (bf16 vs Q8_0) while keeping accuracy paper-faithful.

§08

Attack harness

twelve attacks · paper-published expectations · per-attack reading guide

Our threat model is paper §3.2 + activation-privacy: the cloud server is honest-but-curious; the attacker has access to the obfuscated model θ̃, the obfuscated prompts τ(x), the obfuscated responses τ(y), and captured runtime activations from server-side forward inference. The harness implements 12 attacks across three surface families: (1) static obfuscated weights — VMA, IA, IMA-EmbedRow-ridge, IMA-EmbedRow-transformer, QK-norm Γ eigendecomposition; (2) forward-pass activations — NN, IMA-L0-activation, IMA-L0-transformer, ISA HiddenState, ISA AttnScore; (3) wire-side response stream — TFMA, SDA. All recover τ — the secret permutation that links plaintext to obfuscated token ids — so all are prompt-inversion attacks against the deployment. Captures + attacks complete 2026-05-19; static-weight prompt-inversion baselines added 2026-05-20.

Naming convention. Two static-weight prompt-inversion attacks reuse the AloePri reference's inverter recipes against the obfuscated embedding-row surface: IMA-EmbedRow-ridge (paper §F.1's IMA basic) and IMA-EmbedRow-transformer (paper §F.1's IMA paper-like). Two activation-surface variants from the AloePri harness apply the same recipes to layer-0 hidden-state captures: IMA-L0-activation (ridge) and IMA-L0-transformer (trained transformer). All four recover τ; the surface (static W̃embed rows vs layer-0 activations) and the inverter type (ridge vs trained transformer) are explicit in the name.

Conditions under test

Condition Artifact Role
Plain (4B) plaintext Qwen3 4B Q8_0 (untied) (Qwen3-4B-Q8_0-untied.gguf) Sized to match the Obfuscated (4B) cell so plain/obfuscated comparisons stay within one model family. An attack that doesn't succeed here is broken and any defence reading is meaningless.
Obfuscated (4B) — current best utility+defense untied-keymat-h128-pi-noise-ae1.0-ah0.2-alg2-matrix-gamma-beta2-paperK-uvo-pow2e1-bf16-native.gguf (9.7 GB bf16; d=2560, h=128, β=2, true-paper-K, H=I, pow2-monomial UVO ±1). Built from Qwen3-4B-Q8_0-untied.gguf via untie_qwen3_gguf.py (Qwen3-4B ships with tie_word_embeddings: true; obfuscator's P̂R / Q̂RT transforms require a separate output.weight).

Full config (2026-05-27 calibrated Q/K + bf16-safe UVO frontier — see docs/prototype/aloepri-qk-pow2-hybrid-findings-2026-05-27.md):
  • Algorithm 1 keymat: --mode keymat --expansion-size 128 (d→dobs=2816, λ=0.3, paper §5.2.1)
  • §5.2.2 noise + Π: --pi --noise-alpha-e 1.0 --noise-alpha-h 0.2 (load-bearing for VMA / IMA-EmbedRow / TFMA / SDA)
  • Algorithm 2 — calibrated true-paper-K matrix-Γ: --alg2 --alg2-qk-norm-matrix --alg2-beta 2 --alg2-paper-literal-k (paper line-6 K side, R̂qk·Ĥqk-1·ẐblockT; Ĥ=I, no --alg2-h-hadamard-signs). β=2 is the largest tested point that preserved the n=20 HumanEval utility gate.
  • Algorithm 2 — Ûvo: --alg2-u-vo --alg2-u-vo-mode pow2-monomial --alg2-u-vo-pow2-exp 1 (bf16-commuting signed permutation + power-of-two channel scaling; avoids the dense/raw UVO bf16 cancellation loss)
  • Storage: --output-dtype bf16
  • Rejected neighbors: true-paper-K/no-R β=8 and true-paper-K β=4 failed the quality gate; β=2/no-H scored 6/20 HumanEval but gave only default-like AttnScore defence.
  • Πhead (τkv + τgroup) is dead-weight: per-head fingerprint + V/O channel-pair attacks land at random chance regardless (Alg1's rectangular keymat collapses per-head SVD spectrum before Πhead kicks in). Comes free with --alg2; no separate flag to disable.
Larger-d sibling deployment (+25 % dim headroom vs 1.7B). Current measured utility+defense compromise: (αe=1.0, αh=0.2, h=128, β=2) + true-paper-K matrix-Γ + pow2-monomial Ûvo. Why this cell: β=2/no-H preserves the small HumanEval reference (6/20) and improves paper-faithful HiddenState (3.89 % top-1), but its AttnScore defence is weak; β=4/8 variants fall off the quality cliff.
Obfuscated (8B) — current best utility+defense keymat-h128-pi-noise-ae1.0-ah0.2-alg2-matrix-gamma-beta2-paperK-uvo-pow2e1-bf16.gguf (17.4 GB bf16; d=4096, h=128, β=2, true-paper-K, H=I, pow2-monomial UVO ±1). Built directly from bartowski/Qwen_Qwen3-8B-GGUF's bf16 release — Qwen3-8B is already untied so no Path-B step needed. κ(K_d) = 7.787.

Full config (2026-05-27, identical recipe to the 4B utility+defense cell — see docs/prototype/aloepri-qk-pow2-hybrid-findings-2026-05-27.md):
  • Algorithm 1 keymat: --mode keymat --expansion-size 128 (d→dobs=4352, λ=0.3, paper §5.2.1)
  • §5.2.2 noise + Π: --pi --noise-alpha-e 1.0 --noise-alpha-h 0.2
  • Algorithm 2 — calibrated true-paper-K matrix-Γ: --alg2 --alg2-qk-norm-matrix --alg2-beta 2 --alg2-paper-literal-k (paper line-6 K side, R̂qk·Ĥqk-1·ẐblockT; Ĥ=I, no --alg2-h-hadamard-signs).
  • Algorithm 2 — Ûvo: --alg2-u-vo --alg2-u-vo-mode pow2-monomial --alg2-u-vo-pow2-exp 1 (bf16-commuting signed permutation + power-of-two channel scaling)
  • Storage: --output-dtype bf16
  • Rejected neighbors at 8B: h=256/β=2 read as Paris-then-collapse but scored 0/20 HumanEval (per-token deterministic loops); h=128/β=4 and h=256/β=4 failed the readability gate (single-token loops). The β-bifurcation explains the β cliff: with H=I, paper-literal-K differs from default-K only by Ẑ² between the two rotations, and Ẑ² = I deterministically at β=2 but ≠ I generically at β≥4 (see aloepri-h-beta-interaction-2026-05-27.md).
Larger-d sibling deployment (+100 % dim headroom vs 1.7B, +60 % vs 4B). Same recipe as the 4B utility+defense cell. Why this cell: at 8B, h=128/β=2 is the only quality- coherent (h, β) point measured (HumanEval n=20 = 8/20 = 40 % vs plain 10/20 = 50 %, −10 pp). Larger h or larger β collapses multi-step coherence into deterministic argmax loops; the β=2 cell preserves task-conditioning across 36 layers at d=4096.

Attacks under test

Three surface families, all targeting τ-recovery. Paper (Qwen3) quotes paper numbers for Qwen3 where the surface matches. Static-weight Plain (4B) cells run with τ=identity (Wplain=Wobfuscated) — they should land near 100 %, validating the attack itself.

What's covered: 4 activation-attack families (NN / IMA-L0-activation / IMA-L0-transformer / ISA) × 2 observables (HiddenState across attn_norm-{0,11,23} + Qcur_normed-{0,11,23} + Kcur_normed-{0,11,23}, plus AttnScore at kq-23; captured via two server passes — flash-attn on for HiddenState, off for AttnScore so kq-23 materialises in the compute graph). 4 static-weight prompt-inversion families (VMA, IA, IMA-EmbedRow-ridge, IMA-EmbedRow-transformer) via m2_7/run_static_attacks.py + m2_7/run_ima_embedrow_attacks.py, both loading the plain + obfuscated GGUFs and the obfuscator's .key.npz (which carries τ).

Methodology — vocab-disjoint splits (paper-faithful). IMA + ISA ridge splits partition by token id (paper §F.1 / AloePri reference's _sample_ima_splits). Train and test never share a token id, so the inverter has to generalise rather than memorise per-token bias. IMA-EmbedRow attacks use the same protocol on static embedding rows.

Threat-model framing. All 12 attacks recover τ. Static-weight attacks (VMA / IA / IMA-EmbedRow-*) recover τ from the publicly-distributed obfuscated weights; activation attacks (NN / IMA-L0-* / ISA) recover τ from captured server-side activations; wire-stream attacks (TFMA / SDA) recover τ from the obfuscated response stream. Recovering τ in any family decodes every wire-side prompt to this deployment, so the privacy claim must hold against every attack.

Attack Scope Paper (Qwen3) Plain (4B) Obfuscated (4B) Obfuscated (8B) Notes
NN
cosine top-K against the plain embedding table; surface = attn_norm-0
Recovers τ. Treat each captured obs row as a query against the plain We table and pick the top-K cosine match — if it lands on the true plain row, the token id is exposed. No training. Standard embedding-inversion baseline (Morris et al., EMNLP 2023, "vec2text"); AloePri §F.1. 0.0 % (Q2.5-14B, T1) 100.0 % @attn_norm-0 0.0 % 0.0 % Pass. Naive cosine truncated to first d dims of plain We can't bridge the (d+2h)-dim keymat rotation; no training. Plain (4B) control = identity match at L0 ✓ (cosine top-1 on plain embed table is the embedding row itself).
ISA HiddenState
multi-α ridge at layer 23; surfaces: attn_norm-23, Qcur_normed-23, Kcur_normed-23
Recovers τ. Same ridge primitive as IMA-L0-activation but at layer 23 — deep residual stream after 22 layers of attention + FFN. Paper measures at attn_norm-23; Q/K post-norm surfaces are AloePri additions to observe Algorithm 2's intra-head defence directly. Internal-state-attack family (AloePri §F.1). 0.0 % (Q2.5-14B, T1) at attn_norm-23
T3 ablation: Noise+KeyMat = 0.82 % 		attn_norm-30 row-split: 18.75 % top-1, 35.94 % top-10
attn_norm-30 vocab-disjoint: 4.81 % top-1, 25.0 % top-10
2026-05-20 plain rerun. L=21 captures failed; only L=30 measured. Description surface (L=23) is aspirational — measured surface is L=30 on the 4B (36-layer) model. 		attn_norm-21: 9.6 %
attn_norm-30: 10.6 % 		attn_norm-21: 16.3 %
attn_norm-30: 12.5 % 		4B passes; 8B marginal (l21 above gate). Alg 2's Ẑblock per-pair shuffle damps the residual signal through the layer stack. 8B's larger d gives the ridge inverter more room, lifting l21 above the 15 % gate. Note: this row uses the τ-leak attacker (deployment τ appears in training labels) and is over-powered relative to paper §3.2. See row below for paper-faithful threat-model measurement.
ISA HiddenState (paper-faithful)
multi-α ridge at attn_norm-17; K=64 attacker keymats synthesised via Algorithm 1; row-split threat-model-realistic methodology
Recovers plain_id from runtime hidden state under paper §3.2 threat model (Kerckhoffs: algorithm + plain model + own randomness, no deployment τ or K). Attacker runs plain Qwen3 locally on own plaintext prompts → captures State_plain[L]; samples K=64 own keymats; synthesises State_a^k[L][i] = State_plain[L][i] @ K_a^k; ridge trains across multi-key concatenation, forcing key-invariant inversion; tests on deployment's actual State_d. Layer L=17 ≈ 48 % depth on 36-layer Q3 matches paper's attn_norm-23/48. See docs/research/aloepri-attacks.md §ISA HiddenState for full step-by-step + layer-tradeoff space. 0.0 % (Q2.5-14B, T1)
paper Table 4 ridge ISA — methodology not specified 		15.04 % top-1 (identity-τ, K=64 single seed)
paper-faithful plain ceiling; per aloepri-keymat-variance.md single-seed K=64 has σ ≈ 5 pp pool noise 		3.89 % top-1, 20.92 % top-10
β=2/no-H true-paper-K + pow2 UVO; K=64, row split, attacker seed 20260521; prior default-UVO 3-seed mean: 8.54 % ± 4.74 		10.22 % top-1, 20.68 % top-10
β=2/no-H true-paper-K + pow2 UVO; K=64 multikey, row split, attacker seed 20260521, ridge solve fell back GPU→CPU LAPACK on OOM at d=4096×4352 — correctness-preserving. 8B HS is worse than 4B's 3.89 %, confirming d-driven Ûvo attenuation collapse (chronicle 2026-05-21). 		4B beta2/no-H passes cleanly; 8B marginal-medium. The 8B run lands at 10.22 % top-1, sliding from the 4B's 3.89 % (low) into medium-risk territory under the same recipe. Falsifies the "larger d buys more HiddenState privacy" hypothesis: at d=4096 the K=64 ridge attacker has more observed coordinates and the bf16-commuting pow2 Ûvo no longer attenuates the residual signal meaningfully.
ISA AttnScore
multi-α ridge at kq-0
 Recovers token identity per row. Ridge inversion from pre-softmax Q·KT tensor at layer 0 (captured via --flash-attn off). Surface is structurally Ka-invariant — Algorithm 1 keymat designed so W̃q's input axis cancels Ka (X̃·W̃q = X·Wq), and Algorithm 2 designed to preserve attention scores on top (Mq·MkT = I cancels). Internal-state-attack family, AttnScore variant. 0.0 % (Q2.5-14B, T1; T3 full) kq L0: 48.63 % ± 0.80
kqv_out L0: 97.46 % ± 0.40
2B.1 (2026-05-26), 512 prompts, 10-seed row-split mean, multi-α grid. kqv_out is the §5.4-bounded surface (per-head V·attention output before W_o); the L=0 row directly encodes value-projected token embedding, hence near-ceiling recovery on plain. See evals/aloepri-attacks/results/sweep/2B1-attn-output-vs-kq-comparison.md. 		kq L0: 41.97 % ± 0.81
kq L5: 34.55 % ± 0.59
kqv_out L0: 72.14 % ± 1.03
kqv_out L5: 45.05 % ± 1.10
β=2/no-H true-paper-K + pow2 UVO; first 256 prompts, 3 row-split seeds, GPU ridge 		kq L0: 48.56 % ± 0.67 (plain 49.02 % ± 0.61, Δ +0.46 pp)
kq L5: 39.59 % ± 0.65 (plain 39.52 %, Δ −0.07 pp)
kqv_out L0: 82.26 % ± 1.38 (plain 93.02 %, Δ +10.76 pp)
kqv_out L5: 49.44 % ± 1.22 (plain 50.23 %, Δ +0.79 pp)
β=2/no-H true-paper-K + pow2 UVO; 512 prompts, 3 row-split seeds, ROCm rocSOLVER ridge 		Fail; calibrated tradeoff exposed at both model sizes. On 8B, kq defence is essentially zero (Δ +0.46 pp L0, −0.07 pp L5) — predicted exact by the β-bifurcation: paper-literal-K with H=I differs from default-K only by Ẑ² between the rotations, and Ẑ² = I deterministically at β=2 (every window σ ∈ S2 is involutive). Ûvo pow2-monomial drives +10.76 pp at kqv_out L0 but vanishes by L5. Both absolute obf TTRSR figures stay in the 48–82 % band — far above any "defended" threshold. To defend kq at all we need to change the K-side construction kind, not just the magnitude (see aloepri-h-beta-interaction-2026-05-27.md §4).
ArrowMatch
chunked cosine direction matching on token_embd; paper §3.2 "Game of Arrows" (Wang et al., USENIX Sec '25, 2026-05-27 port)
Recovers τ from (Wplain, Wobfuscated) via direction similarity. For each obfuscated embedding row w̃i, find σ̂(i) = argminj cos(w̃i, wj) over all plaintext rows; recovery rate = how often σ̂(i) = τ−1(i). Stage 2 length-adjustment skipped (broadcast bug on active-vocab vs full-vocab shape — defers a 1-line patch). Paper §3.2 Obs2: matrix-multiplication obfuscation is immune to this attack; only per-vector ops (Π+D) leak direction signal. AloePri's W̃ = Π · W · Q̂ + ε uses matrix-mult Q̂, so the attack is bounded by construction. Attack driver: run_arrowmatch.py. novel (path-2 port; paper §3.2 framing) 98.58 % top-1
98.59 % top-10
identity-τ control, cosine at correct = 1.000; 1.4 % residual gap from duplicate / near-duplicate embed rows in Qwen3-4B's tokenizer 		0.00 % top-1
0.008 % top-10
β=2/no-H true-paper-K + pow2 UVO h128; cosine at correct mean = −0.0001 ± 0.020 (statistically zero); 100 s on ROCm 		0.00 % top-1
0.01 % top-10
β=2/no-H true-paper-K + pow2 UVO h128 (8B d=4096 → dobs=4352); plain control 100.00 % at cosine 1.000; cosine at correct mean = 0.0002 (statistically zero); 177 s on ROCm 		Pass at both sizes — structurally defeated by construction. Plain control hits 98.58 % (4B) / 100.00 % (8B) at cosine 1.000, confirming the attack works on per-vector obfuscations. On the obfuscated cells, cosine alignment between corresponding plain/obf rows collapses to noise (mean ≈ 0, p10/p90 ≈ ±0.025) — AloePri's matrix-multiplication keymat Q̂ zeroes the direction signal exactly as Obs2 anticipates. No follow-up needed unless future Alg2 variants drop the matrix-mult keymat structure. The 4B vs 8B parity (0.00 % → 0.00 %) shows the result is d-invariant — the structural argument doesn't depend on dimension.
VMA
RowSort weight-pair recovery — AloePri §F.1 Table 8
Recovers τ from (Wplain, Wobfuscated). Row-sort paired weight products (WeWhead, WeWq, WeWgate, …) then neighbour-match rows; multi-source voting across 17 (layer × projection-kind) sources via sorted-quantile features. RowSort recovery of permutations from Y = Z₁XZ₂ structure (Thomas et al., ICML 2025, "Hidden No More"). 13.51 % (Q2.5-14B, T1)
25.05 % (Q3 14B, T2)
19.64 % (Q3 32B, T2) 		98.44 % (identity-τ, 3-seed) 8.98 % top-1
23.44 % top-10
pow2 UVO h128 β8; static harness, n_test=256 		1.17 % top-1
3.52 % top-10
β=2/no-H true-paper-K + pow2 UVO h128; static harness, n_test=256, 36 layers × 7 projection kinds 		Pass. Static RowSort remains well below the 15 % top-1 gate on the pow2-UVO cell at both sizes. The 8B figure is lower than 4B (1.17 % vs 8.98 %) because the larger d shifts more rows out of the sorted-quantile voting band.
IA
Gate-IA + Attn-IA invariants — AloePri §F.1
Recovers τ from weight invariants that survive obfuscation. Gate-IA: Avg(eWgate) = Avg(ẽW̃gate). Attn-IA: per-block quadratic form e(QTQ)−1eT. Reported metric: max(Gate-IA, Attn-IA) per paper Table 1. Invariant-attack family (Lin et al., EMNLP 2024 — same author group as AloePri). 5.95 % (Q2.5-14B, T1) 98.5 % (identity-τ) 0.05 % top-1 / 0.20 % top-10 0.07 % top-1 / 0.27 % top-10
gate-IA 0.07 % / attn-IA 0.02 % (β=2/no-H true-paper-K + pow2 UVO h128; 4096 test tokens at L0) 		Pass. IA is at floor on the β=2/no-H pow2-UVO cell at both sizes; rectangular Alg1 keymat already breaks the gate/attention invariants. 8B and 4B numbers are indistinguishable within noise.
Per-head fingerprint Q/K/V/O
per-attention-head top-16 SVD spectrum signature; static weight attack (new 2026-05-25)
Recovers τkv ∈ S8 and τq ∈ S32 per layer — the inter-head shuffle from Algorithm 2 (paper §5.2.3 step 5). For each plain attention head i ∈ {Q, K, V, O}, compute the top-16 singular values of W[head_i] reshaped to (head_dim, d_residual). NN match obfuscated → plain heads by L2 distance on the SV vector. SVs are right-mult-invariant under R̂qk + Ĥqk±1 + Ûvo, so the signature targets Πhead specifically. AloePri attack driver: run_per_head_fingerprint.py. novel (aloepri only) 100.0 % (identity perm — trivial) Q: 4.25 % (≈ random 1/32 = 3.13 %)
K: 13.54 % (≈ random 1/8 = 12.5 %)
V: 12.50 % (= random 1/8)
O: 3.21 % (≈ random 1/32) 		Q: 3.73 % (≈ random 1/32 = 3.13 %)
K: 13.19 % (≈ random 1/8 = 12.5 %)
V: 13.89 % (≈ random 1/8)
O: 3.21 % (≈ random 1/32)
β=2/no-H true-paper-K + pow2 UVO h128, 36 layers 		Pass at random chance — but Alg 1 alone defeats. All four surfaces land within 1 σ of uniform-random guessing, which means zero detectable information leak about τkv or τq. Designed to target Πhead, but Alg 1's rectangular keymat (d=2560 → dobs=2816) already collapses per-head SVD spectra before Πhead kicks in. Confirmed by Alg 1-only cell (no Πhead): same random-chance numbers. Key-space `(8! · 4!)36 ≈ 10214` — intractable to brute-force. K/V at 1/8 because Qwen3 uses GQA (32 Q-heads share 8 K/V-heads).
V/O channel-pair V/O
per-head V row + O column L2 magnitudes; static weight attack (new 2026-05-25)
Recovers τkv + τq AND probes Ûvo detectability. For each plain V-head: L2 row norms across head_dim. For each plain O-head: L2 column norms within the O-head's column range. NN match plain → obfuscated by L2 distance on magnitude vectors. Three sub-attacks: V-match, O-match, and (V, O) joint under GQA grouping. Magnitudes are preserved by all Alg 2 components EXCEPT Ûvo (QR+0.05 σ Gaussian perturb shifts per-head channel L2) → signature designed to expose Ûvo. AloePri driver: run_vo_channel_pair.py. novel (aloepri only) 100.0 % (identity perm — trivial) V: 12.50 % (= random 1/8)
O: 3.12 % (= random 1/32)
(V, O) pair: 3.12 % (≈ random — Q-side is the bottleneck) 		V: 12.50 % (= random 1/8)
O: 3.21 % (≈ random 1/32)
(V, O) pair: 3.82 % (≈ random)
β=2/no-H true-paper-K + pow2 UVO h128, 36 layers 		Pass at random chance — but Alg 1 alone defeats. Same Alg 1-collapsing mechanism as per-head fingerprint. Ûvo's per-head magnitude perturb is shadowed by Alg 1's uniform d→dobs shift across all heads — attacker can't distinguish Ûvo's perturb from the keymat's uniform widening. (Ûvo still contributes to VMA via column-mixing — different mechanism, different attack.)
IMA-EmbedRow-transformer
trained 2-layer 8-head Qwen2 inverter on (plain_embed, synthetic-obfuscated_embed) pairs from a public corpus — paper §F.1 canonical IMA
Recovers τ from static embedding rows under the no-τ-leak threat model. Attacker has the obfuscation algorithm (Kerckhoffs), runs it with their own pseudo-τ on a public corpus to generate synthetic pairs, trains a 2-layer transformer inverter, applies it to the actual deployment's W̃[k]. Paper §F.1 reference attack. 0.0 % (Q2.5-14B, T1)
single-key (weaker) 		13.5 % @ep=100 (identity-τ, still rising) 0.00 % top-1
0.00 % top-10
pow2 UVO h128 β8; 2-epoch paper-like transformer 		0.00 % top-1
0.00 % top-10
β=2/no-H true-paper-K + pow2 UVO h128; n_train=4096 paper- like transformer (Qwen-backbone, 2-layer, 8-head) on 128 public-corpus sequence windows × 32 tokens; best_epoch=0 (no learning signal) 		Pass at both sizes for the paper-faithful trained inverter. Both 4B and 8B paper-literal-K + pow2-UVO cells land at 0.00 % top-1/top-10. The previous "3.13 % multi-key" figure was the H128 β8 cell; the current utility+defense cell (β=2/no-H) saturates the paper attack at zero.
QK-norm Γ eigendecomposition
eigendecompose blk.*.attn_q_norm.weight / attn_k_norm.weight; AloePri matrix-Γ deployment, 2026-05-19
Recovers the intra-head transform M (and thus τ at the attention sites). Server with code execution reads the 2D Γ tensors at the QK-norm sites and runs numpy.linalg.eig(Γ): because Γ = MT·Diag(γqk)·M is a similarity transform of a diagonal matrix, eigenvalues recover γqk and eigenvectors recover M up to head_dim permutation in ms; defeats R̂qk + Ẑblock + Ĥqk at every layer. AloePri-specific surface (no paper analogue); docs/research/aloepri-qk-norm-matrix-gamma-threat-model.md. novel n/a not measured · not defended not measured · not defended (kernel identical) Not defended by design. Γ = MT·Diag(γ)·M is similarity-transformed of a diagonal — numpy.linalg.eig(Γ) recovers M (eigenvectors) + γ (eigenvalues) in ms. Out of AloePri's static-obfuscation scope by construction; fix requires per-prompt fresh M (GELO; breaks the no-infra-change thesis) or σ·N noise on Γ tuned to confound eig (hard: γ_q eigenvalue spread 1e-1 ... 7e1, no single σ works across heads).
TFMA
token-frequency matching on the response stream
Recovers τ from the wire-side response stream. Count obfuscated-token-id frequencies in returned generations; match against a prior corpus's plain token-frequency distribution. AloePri §F.1 + Table 4. 0.14 % zero-knowledge
3.19 % distribution-aware
(Q2.5-14B, T4) 		0.0 % top-1 (identity-τ, no permutation to recover) 0.78 % top-1 / 4.69 % top-10 0.4 % top-1 / 4.3 % top-10 Pass. Pow2 UVO does not matter here; Π/token permutation keeps frequency matching below gate on the 64-prompt stream.
SDA
substitution-cipher decipherment on the response stream
Recovers τ from the wire-side response stream. Train a transformer on (plain, cipher) pairs converted to integer recurrence-rank sequences; translate fresh ciphertext. Substitution-cipher decipherment family (Aldarrab & May, EMNLP 2021); AloePri §F.1 + Table 4. BLEU-4 0.01 zero-knowledge
BLEU-4 2.10 distribution-aware
(Q2.5-14B, T4) 		BLEU-4 3.5×10−4 (identity-τ, no cipher) BLEU-4 1.28×10−5 BLEU-4 1.6×10−5 Pass. SDA remains at near-zero BLEU on the 64-prompt stream.
Acceptance gates — click to expand per-gate ledger

Paper §6.3 recommends TTRSR ≤ 15 % per attack at the default hyperparameters. Each obfuscated-side gate pairs with a plain-side control: an attack that doesn't succeed on plain inference is a broken attack, not a defence signal. Results are against the current obfuscated deployment (R̂qk + Ẑblock + ±1 Ĥqk + Πhead).

Gate Threshold Result How to read
nn_obfuscated_below_15pct NN ≤ 15 % ✓ 0.0 % Naive-truncation attacker can't recover anything from the keymat-rotated 2304-dim obs.
isa_hidden_state_below_15pct
(attn_norm-23 surface) 		ISA HiddenState ≤ 15 % ✓ 3.89 % Algorithm 2's main observable win on the residual stream. Per-pair Ẑblock permutation propagates obfuscation through 22 layers of attention into the deep residual, eroding the keymat decoy structure that ridge had been exploiting.
isa_attn_score_below_15pct
(kq-23 surface) 		ISA AttnScore ≤ 15 % ✗ 41.97 % Fails under the 2026-05-27 beta2/no-H utility cell. Stronger Q/K perturbations lower AttnScore recovery, but β=4/8 variants fail the quality gate.
vma_below_15pct VMA ≤ 15 % ✓ 0.0 % RowSort + voting across 17 sources finds zero signal on FULL-zfix-hadamard. Better than every Qwen3 size in the paper (Q3 14B 25 %, Q3 32B 20 %). Pass.
ia_below_15pct IA ≤ 15 % ✓ 0.0 % Gate-IA + Attn-IA invariants both collapse to chance under αe=1.0 + Algorithm 2 head/block permutations. Better than paper Q2.5-14B (5.95 %). Pass.
ima_embedrow_transformer_below_15pct
(static embedding-row surface, paper §F.1 trained Qwen2 inverter on public-corpus pairs) 		IMA-EmbedRow-transformer ≤ 15 % ✓ pass — 3.13 % (4B, 8B) Paper's canonical IMA under the no-τ-leak threat model. Multi-key paper-faithful attacker (K=64 attacker-generated keymats per batch + fresh noise); best ep=15 (4B) / 10 (8B). Driver: evals/aloepri-attacks/m2_7/run_ima_embedrow_attacks_multikey.py.
tfma_below_15pct TFMA ≤ 15 % ✓ 0.8 % Independent of Algorithm 2 (acts on token stream, not hidden states). Passed since §05.
sda_below_bleu4_5 SDA BLEU-4 ≤ 5 ✓ 1.4×10−5 Same as TFMA — token-stream surface, unaffected by Algorithm 2.
nn_plain_at_least_95pct NN ≥ 95 % (plain control) ✓ 100.0 % (4B) Plain-side sanity. Cosine attack on plaintext early-layer hidden state must succeed for the obfuscated number to be meaningful.

Headline reading. Under paper's no-τ-leak threat model, the calibrated beta2/no-H utility cell passes ISA HiddenState (3.89 %), NN, IA, IMA-EmbedRow-transformer, TFMA, and SDA, but ISA AttnScore remains the open failure at 41.97 % on kq L0. The Q/K defense frontier is steep: β=2 preserves utility but loses most AttnScore defense, while β=4/8 variants fail quality. Linear-ridge variants on the embedding-row surface (IMA-EmbedRow-ridge, IMA-L0-activation) were retired — they require a deployment-τ partial leak that AloePri's wire-side TFMA defence makes unrealistic; see docs/research/aloepri-attacks.md.

Two reading principles

1 — TTRSR per attack, not TTRSR overall. Each attack probes a different observable. Static-weight attacks (VMA / IA / IMA-EmbedRow-*) recover τ from the publicly-distributed obfuscated weights; activation-surface attacks (NN, IMA-L0-*, ISA) recover τ from captured server-side forward-pass activations; wire-stream attacks (TFMA, SDA) recover τ from the obfuscated response stream. The acceptance gate is per-attack — a low average can hide one high-leaking surface, and all three families decode every wire-side prompt once τ leaks. The "How to read" column above states what each attack's number specifically asserts.

2 — Plain-side TTRSR matters as much as obfuscated TTRSR (when applicable). Activation attacks must succeed against plain inference to be meaningful as defence tests. If NN doesn't hit ~100 % on plain we have an attack-implementation bug; if it does and ≤ 5 % on the obfuscated artifact, we have a real defence signal. The "n/a (attack requires θ̃)" rows (VMA / IA / IMA-EmbedRow-* / TFMA / SDA) are honest: those attacks require the obfuscated tensors or the τ-permuted stream to even apply, so plain has no meaningful number — not a bug, just the threat-model boundary.

Implementation: evals/aloepri-attacks/. Operator runbook: evals/aloepri-attacks/README.md. The wire-side follow-up is a capture binary that drives llama-server against the obfuscated GGUF, records wire-side observables, and feeds them to the same Python attack drivers.

§09

Status & gaps

what landed · what's partial · what's open
Component Status Notes
Π token-level permutation + client wrapper (paper §5.2.2) done τ ∈ S151 669, padding slots [151 669, 151 936) stay identity. Client uses native /completion with int-array prompt + return_tokens; no tokenizer roundtrip on the wire. Smoke: "What is the capital of France?" → coherent answer.
Algorithm 2 attention obfuscation (paper §5.2.3) deployed (matrix-Γ kernel, 2026-05-19) τkv + τgroup head-shuffle + R̂qk (RoPE-aware rotation per NEOX pair) + Ẑblock (β-wide locality-preserving permutation) + Ĥqk (±1 Walsh-Hadamard, keeps Mq orthogonal for the matrix-Γ algebra). Not deployed: distinct Ẑblock_η for the K side (uses same Z as Q with repaired Mk construction). Per-head distinct Mq[h] not deployed (shared Mq per layer; would require batched matmul in the kernel). See docs/handoffs/2026-05-19-option-c-steps-0-1-2a-findings.md for the full ramp.
αe / αh Gaussian noise on embed + head (paper §5.2.2) done αe=1.0, αh=0.2 (paper defaults). σ_e=σ_h=0.0345 (W_e and W_h are byte-identical in the source GGUF; noise is sampled independently for each). Generation stays coherent under noise.
Attack-resistance benchmark (NN, IMA-L0-activation, ISA HiddenState, ISA AttnScore, VMA, IA, IMA-EmbedRow-ridge, TFMA, SDA — see §08) measured · most gates pass at strongest utility-preserving condition · ISA AttnScore remains open See §08 for the full ledger. Pass: NN, ISA HiddenState (3.89 % on the beta2/no-H utility cell), VMA (8B), IA, IMA-EmbedRow-transformer (3.13 % on both Q3-4B and Q3-8B, multi-key paper-faithful attacker), TFMA, SDA. Marginal: VMA (4B) ≈ 18 %. Fail/open: ISA AttnScore on the utility-preserving beta2/no-H cell (41.97 % at kq L0), plus retired IMA-L0-activation and IMA-EmbedRow-ridge probes on the embedding-row surface. Stronger Q/K perturbations improve AttnScore but currently fail quality.
Production-quantisation validation done (verdict: fp32) Q8_0/Q6_K/Q5_K_M all fail (heavy-tailed obfuscated weights · per-block scaling rounds outliers to zero).
Determinism verification done Byte-identical replicates on Vulkan iGPU + flash-attn + np=1 + temperature 0 + seed 0.
Mini accuracy benchmark partial (3 of 4 tasks) MMLU / PIQA / HumanEval measured (see §07). IFEval deferred.
Gemma 4 backbone support deferred 5 residual norm sites per block (vs Qwen3's 2) push κ-compounding past comfortable accuracy bounds; needs the per-site-κ tuning research below first. See §04.
Streaming / EOS handling deferred Server's EOS detection looks for the plaintext EOS ID; after Π, the model emits τ(eos) which the server doesn't recognise. The current implementation relies on the n_predict bound. Either pass a stop array containing the obfuscated EOS, or accept the cutoff.

The Qwen3 QK-norm gap and how it was closed (matrix-Γ, 2026-05-19)

Original problem. R̂qk is a block-diagonal 2D rotation per RoPE pair. To commute with RoPE, it must be applied to the projection output before RoPE. Qwen3 has QK-norm between the projection and RoPE — a per-head RMSNorm with per-head_dim γ vectors. R̂qk does not commute with per-element γ. The §5.2.5 fold trick (replace γ with scalar κ and fold the per-element factor backward into the linear weight) is exact only under i.i.d. Gaussian-Q assumption; trained Qwen3 Q/K vectors correlate with γ, so per-input κ ≈ √(mean(γ2)) corrupts attention enough that the model degenerates. The §05 deployment therefore left intra-head dormant.

The matrix-Γ kernel extension. Instead of folding γqk backward, the patched llama.cpp kernel keeps the QK-norm op and consumes a 2D tensor Γ = MT·Diag(γqk)·M in place of the original 1D γqk at blk.*.attn_q_norm.weight / blk.*.attn_k_norm.weight. Algebraically (Q / RMS(Q)) · Γ = Qplain_normed · M when M is orthogonal (exact to fp32 noise). The obfuscator builds 2D Γ per layer; the kernel detects via the aloepri.qk_norm_matrix metadata flag and branches the graph to a 128×128 ggml_mul_mat at q_norm. Storage cost +42 MB / fp32 model; compute cost <1 % at decode.

Sub-fixes landed alongside. (1) generate_block_perm in lib/alg2.py was rewritten as fixed β-wide windows after diagnosing that the dynamic-window softmax always collapsed to size-1 windows under the paper's default params, silently producing the identity permutation. (2) The reference's k_matrix = R · H−1 · Z−1 construction only collapses Mq·MkT to I when Z2 = I — the identity-Z degeneracy silently provided this. Repaired to k_matrix = R · H−1 · Z, which gives Mq·MkT = I for any orthogonal Z. (3) Ĥqk uses ±1 Walsh-Hadamard signs (involutive, preserves orthogonality). See docs/handoffs/2026-05-19-alg2-z-block-degeneracy.md.

What the ramp closed. ISA HS at attn_norm-23 drops 16.3 % → 11.5 % when Ẑblock lands (passes the §6.3 15 % gate). ISA HS at the Q-surface (Qcur_normed-23) tightens 12.5 % → 9.6 % under +R̂qk. Algorithm 2's defence is measurable on the surfaces where it acts.

What the ramp does not close. IMA-L0-activation stays at 88.9 % across every Algorithm 2 build variant because attn_norm-0 is computed pre-Wq — Algorithm 2's intra-head Mq acts post-Wq and structurally cannot reach this surface. This is an AloePri-specific attack the paper does not measure; closing it requires Algorithm 1 / §5.2.2 levers, not Algorithm 2.

Reference-code state. The public reference (sheng1feng/Aloepri @ 60e8ea3) has no QK-norm handling — all attention paths import transformers.models.qwen2.modeling_qwen2, and Qwen2.5 has no QK-norm. Paper §7.1 explicitly lists Qwen3 in its evaluated models, so ByteDance's internal codebase presumably has a fix; the academic release does not, and the matrix-Γ extension here is one viable way to bridge it.

Remaining levers (for IMA-L0-activation specifically).

Other open questions

Why our IMA-EmbedRow-ridge = 99.22 % diverges from paper's ≈ 0 % — dimensional headroom (2026-05-20)

Setup. Paper Table 2 reports IMA TTRSR = 0.0 % on Qwen2.5-14B at the default config (d=5120, αe=1.0, h=128, λ=0.3). Our measured IMA-EmbedRow-ridge on Qwen3-1.7B at the aloepri default (d=2048, αe=0.1, h=128, λ=0.3, Algorithm 2 enabled with matrix-Γ) is 99.22 %. The 4 % vs 99 % gap is not an obfuscation bug — it is a dimensionality argument inherent to the attack and the architecture.

The attack. IMA-EmbedRow-ridge fits a single linear map V ∈ ℝ(d+2h)×d such that V·W̃e[τ[i]] ≈ We[i] on a held-out vocab split. Training pairs come from comparing the plain GGUF's embedding rows against the obfuscated GGUF's embedding rows — the paper's own threat model (§3.2: "the attacker can access the model") grants this. Closed-form ridge solution exists; the only thing standing between the inverter and τ-recovery is the per-row noise αe·σe·ε and the conditioning of P̂.

Why high d helps the defender. The per-element noise std is αe·σe; the per-row noise norm is αe·σe·√d. The signal in We concentrates in a low-rank "useful" subspace (semantic directions actually used by attention); noise spreads uniformly across all d directions. Higher d means more "garbage directions" the noise can dilute into without corrupting the directions the model — and the inverter — actually use. The effective SNR in the attack-useful subspace falls as d grows, for fixed αe.

The numbers. Paper noise norm per row at d=5120, αe=1.0: 1.0 · σ · √5120 ≈ 71.5 · σ. Our aloepri default at d=2048, αe=0.1: 0.1 · σ · √2048 ≈ 4.5 · σ. Ratio is ~16× weaker per row, in 2.5× fewer dimensions — net effective attack-utility ratio on the order of 25× weaker noise scatter. Predicted outcome: ridge inverts ~cleanly. Measured outcome: top-1 99.22 %, top-10 100 % at our config. Both the prediction and the measurement land where the dimensional argument places them.

Why we cannot just raise αe to paper's 1.0. At d=2048, αe=1.0 destroys generation accuracy (HumanEval pass@1 collapses from 50 % plain → ≪ 5 %, with multi- language gibberish on hard prompts; see the 2026-05-20 ablation ledger). We measured the noise/accuracy tradeoff and settled at αe=0.1 as the accuracy ceiling — but that's structurally insufficient for IMA-EmbedRow defence at this d.

Where Algorithm 2 fits (and doesn't) in this picture. Algorithm 2's intra-head + inter-head transforms only touch Wq,k,v,o. They do not transform We; they cannot affect this attack regardless of whether the matrix-Γ kernel is deployed. The §08 measured ledger confirms: IMA-EmbedRow-ridge is 99.22 % both with and without Alg 2.

Structural paths that would close the gate.