python -m demo.plan_pytorch_model \
--model bert --network 10Mbps/1ms
# on the server cipherflow netbench --serve # on the client python -m demo.plan_pytorch_model \ --model bert --ip SERVER_IP
NeurIPS 2026 · Apache-2.0
CipherFlow: Compiling AI Programs for Efficient Cryptographically Secure Inference
CipherFlow takes a high-level AI computation graph, available cryptographic primitives, and a target hardware and network configuration to construct a latency-optimized hybrid secure inference plan.
Programming usability. AI developers and data scientists express model computations while the compiler selects HE/MPC operators, conversions, state alignment, and ciphertext maintenance.
Execution efficiency. MicroBench profiles deployment costs; state-expanded planning jointly optimizes feasible execution decisions; and BlockFold scales planning across branched Transformer graphs.
Interactive demoPlan BERT-base for your server and network
1Server
RTT 40 ms
3The planblock 1 of 12, top to bottom · hover or tap an operator
Plans by cipherflow.compile with the paper's profiles (HE: NEXUS; MPC: SHAFT, two parties and a trusted dealer). Latency is the whole 12-block model, estimated from measured costs.
Results
Performance
16.5×
speedup on BERT-base over state-of-the-art static hybrid placements (BumbleBee, BOLT, BLB)
22×
speedup on ViT, measured on commodity CPU and GPU platforms in the NeurIPS 2026 evaluation
vs static strategies
vs other compilers
Compare with
Explore
From the idea to your first plan
How it worksFrom a PyTorch model to a placement planThe stack, the planner's three ideas, and the measured HE and MPC backends.Read how it works →
Get startedInstall CipherFlowPick your setup for the install command and a first plan, then watch a plan being made in the terminal.Get started →
TutorialsPlan a model, then bring your ownFive steps from a PyTorch model to a checked plan, and three ways to bring your own kernels, primitives and applications.Open the tutorials →
RoadmapFrom encoders to generative modelsWhat is done, what comes next, and the operators decoders still need.See the roadmap →
How it works
From a PyTorch model to a placement plan
CipherFlow plans against an abstraction of the backends: methods, each one operator in one domain with the CKKS levels it consumes, the inputs it accepts and its measured cost. A library plugs in by registering its methods and a cost profile, or by declaring its primitives.
Model
PyTorch model → operator graph
nn.Module, e.g. demo/transformer.py (BERT-base, ViT-B/16), traced by cf.models.from_torch into operators and residuals①graph G = (V, E)operators · tensor shapes · residual edges
Planning
②Deployment D = (HW, BW, RTT)
- HW:
cpu|gpuprofile, or this machine (MicroBench) - BW/RTT:
--network 10Mbps/1ms, or measured:--ip SERVER - security:
2pc|2pc-dealer
optCalibrationvalue ranges on real inputs, so approximations hold where HE runs
CipherFlowplanner
- MicroBenchmeasures missing costs
- State expansiondomain × CKKS level
- BlockFoldfolds residual blocks
plan = cf.compile(graph, deployment, profile, calibration=None)
returns plan.json · plan.explain(steps=True) · plan.steps()
④plan.jsoneach operator: HE | MPC, level, kernel; each edge: conversions, bootstraps
③methods + measured costsper operator and domain: time, bytes, rounds, CKKS levels used
Backends
HE and MPC libraries
HE ⇄ MPC conversions (hardened for OpenFHE) · bootstrapping
HE: OpenFHE · NEXUSOpenFHE CKKS (CPU)NEXUS: SEAL (CPU), Phantom (CUDA)
MPC: SCI · SHAFTSCI/BOLT: 2PC (CPU)SHAFT: 2PC + dealer (CPU, GPU)
runs the plan
Execution
cipherflow.runtime: the plan on real inputscheck_plan (domains, levels) · calibrate (value ranges)emulate: error vs PyTorch, GLUE accuracy (no cryptography)encrypted two-party execution: planned (ROADMAP §6)
Three ideas behind the planner
MicroBench
Costs measured where you deploy
Every operator, conversion and bootstrap is timed on the target hardware, with its bytes and rounds, so the plan follows measured costs instead of a fixed rule.
State expansion
One search over domains and levels
Each operator's state is its domain and, in HE, the CKKS levels left. Placement, conversions and bootstraps become one shortest-path search over these states.
BlockFold
Residual blocks folded into steps
Each residual block is a single-entry, single-exit region. Folding it into one step contains the growth from branching joins and keeps planning tractable on Transformer-scale graphs.
Backends
Measured HE and MPC backends
Each backend reports its methods and the measured cost of every operator, conversion and bootstrap. CipherFlow plans with each pair of backends that fits the deployment and keeps the best plan.
Homomorphic encryption
- ✓OpenFHE CKKSAll operators on one 128-bit parameter set, fast and precise bootstrapping, hardened HE↔MPC conversion, wide-range GeLU, Softmax and LayerNorm
- ✓NEXUSSEAL on CPU, Phantom on CUDA; reference-scaled costs
Secure multi-party computation
- ✓SCI / BOLT (EzPC)Two parties, no third party; CPU
- ✓SHAFT (CrypTen)Two parties and a trusted dealer; CPU and GPU
- ○SPUCheetah and SEMI2K; planned
Your libraryDeclare its primitives and what one call costs with
NextInstall CipherFlow and plan your first modelGet started →
cf.PrimitiveBackend; CipherFlow lowers every operator to them.Bring your own primitives →Get started
Install CipherFlow
Choose how you install and what you want to plan. The commands follow your choices.
Package
Model
Interface
Server
Network
Run this command
Then plan a model
Notes
Demo
A Transformer in, the plan out
Plan a BERT-base encoder written in PyTorch and read the plan operator by operator: HE or MPC, the CKKS levels, the conversions and bootstraps before each operator, and where each operator is in the code.
Run from the repository root, after installing from source.
Tutorials
Plan a model, then bring your own applications & primitives
Two walkthroughs with real code and the output it prints. The first plans a BERT-base encoder step by step; the second brings kernels, primitives and applications of your own.
Tutorial 1
Plan a model, step by step
A plan says, for every operator, whether it runs in HE or MPC, how many CKKS levels it uses, and what converts or bootstraps before it. This tutorial plans the demo's BERT-base encoder. Install from source and run it from the repository root.
-
Step 1: Trace the model
from_torchtraces your encoder with torch.fx and maps it onto CipherFlow's operators: the Q/K/V projections, QKᵀ, Softmax, Attn·V, the linear layers, LayerNorm, GeLU and the residual adds. Each operator keeps its place in your code.An operation outside this set raises
cf.UnsupportedModel, which names each one.import torch import cipherflow as cf from demo.transformer import bert # BERT-base in plain PyTorch encoder = bert() graph = cf.models.from_torch(encoder, torch.zeros(1, 128, 768)) for node in graph.nodes[:4]: print(node.op_type, node.input_shape, node.attributes["source"])
OutputInput (1, 128, 768) input x Linear_QKV (1, 128, 768) blocks.0.attn.{q,k,v} Attention_QK_MatMul (3, 1, 128, 768) blocks.0.attn: matmul Softmax (1, 12, 128, 128) blocks.0.attn: softmax -
Step 2: Describe the deployment
A deployment is the server's hardware and the network between the client and the server.
cpuandgpuselect the built-in cost profiles. Give the network as bandwidth / round-trip time, or measure it withcipherflow netbench.cf.matchshows which HE and MPC backend pairs can plan the model there. The default security model,2pc, allows two parties and no third party.# hardware profile; network as bandwidth / round-trip time deployment = cf.Deployment.of("cpu", "10Mbps/1ms") print(cf.match(graph, deployment)) # HE + MPC pairs that fit
Outputbackends for torch_BertEncoder: nexus+sci ok 30/30 cost entries exact; HE costs reference-scaled openfhe-fast+sci ok 30/30 cost entries exact openfhe-precise+sci ok 30/30 cost entries exact nexus+shaft excluded security model 2pc-dealer (two parties + trusted dealer) is not allowed by security='2pc'; pass security='2pc-dealer' to allow it … -
Step 3: Compile
compileplans the model with each backend pair that fits and keeps the best plan. It searches each operator's domain and CKKS level together, so it places conversions and bootstraps where they cost least.plan.alternativeslists the other pairs and their latencies.plan = cf.compile(graph, deployment) print(plan.explain())
OutputCipherFlow plan graph=torch_BertEncoder hardware=cpu BW=10 Mbps RTT=1 ms strategy=compiler latency 25262.7 s (operators 18549.8 s, conversions 1702.4 s, bootstraps 5010.5 s) placement: 132 HE / 12 MPC operators, 24 conversions, 59 bootstraps, 16.47 GiB online communication HE backend: openfhe-fast (OpenFHE CKKS, one-pass bootstrapping; 10.5-bit bootstrapping; costs measured) MPC backend: sci (SCI/BOLT (EzPC); two parties, no third party) … -
Step 4: Read the plan
explain(steps=True)lists the operators in execution order. Each row gives HE or MPC, the CKKS level before and after, and the line of your PyTorch code. The conversions and bootstraps that run before an operator sit above it.Here Softmax runs in MPC, between two conversions, and everything else stays encrypted.
print(plan.explain(steps=True))Output# operator domain CKKS level PyTorch 0 Input HE 8 -> 8 input x (encrypted by the client) 1 Linear_QKV HE 8 -> 7 blocks.0.attn.{q,k,v} 2 Attention_QK_MatMul HE 7 -> 3 blocks.0.attn: matmul convert HE -> MPC [1, 12, 128, 128] (#2 -> #3) 17.72 s 3 Softmax MPC blocks.0.attn: softmax convert MPC -> HE [1, 12, 128, 128] (#3 -> #4) 124.15 s 4 Attention_V_MatMul HE 7 -> 3 blocks.0.attn: matmul 5 Out_Projection HE 2 -> 1 blocks.0.attn.o 6 Residual_Add HE 1 -> 1 blocks.0: add bootstrap at level 1 (#6 -> all its consumers) 52.74 s 7 LayerNorm HE 4 -> 1 blocks.0.ln1 bootstrap at level 1 (#7 -> #8) 52.74 s 8 FFN_Linear_1 HE 8 -> 7 blocks.0.mlp.fc1 bootstrap at level 7 (#8 -> #9) 210.97 s 9 GeLU HE 8 -> 2 blocks.0.mlp.act … -
Step 5: Save it and check it on real data
plan.jsonholds the whole plan for other tools.emulateruns the model in float64 with each operator in its planned domain: the HE kernels' approximations, fixed point in MPC, and each conversion's rounding. It runs at plaintext speed and does not run the cryptography.Calibrate on real inputs with
cf.runtime.calibrate, and pass the calibration tocompile. Operators whose values fall outside an approximation then stay out of HE.plan.to_json("plan.json") # cipherflow explain plan.json --steps x = torch.randn(1, 128, 768) # hidden states of a real input result = cf.runtime.emulate(plan, encoder, x) print(result.summary())
Outputoutput vs plaintext: max |err| 0.017, relative 0.00248 conversions 24, bootstraps 59 executed (plan: 24 / 59)
Tutorial 2
Bring your own applications & primitives
CipherFlow is not tied to the models and backends it ships with. Bring a faster kernel, a library's primitives, or an application with a step of its own; declare what one call costs, and the planner places it as it does the built-in ones. Each card shows the layer your code goes into. The examples continue the tutorial above: graph is BERT-base and deployment is a CPU server at 10 Mbps / 1 ms; each has a demo to run from the repository root.
Run the demo
python -m demo.extensions.new_he_primitive
ext = cf.Extension("my-he-softmax") ext.primitive("Softmax", "HE", levels=4, # CKKS levels one call consumes cost=cf.Cost(local_ms=31_000), # one call on scores [1, 12, 128, 128] accepts=lambda node: None if node.input_shape[-1] == 128 else "rows of 128 scores only") plan = cf.compile(graph, deployment, he="openfhe-fast", mpc="sci", extensions=[ext]) print(plan.explain(steps=True))
Output
latency 13676.2 s (operators 6134.0 s, conversions 0.0 s, bootstraps 7542.1 s) placement: 144 HE / 0 MPC operators, 0 conversions, 83 bootstraps, 0.00 GiB online communication … 2 Attention_QK_MatMul HE 7 -> 3 blocks.0.attn: matmul bootstrap at level 3 (#2 -> #3) 105.48 s 3 Softmax [my-he-softmax] HE 8 -> 4 blocks.0.attn: softmax bootstrap at level 4 (#3 -> #4) 105.48 s 4 Attention_V_MatMul HE 7 -> 3 blocks.0.attn: matmul …
Softmax now runs in HE, and the plan for BERT-base drops from 25,263 s to 13,676 s with no conversions left. The step table tags the operators your kernel runs.
Run the demo
python -m demo.extensions.primitive_backend
my_ckks = cf.PrimitiveBackend.he(
"my-ckks", slots=32768, levels=10,
bootstrap=cf.Cost(local_ms=9_000), # one ciphertext
primitives={"add": cf.Cost(0.3), "mul_plain": cf.Cost(3),
"mul": cf.Cost(40), "rotate": cf.Cost(35)}) # one call on one ciphertext
my_2pc = cf.PrimitiveBackend.mpc(
"my-2pc",
primitives={"mul": cf.Cost(3e-4, 128, 1), # per element: ms, bytes; rounds per call
"matmul": cf.Cost(2e-5, 8, 2),
"trunc": cf.Cost(2e-4, 128, 1),
"cmp": cf.Cost(2e-3, 4000, 16)})
print(my_ckks.lower(graph)) # each operator as primitive calls
plan = cf.compile(graph, deployment, he="openfhe-fast", mpc="sci",
extensions=[my_ckks.extension(), my_2pc.extension()])
print(plan.explain())Output
my-ckks (HE): every operator as primitive calls
Linear_QKV blocks.0.attn.{q,k,v} 53.93 s levels 1 [mul_plain 9,216, add 9,216, rotate 672]
BSGS diagonal matrix-vector product, 768 diagonals x 3 block(s)
Softmax blocks.0.attn: softmax 59.61 s levels 19 [mul_plain 12, mul 102, rotate 42, add 66, bootstrap 6]
exp by 8 squarings, row sums by rotation, 1/x by 4 Newton steps
…
latency 8435.3 s (operators 6185.5 s, conversions 8.8 s, bootstraps 2241.0 s)
placement: 143 HE / 1 MPC operators, 1 conversions, 59 bootstraps, 0.06 GiB online communicationEach operator becomes calls of your operations, with the CKKS levels they use and bootstraps inside a kernel that needs more levels than the budget. With these illustrative costs, the plan runs 143 operators on my-ckks and one LayerNorm on my-2pc.
Run the demo
python -m demo.extensions.private_rag_end_to_end --uncalibrated
from demo.extensions.private_rag import MeanPool, DocScores, TopK, rag_extension class Retriever(nn.Module): # the application: e5-base-v2's encoder, then the retrieval def __init__(self, encoder, docs, k): super().__init__() self.encoder, self.pool, self.scores, self.topk = encoder, MeanPool(), DocScores(docs), TopK(k) def forward(self, x): # x: the claim's token embeddings, from the client return self.topk(self.scores(self.pool(self.encoder(x)[0]))) rag = rag_extension() # the three operators, a kernel per domain, their costs retriever = Retriever(e5.encoder, doc_embeddings, k=10) # SciFact's 5,183 abstracts graph = cf.models.from_torch(retriever, x, extensions=[rag]) # 1. trace cal = cf.runtime.calibrate(retriever, training_claims, extensions=[rag]) # 2. calibrate plan = cf.compile(graph, cf.Deployment.of("cpu", "100Mbps/20ms"), extensions=[rag], calibration=cal) # 3. compile em = cf.runtime.Emulator(plan, retriever, extensions=[rag]) # 4. emulate: em(claim).output
Output, 300 test claims
calibrated uncalibrated
same top-10, in the same order 283/300 (94.3%) 0/300 (0.0%)
same top-10 documents 298/300 (99.3%) 0/300 (0.0%)
documents in common, on average 99.9% 0.1%
nDCG@10 (plaintext 0.7194) 0.7196 0.0000
recall@10 (plaintext 0.8465) 0.8465 0.0000
HE inputs outside their kernels' ranges none GeLU x12, Residual_Add x12
plan check passes passes
latency per query (estimate) 11757 s 10410 sThe retrieval's steps are PyTorch modules that CipherFlow traces as new operators: the document scores run in HE, the top-k in MPC. Calibrated on real claims, the private plan returns the plaintext model's top-10 for 298 of SciFact's 300 test claims. Planned without the calibration, HE GeLU leaves its range and the retrieval returns none of them. The demo needs transformers and datasets, and downloads e5-base-v2 and SciFact the first time.
Plans use the costs and levels you declare, as given. CipherFlow does not check an extension's correctness, precision or security, and does not execute extensions yet.
Read the full tutorial Limits on the roadmapRoadmap
From encoders to generative models
BERT and ViT encoders are done. Generative models come next, first the backend operators decoders need and then the models; after them, a runtime that executes plans under encryption.
- ✓Done · v0.1–v0.4
① Discriminative TransformersBERT-base, ViT-B/16
- PyTorch front end
- HE and MPC backends, measured
- plans, calibration, emulation
- Next
② Generative modelsGPT-2, then LLaMA-style decoders
- backend operators for decoders
- torch.export front end
- prefill and decode plans
- Then
③ Private inference runtimeplans executed under encryption
- two-party runtime: keys, transport
- encrypted execution, BERT to LLMs
- GPU backends and SPU
Details · ② Generative modelstwo steps: the operators decoders need, then the models
Step 1 · backend operatorscausal mask, RoPE, RMSNorm, SwiGLU, KV cache, embedding, LM head, sampling: HE kernels, MPC protocols, measured costs
Step 2 · the modelstorch.export front end; prefill and decode planned separately, KV cache as state; per-token latency; GPT-2, then LLaMA-style
Step 1 in one LLaMA-style decoder blocksupported todayadded in step 1(execution order)
- Embedding
- RMSNorm
- Q/K/V proj
- RoPE
- QKᵀ
- causal mask
- Softmax
- Attn·V
- KV cache
- Out proj
- Add
- RMSNorm
- Gate/Up proj
- SiLU
- Gate × Up
- Down proj
- Add
- LM head
- Sampling
Citation
Cite CipherFlow
Hedong Zhang, Mengxin Zheng, and Qian Lou. CipherFlow: Compiling AI Programs for Efficient Cryptographically Secure Inference.
Read the paper · Download BibTeX
@inproceedings{zhang2026cipherflow,
title = {{CipherFlow}: Compiling {AI} Programs for Efficient Cryptographically Secure Inference},
author = {Zhang, Hedong and Zheng, Mengxin and Lou, Qian},
booktitle = {Advances in Neural Information Processing Systems (NeurIPS)},
year = {2026},
url = {https://loopleap-lab.github.io/CipherFlow/assets/cipherflow-paper.pdf}
}