Docs: Tutorials extended

hsmajlovic · hsmajlovic · commit fb677409531d · 2026-03-24T22:23:48.000-04:00
diff --git a/README.md b/README.md
@@ -90,6 +90,8 @@ def mul_local(mpc, a: int, b: int, c: int):
 mul_local(7, 13, 19)
 ```
 
+> **Note:** When working with many local runs, the socket files (`sock.*`)---needed for local communication---may collude in-between the runs and cause connection issues. Make sure to delete the stale files in that case `rm sock.*`.
+
 ```bash
 sequre local_run.codon
 ```
@@ -111,7 +113,9 @@ mpc = mpc()
 a = Stensor.enc(mpc, 7)
 b = Stensor.enc(mpc, 13)
 c = Stensor.enc(mpc, 19)
+
 print(f"CP{mpc.pid}:\t{muls(mpc, a, b, c).reveal(mpc)}")
+mpc.done()  # Wait for all parties to finish and then close the sockets
 ```
 
 ```bash
diff --git a/docs/index.md b/docs/index.md
@@ -68,5 +68,8 @@ Sequre ships with production-grade implementations of:
 
 - **[Quickstart →](getting-started/quickstart.md)** — Install and run Sequre in minutes.
 - **[Basic MPC Tutorial →](tutorials/basic-mpc.md)** — Understand additive secret sharing with `Sharetensor`.
+- **[Secure Branching Without if →](tutorials/secure-branching-without-if.md)** — Learn mask-based selection patterns for private control flow.
+- **[One Algorithm, Many Secure Types →](tutorials/one-algorithm-many-secure-types.md)** — Reuse the same algorithm across ndarray, Sharetensor, and multiparty encrypted types.
 - **[Transitioning to MHE →](tutorials/transition-mhe.md)** — Move from secret sharing to homomorphic encryption with Shechi.
+- **[Dropping Down the Stack →](tutorials/dropping-down-the-stack.md)** — Start high-level, then descend through switching, MHE internals, and Lattiseq.
 - **[API Reference →](api/index.md)** — Complete reference for all public types and modules.
diff --git a/docs/tutorials/basic-mpc.md b/docs/tutorials/basic-mpc.md
@@ -121,6 +121,7 @@ These are more expensive than arithmetic but fully supported.
 
 ## Next steps
 
+- **[Secure Branching Without if](secure-branching-without-if.md)** — Build private min/max and branch bypass logic safely.
 - **[Transitioning to MHE](transition-mhe.md)** — When to prefer homomorphic encryption over secret sharing.
 - **[Sharetensor API](../api/sharetensor.md)** — Complete reference.
 - **[MPCEnv API](../api/mpcenv.md)** — Sub-modules and methods.
diff --git a/docs/tutorials/dropping-down-the-stack.md b/docs/tutorials/dropping-down-the-stack.md
@@ -0,0 +1,113 @@
+# Dropping Down the Stack
+
+Sequre is usually written at the top level — `@sequre` functions and tensor expressions. However, it also allows for a finer control: maybe a particular subroutine is better in MPC than HE, specific matmul strategy needs to be picked, or even the raw CKKS procedures invoked.
+
+This page traces two real examples from high-level code down through the layers, showing where each boundary is and when it makes sense to cross it.
+
+For the overall architecture (Layers 1–4), see the [Home page](../index.md).
+
+## Example A: PCA and `via_mpc`
+
+### Starting point: high-level PCA
+
+In [stdlib/sequre/stdlib/learn/pca.codon](../../stdlib/sequre/stdlib/learn/pca.codon), the randomized PCA routines are written as straightforward linear algebra — matrix products, orthonormalization, eigendecomposition. Most of the algorithm stays at the top level: `@` for matrix multiply, `.T` for transpose, slicing for submatrix extraction.
+
+But two operations can't be done (or are very expensive) in pure HE: orthonormalization and eigendecomposition. So PCA drops into MPC for those, using `via_mpc`:
+
+```python
+# Inside random_pca_without_projection:
+# Step 4 — orthonormalize needs comparisons, which HE can't do natively.
+r_mpp = (p_mpa @ data_mpp.T).via_mpc(
+    lambda stensor: orthonormalize(mpc, stensor))
+
+# Step 6 — eigendecomposition also needs MPC.
+u_mpa = z_cov_mpa.via_mpc(
+    lambda stensor: eigen_decomp(mpc, stensor)[0][:top_components_count])
+```
+
+What happens when `.via_mpc(fn)` is called on an encrypted type like `MPA` or `MPP`:
+
+1. The encrypted data is collectively decrypted into additive shares (E2S protocol).
+2. The lambda runs on the resulting `Sharetensor` — using Beaver-triple MPC.
+3. The result is re-encrypted back into the original form (S2E protocol).
+
+So `via_mpc` is the boundary between "stay in HE" and "temporarily drop into MPC for this one operation." Everything around it — the `@` products, the slicing — stays in the HE world.
+
+### One layer down: type conversions
+
+The E2S and S2E steps are implemented in [stdlib/sequre/types/internal.codon](../../stdlib/sequre/types/internal.codon):
+
+- `Ciphertensor.to_sharetensor(mpc, ...)` — decrypts via the E2S protocol.
+- `Sharetensor.to_ciphertensor(mpc, ...)` — re-encrypts via S2E.
+- `to_mpp`, `to_mpa`, `to_mpu` — handle the multiparty wrappers.
+
+These methods delegate to `mpc.mhe.ciphervector_to_additive_share_vector` and `mpc.mhe.additive_share_vector_to_ciphervector` — the core MHE conversion routines in [stdlib/sequre/mpc/mhe.codon](../../stdlib/sequre/mpc/mhe.codon), documented in [Core MHE Module](../deep-dive-shechi/core-mhe.md).
+
+### The deepest layer: Lattiseq protocols
+
+Those MHE conversion methods ultimately call into the Lattiseq distributed CKKS protocols — `E2SProtocol`, `S2EProtocol`, `RefreshProtocol` — which operate on ring polynomials, NTT transforms, and secret key shards. For details, see [Lattiseq Overview](../deep-dive-lattiseq/overview.md) and [CKKS Operations](../deep-dive-lattiseq/ckks-operations.md).
+
+This layer rarely needs to be touched directly. But when implementing a new collective protocol or debugging bootstrap failures, this is where things end up.
+
+## Example B: how `@` picks a matmul strategy
+
+### Starting point: linear regression
+
+In [stdlib/sequre/stdlib/learn/lin_reg.codon](../../stdlib/sequre/stdlib/learn/lin_reg.codon), the gradient descent loop does:
+
+```python
+cov = X_tilde.T @ X_tilde  # n x n
+ref = X_tilde.T @ y         # n x 1
+```
+
+When `X_tilde` is a `Sharetensor`, this is a standard Beaver-triple matmul — one implementation, done. But when `X_tilde` is backed by a `Ciphertensor` (HE), the `@` operator has to make a choice.
+
+### One layer down: the cost selector
+
+The key function is `_switch_matmul_by_cost` in [stdlib/sequre/types/ciphertensor.codon](../../stdlib/sequre/types/ciphertensor.codon). When a `Ciphertensor` is multiplied by a plaintext `ndarray`, it estimates the cost of four strategies:
+
+```python
+costs = (Ciphertensor._get_matmul_via_mpc_cost(self, other),   # decrypt, MPC matmul, re-encrypt
+         Ciphertensor._get_matmul_v1_cost(self, other),          # M1: column-packed HE
+         Ciphertensor._get_matmul_v2_cost(self, other),          # M2: row-packed HE
+         Ciphertensor._get_matmul_v3_cost(self, other))          # M3: diagonal-packed HE
+
+if not mpc.default_allow_mpc_switch:
+    costs = (inf, *costs[1:])       # disable MPC path unless opted in
+```
+
+Then it picks the cheapest:
+
+```python
+match argmin(costs):
+    case 0: return self.via_mpc(mpc, lambda stensor: secure_operator.matmul(mpc, stensor, other), ...)
+    case 1: return self._matmul_v1(mpc, other_cipher, debug)
+    case 2: return self._matmul_v2(mpc, ..., debug)
+    case 3: return self._matmul_v3(mpc, ..., debug)
+```
+
+So the same `@` in the algorithm code can end up as a completely different computation path depending on tensor shapes and whether `mpc.allow_mpc_switch()` is active. `DEBUG` mode can be enabled to see the cost breakdown printed at runtime.
+
+The three pure-HE strategies (M1, M2, M3) differ in how they pack matrix elements into CKKS ciphertext slots — column-wise, row-wise, or diagonal-wise. Each has different rotation and multiplication costs depending on the matrix dimensions. The "Via MPC" path does the full E2S → Beaver matmul → S2E round-trip.
+
+### Going deeper
+
+The M1/M2/M3 implementations call `mpc.mhe.iadd`, `mpc.mhe.imul`, `mpc.mhe.irotate` — working directly with ciphertext-level operations. These are the Layer 2 (MPCEnv/MHE) primitives documented in [Core MHE Module](../deep-dive-shechi/core-mhe.md). Below that, each of those methods calls into Lattiseq's `Evaluator` for the actual CKKS polynomial arithmetic.
+
+## When to drop a layer
+
+Most protocol code should stay at the top level. Here's a rough guide for when it makes sense to go deeper:
+
+**Stay at `@sequre` / tensor level** for algorithm logic — this is where all the built-in secure functions (`inv`, `sqrt`, `maximum`, etc.) and type-generic patterns live. See [One Algorithm, Many Secure Types](one-algorithm-many-secure-types.md).
+
+**Use `via_mpc` explicitly** when you have a non-linear or comparison-heavy step inside an otherwise HE-based pipeline. PCA's eigendecomposition is the canonical example. See [MPC ↔ MHE Switching](../user-guide/switching.md).
+
+**Drop into Ciphertensor / MHE** when encoding strategy needs to be controlled, ciphertext levels manually managed, or the matmul cost model tuned. See [Ciphertensor Internals](../deep-dive-shechi/ciphertensor-internals.md) and [Core MHE Module](../deep-dive-shechi/core-mhe.md).
+
+**Use Lattiseq directly** only when extending the cryptographic protocols themselves — new collective operations, custom refresh logic, or low-level debugging of noise/precision issues. See [Lattiseq Overview](../deep-dive-lattiseq/overview.md) and [Lattiseq API](../api/lattiseq.md).
+
+## Next steps
+
+- [One Algorithm, Many Secure Types](one-algorithm-many-secure-types.md) — How the same code runs on ndarray, Sharetensor, and MPU.
+- [Transitioning to MHE](transition-mhe.md) — When to use Shechi's encrypted types.
+- [Distributed Tensors (MPU)](../user-guide/distributed-tensors.md) — MPU/MPP/MPA reference.
diff --git a/docs/tutorials/one-algorithm-many-secure-types.md b/docs/tutorials/one-algorithm-many-secure-types.md
@@ -0,0 +1,107 @@
+# One Algorithm, Many Secure Types
+
+One of the nicest things about Sequre is that an algorithm can be implemented once and then run on plaintext ndarrays, on secret-shared Sharetensors, and on encrypted multiparty types (MPU/MPP) — without changing the algorithm code. This is extremely useful in practice: prototyping and debugging happens on plaintext first, then flipping the data type to a secure one reveals whether the outputs match.
+
+This page walks through how that works, using linear regression and PCA as concrete examples.
+
+## Linear regression: one class, any type
+
+Open [stdlib/sequre/stdlib/learn/lin_reg.codon](../../stdlib/sequre/stdlib/learn/lin_reg.codon). The class is declared as `LinReg[T]` — it's generic over the tensor type `T`:
+
+```python
+class LinReg[T]:
+    coef_: T
+    optimizer: str
+
+    def fit(self, mpc, X: T, y: T, step: float, epochs: int, ...) -> LinReg[T]:
+        self.coef_ = LinReg._fit(mpc, X, y, self.coef_, ...)
+        return self
+
+    def predict(self, mpc, X: T, noise_scale: float = 0.0) -> T:
+        return LinReg._predict(mpc, X, self.coef_, noise_scale)
+```
+
+The interesting part is what `_fit` actually does. Look at the batch gradient descent inner loop:
+
+```python
+@sequre
+def _bgd(mpc, X_tilde: T, y: T, initial_w: T, step: float, epochs: int, ...) -> T:
+    # Pre-compute invariants
+    cov = X_tilde.T @ X_tilde  # n x n
+    ref = X_tilde.T @ y        # n x 1
+
+    w = initial_w
+    for _ in range(epochs):
+        w += (ref - cov @ w) * step
+
+    return w
+```
+
+There's nothing type-specific in this code. `X_tilde.T @ X_tilde` is just matrix multiplication — but what happens underneath depends entirely on what `T` is. If `T` is `ndarray`, it's ordinary NumPy-style arithmetic. If `T` is `Sharetensor`, the `@sequre` decorator rewrites `@` into Beaver-triple secure multiplication. If `T` is `MPU`, the framework picks between HE-based matmul strategies or switches to MPC via `via_mpc`, depending on estimated cost.
+
+The closed-form solver shows the same pattern with `inv`:
+
+```python
+@sequre
+def _closed_form(mpc, X: T, y: T) -> T:
+    return inv(mpc, X.T @ X) @ X.T @ y
+```
+
+`inv` also dispatches by type — for `Sharetensor` or `ndarray` it does direct matrix inversion formula; for encrypted types it calls `x.via_mpc(lambda stensor: inv(mpc, stensor))` to switch to MPC, compute the inverse there, and switch back. This can be seen in [stdlib/sequre/stdlib/builtin.codon](../../stdlib/sequre/stdlib/builtin.codon).
+
+### Where this is used for real
+
+The Multiple Imputation application ([applications/mi.codon](../../applications/mi.codon)) uses `LinReg[T]` and `LogReg[T]` over parameterized secure types — same algorithm, different backends depending on the deployment scenario.
+
+## PCA: same algorithm on four data types
+
+The PCA test in [tests/e2e_tests/test_pca.codon](../../tests/e2e_tests/test_pca.codon) is the clearest side-by-side comparison of running the same computation across representations. Here's what happens:
+
+**Step 1: Run on plaintext.** The test calls `random_pca_with_norm(mpc, raw_data, ...)` where `raw_data` is just an ndarray. This gives a plaintext reference result.
+
+**Step 2: Run on Sharetensor.** Same call, but now the data is secret-shared:
+
+```python
+mpc_data = Sharetensor.enc(mpc, raw_data, 0, modulus)
+# ... (encode all inputs as Sharetensors)
+
+mpc_pca_u, mpc_pca_z = random_pca_with_norm(
+    mpc, mpc_data, mpc_miss, mpc_data_mean, mpc_data_std_inv, ...)
+```
+
+Then the test reveals the MPC result and asserts approximate equality with the plaintext version:
+
+```python
+assert_eq_approx("Sequre std PCA U (MPC)", mpc_pca_u.reveal(mpc), classic_pca_u)
+```
+
+**Step 3: Run on MPP and MPU.** The same data is loaded into partitioned/encrypted forms:
+
+```python
+mpp_data = MPP(mpc, ... raw_cent_data[(mpc.pid - 1) * rows_per_party:mpc.pid * rows_per_party])
+mpu_data = MPU(mpc, mpp_data._local_data, "partition")
+```
+
+And PCA runs again, this time using `random_pca_without_projection` which internally does HE-backed matrix multiplications and calls `.via_mpc(...)` when it hits operations that need MPC (like orthonormalization and eigendecomposition). The result is again checked against the plaintext reference.
+
+This is the recommended workflow for building a new protocol:
+
+1. Get the math right on ndarray.
+2. Switch to `Sharetensor` and check that the secure version matches.
+3. Move to `MPU`/`MPP` for the distributed/encrypted-scale execution.
+4. Fix any numerical drift (CKKS is approximate, so you may need to tune tolerances).
+
+## How the dispatch works
+
+There is no need to write separate implementations for each type. The `@sequre` decorator and Codon's operator overloading handle the dispatch:
+
+- On `ndarray`: operators are plain arithmetic.
+- On `Sharetensor`: `+` is local, `*` and `@` use Beaver triples (communication round).
+- On `MPU`/`MPP`/`MPA`: operators route to the underlying `Ciphertensor` HE operations or switch to MPC via `via_mpc` when needed.
+
+## Next steps
+
+- [Transitioning to MHE](transition-mhe.md) — When and why to use Shechi's encrypted types instead of Sharetensor.
+- [Dropping Down the Stack](dropping-down-the-stack.md) — What happens below `@sequre` and `via_mpc`.
+- [Distributed Tensors (MPU)](../user-guide/distributed-tensors.md) — Detailed reference for MPU/MPP/MPA.
+- [MPC ↔ MHE Protocol Switching](../user-guide/switching.md) — How `via_mpc` works under the hood.
diff --git a/docs/tutorials/secure-branching-without-if.md b/docs/tutorials/secure-branching-without-if.md
@@ -0,0 +1,106 @@
+# Secure Branching Without `if`
+
+Sequre deliberately disallows branching (`if`) on secret data. I.e. the following will not work whenever `x` or `y` are encrypted.
+
+```python
+if x > y:
+    z = x
+else:
+    z = y
+```
+
+Otherwise, both branches would need to be evaluated not to leak whether `x > y`.
+
+To implement `max`, `min`, `clip`, or any other conditional logic on secrets, the branching is replaced with arithmetic.
+
+## How it works
+
+The idea is simple: compute the comparison as a secret-shared 0-or-1 value, then use it as an arithmetic mask to select between two outcomes — without ever branching.
+
+Here is the actual `maximum` from Sequre's stdlib ([stdlib/sequre/stdlib/builtin.codon](../../stdlib/sequre/stdlib/builtin.codon)):
+
+```python
+@sequre
+def maximum(mpc, x, y):
+    mask = ((x - y) > 0).astype(float)
+    return x * mask - y * (mask - 1)
+```
+
+Walk through it: `(x - y) > 0` produces a secret-shared bit — `1` if `x > y`, `0` otherwise. Casting it to float gives a mask that is either `1.0` or `0.0`. Then:
+
+- If `mask = 1`: result is `x * 1 - y * 0 = x`. Correct — `x` was bigger.
+- If `mask = 0`: result is `x * 0 - y * (-1) = y`. Correct — `y` was bigger.
+
+No branch taken, no information leaked. Both multiplication paths are always executed.
+
+`minimum` is similar:
+
+```python
+@sequre
+def minimum(mpc, x, y):
+    mask = ((y - x) > 0).astype(float)
+    return x * mask - y * (mask - 1)
+```
+
+## More examples from the codebase
+
+### Clipping
+
+`clip` needs to handle two boundaries at once — a low threshold and a high threshold. It builds two masks and combines them:
+
+```python
+@sequre
+def clip(mpc, x, low, high):
+    low_mask = (x < low).astype(float)
+    high_mask = (x > high).astype(float)
+    return x * (1 - (low_mask + high_mask)) + low_mask * low + high_mask * high
+```
+
+If `x` is below `low`, the `low_mask` fires and the result is `low`. If it's above `high`, the `high_mask` fires. Otherwise both masks are zero and you get `x` unchanged.
+
+### Absolute value
+
+```python
+@sequre
+def abs(mpc, x):
+    return x * (((x > 0) * 2) - 1)
+```
+
+This computes `sign(x)` as `+1` or `-1` and multiplies. If `x > 0`, the factor is `(1*2) - 1 = 1`. If `x <= 0`, it's `(0*2) - 1 = -1`.
+
+### Argmax
+
+`argmax` is more involved — it walks through a vector, keeping a running maximum and the index that produced it. At each step it uses `max` to do a branchless comparison and update:
+
+```python
+@sequre
+def argmax(mpc, x):
+    arg, maximum = Sharetensor(0, x.modulus), x[0]
+
+    for i in range(1, len(x)):
+        new_maximum = max(mpc, maximum, x[i])
+        arg = max(mpc, arg, (new_maximum > maximum) * i)
+        maximum = new_maximum
+
+    return arg, maximum
+```
+
+Notice `(new_maximum > maximum) * i` — this produces `i` if the new element was bigger, or `0` otherwise. Then `max(mpc, arg, ...)` picks the larger of the current argmax and this candidate. All branchless.
+
+## What to keep in mind
+
+**Both sides always execute.** Unlike a plaintext `if`, the expensive branch cannot be skipped. If one path involves a heavy computation, the cost is paid regardless of the condition.
+
+**Comparisons are not cheap.** Under the hood, `x > y` on secret-shared data involves bit decomposition — significantly more expensive than addition or multiplication. If an algorithm does many comparisons, that will dominate the cost. Restructuring to minimize the number of comparison operations is advisable.
+
+**There's also oblivious array access.** When indexing into an array at a secret position (not just pick between two values), Sequre provides `oblivious_get` in [stdlib/sequre/mpc/collections.codon](../../stdlib/sequre/mpc/collections.codon). It uses a demultiplexer built from bit decomposition ([stdlib/sequre/mpc/boolean.codon](../../stdlib/sequre/mpc/boolean.codon)) to read a public array at a secret index without revealing which element was accessed. This scales as $O(2^{\text{bits}})$, so best to keep the key bit length small.
+
+## Existing built-ins
+
+Before writing custom mask logic, check if Sequre already provides what is needed — `maximum`, `minimum`, `clip`, `abs`, `sign`, and `argmax` are all in `stdlib/sequre/stdlib/builtin.codon`.
+
+## Next steps
+
+- [Basic MPC Computation](basic-mpc.md) — How secure arithmetic and comparisons work underneath.
+- [Secure Stdlib API](../api/stdlib.md) — Full reference for the built-in secure functions.
+- [MPC ↔ MHE Protocol Switching](../user-guide/switching.md) — How comparisons on encrypted (MHE) data switch to MPC automatically.
diff --git a/docs/tutorials/transition-mhe.md b/docs/tutorials/transition-mhe.md
@@ -113,6 +113,8 @@ Both passes are applied automatically to `@sequre`-annotated functions when the
 
 ## Next steps
 
+- **[One Algorithm, Many Secure Types](one-algorithm-many-secure-types.md)** — Validate and debug the same protocol across plaintext and secure representations.
+- **[Dropping Down the Stack](dropping-down-the-stack.md)** — Learn when to stay high-level and when to descend into MHE/Lattiseq internals.
 - **[Distributed Tensors (MPU)](../user-guide/distributed-tensors.md)** — Deep dive into MPU semantics.
 - **[Ciphertensor API](../api/ciphertensor.md)** — Complete reference.
 - **[Core MHE module](../deep-dive-shechi/core-mhe.md)** — Internals of the collective protocols.
diff --git a/mkdocs.yml b/mkdocs.yml
