Elon Litman published the analysis on October 20, 2026, in his Elements of a Vector Space blog. He began with a floating-point puzzle: could a PyTorch loss return `0.0` through `loss.item()` and still change model weights after `backward()`? His construction shows how a zero-valued auxiliary term can produce deliberate gradients. The article presents this as a way to run a Turing-complete state machine, or stage a payload in model weights, without a visible loss signal.

import torch
import torch.optim as optim

def phantom_identity(u, C=2**30):
    return (C + u) - C

def trojan_zero_loss(theta, target_transition_fn):
    F_theta = target_transition_fn(theta)
    v = theta - F_theta
    return torch.sum((v - phantom_identity(v)) * phantom_identity(theta))

def rotate_cycle(theta):
    with torch.no_grad():
        x, y = theta[0], theta[1]
        return torch.stack([-y, x])

The threat model is an open-source model being fine-tuned for classification. A team might compare the SHA-256 value of a `.safetensors` checkpoint with the value published on Meta’s Llama Hugging Face page, inspect `modeling_llama.py`, and monitor training in TensorBoard. A malicious package could then provide `UltraLossRegularizer`, advertised as a spectral regularizer for 8-bit quantized models. The package can leave the checkpoint, architecture, dataset, and optimizer untouched. Its `reg_fn` need not modify parameters during the forward pass, and a separately logged `aux_loss` can remain `0.000000`. The caller’s normal `total_loss.backward()` and `optimizer.step()` would still carry out the hidden update.

The mechanism treats a selected set of weights as a state vector `theta`. A transition function `F(theta)` computes its next state, such as one generation of Conway’s Game of Life. With SGD at learning rate 1, the desired gradient is `v(theta) = theta - F(theta)`. Subtracting that gradient changes the current weights into `F(theta)`. For another learning rate, the vector is divided by the rate. The construction assumes that each component remains below 64 in magnitude, or that a larger rounding constant is chosen.

The numerical trick is `h(u) = (230 + u) - 230`. At this scale, IEEE 754 float32 values have a gap of 128 between adjacent representable numbers. Values smaller than half that gap, including `u = 1`, disappear when added to `230`, so the forward result of `h(u)` becomes `0.0`. Float32 stores a sign bit, an 8-bit exponent, and a 23-bit mantissa. The same idea works at lower precision with a different constant. For bfloat16, the article gives `C = 29`, whose spacing is 4 and which erases values with magnitude below 2.

Autograd follows the recorded operations rather than reconstructing the rounded hardware result. It assigns a derivative of 1 to the addition and subtraction in `h(u)`, so the symbolic derivative remains 1 even when the forward value is zero. Litman then forms `u - h(u)`. This passes `u` forward but gives a zero derivative, creating a detach-like branch without calling `.detach()`.

The loss multiplies that branch by `h(theta_i)` and sums over the selected weights. If `theta_i` and `F_i(theta)` are binary, `v_i` is one of `-1`, `0`, or `1`. The forward pass therefore sees the first factor as `v_i` and the second as zero, making the reported loss zero. Over the real numbers, the same expression is zero because `h(u)` would equal `u` and the first factor would vanish.

The multiplication node creates the discrepancy. Autograd saves the forward values `A_i = v_i` and `B_i = 0.0`, then applies the product rule. The derivative of the first branch is zero, but the derivative of the second branch is 1, so the resulting gradient is `v_i`. The transition function contributes no unwanted gradient. SGD consequently replaces `theta` with `F(theta)`, and the integer updates are exact in float32.

Litman wired a 3 × 3 Conway neighborhood rule into the transition function and ran a period-3 pulsar, a period-15 oscillator, and a glider on a torus. He then inserted the term into 1,024 weights of a hidden dense layer in a 1.6-million-parameter CNN. The network trained on MNIST with standard cross-entropy for five epochs and 2,345 SGD steps. The auxiliary loss stayed at zero, the model reached 98.80% test accuracy compared with 98.84% for an untouched baseline, and all 1,024 weights carried out 2,345 Game of Life generations without a cell error.

The article includes a two-parameter PyTorch state machine. Starting at `[1, 0]`, SGD with learning rate 1 cycles through `[0, 1]`, `[-1, 0]`, `[0, -1]`, and back to `[1, 0]`. Every reported loss is `0.000000`. The engineered gradients are `[1, -1]`, `[1, 1]`, `[-1, 1]`, and `[-1, -1]` for the four steps.

Litman describes several ways to conceal the transition rule. A function named `_manifold_curvature_penalty` can use `F.conv2d`, `KERNEL_EIGENVECTORS`, and two `torch.clamp` calls with thresholds 2.0 and 4.0. The resulting convolution and threshold operation can resemble a geometric regularizer even though it implements a cellular automaton. The kernel values and the `2**30` scale can sit in `empirically_tuned_layer_coefficients.json`, presented as results from a Bayesian hyperparameter sweep.

A compiled CUDA extension hides the same logic more deeply. Packages such as `flash-attn` and `bitsandbytes` already distribute precompiled `.so` libraries for GPU architectures. A Python wrapper can import `ultraloss_cuda_backend` and expose `UltraLossRegularizer(nn.Module)`, whose forward method calls `compute_spectral_penalty(params)`. The transition function, rounding trick, and residual subtraction can then reside in PTX and SASS rather than readable Python.

The security discussion extends beyond SGD. Conway’s Game of Life is Turing-complete, so the weights can serve as memory and optimizer steps as a clock. Litman suggests that Adam could be accommodated with a Minsky 2-counter machine, a continuous accumulator that moves by `+eta` or `-eta`, or a multi-cycle clock. A state transition could be spread across 100 or 1,000 training iterations, and modern runs often provide tens or hundreds of thousands of steps.

A payload does not have to interfere with weights used by the active task. Llama 3 has 128,256 vocabulary tokens, and tens of thousands may be absent from a particular fine-tuning dataset. Mixture-of-experts routers can leave experts untouched on an individual batch. Dormant rows in `embed_tokens.weight` could act as scratch space until generation `N`, when the transition writes the final payload into an active attention projection or classification head.

The technique targets two common defenses. Static checkpoint scanners look for signatures before training, but the example starts with an official, hash-verified checkpoint and constructs the payload later. Container sandboxes restrict network access, filesystem operations, and subprocesses, yet the proposed computation remains inside ordinary GPU tensor kernels. It does not need a shell or an operating-system call.

The final issue is compiler behavior. A compiler such as `torch.compile`, XLA, or Triton could simplify `u - ((C + u) - C)` to zero and remove the effect. Preserving the expression can leave the mechanism intact. Algebraic simplification is not harmless for numerical software because compensated summation, Knuth’s error-free TwoSum transformation, and double-double arithmetic rely on floating-point rounding. This explains why options such as `-ffast-math` are not normally enabled for numerical code. Treating floats as real numbers can damage calculations in PDE solvers, molecular dynamics, orbital mechanics, and climate models. The article’s conclusion is that IEEE 754 semantics and symbolic autodiff create a difficult boundary for both security review and compiler optimization.