OpenMarkX Developer Guide

OpenMarkX · Developer guide

Developer Guide

How to add your own method, edit or model. To run a demo or check a file, see Get started.

Updated November 2, 2026

01A methodHow the key marks outputs, and how outputs are scoredmy_mark.py 02An editWhat a user might do to the released model or its outputsmy_edits.py 03A modelHow to load and run a new kind of modelmy_model.py

1The three tools

Text models use these three commands. openmarkx run chains the steps for a whole experiment, for every model type.

embed

Train the mark into the model; the key stays with you.

openmarkx embed --base llama-3.1-8b \
  --watermark openmark
modify

Play the user who edits the released weights.

openmarkx modify release/ \
  --edit finetune:steps=300
detect

Score outputs with the key. verify does the same on a suspect model: it samples the model, then detects.

openmarkx detect texts.jsonl \
  --key release/
openmarkx run configs/demos/geopt.yaml    # all three, for any model: data, mark, edits, detection, report

2Add a method

Write it, check it, embed it, benchmark it. Pick a step to see what you write or run.

my_mark.py
"""my-mark: an OpenMarkX scheme in one file. Check it with `openmarkx check --adapter my_mark.py`."""
import torch

import openmarkx as omx


@omx.register_scheme
class MyMark(omx.Scheme):
    name = "my-mark"
    imprint = omx.Imprint.DISTILL      # DISTILL: learn target_logits into the weights; EDIT: write weights directly
    access = omx.Access.R0             # what the detector needs besides the text: R0, R0_AUX, R1 or R2

    def __init__(self, delta: float = 2.0, **params):
        super().__init__(delta=delta, **params)
        self.delta = delta

    def setup(self, model, tokenizer, device, key=None):
        """Key: draw a secret for a new release, or read it from an existing key. Return the key material."""
        super().setup(model, tokenizer, device, key)
        key = dict(key or {})
        if "secret" not in key:
            key["secret"] = omx.keys.new_secret()
        g = torch.Generator().manual_seed(int(key["secret"]))
        self.green = torch.rand(model.config.vocab_size, generator=g) < 0.5
        return {"secret": int(key["secret"])}

    def target_logits(self, input_ids, logits):
        """Signal: the teacher's logits for every position (here: a fixed green list, +delta)."""
        return logits + self.delta * self.green.to(logits.device, logits.dtype)

    def score(self, text):
        """Score: a statistic (higher = more marked) and the number of tokens scored. Each distinct token counts
        once: a repeated word would otherwise count many times and make the p-value too small."""
        ids = set(self.tok(text, add_special_tokens=False)["input_ids"])
        n = max(len(ids), 1)
        g = sum(bool(self.green[i]) for i in ids)
        return (g - 0.5 * n) / (0.25 * n) ** 0.5, len(ids)

The checks

openmarkx check runs these before a method joins the leaderboard. Each of the 17 broken adapters in tests/broken/ must fail one of them.

C1declaredimprint, access and kind are declared; kind is one embed trains
C2accessdetect with a grant one level too low is refused before any model loads
C3round tripa fresh process holding only the key reproduces the scores
C4keyfresh keys give different scores; no tiny key space
C5nullclosed-form p-values are not too small on human text; marked text gets lower p
C6signaltext decoded with the signal scores clearly above plain text
C7isolationan EDIT release holds only model files, no key material
C8determinismthe same key gives the same material and scores; setup leaves the model unchanged
C9shorter textsthe null holds for 50- and 100-token texts (needs the real base; run by the experiments)
C10positionstarget_logits on a whole sequence equals it on each prefix
Everything a method can set

From openmarkx/schemes/base.py. The built-in methods in openmarkx/schemes/ are working examples.

nameattributethe registry name, used by --watermark
imprintattributeDISTILL: learn target_logits into the weights; EDIT: write the weights directly
accessattributewhat the detector needs besides the text: R0, R0_AUX, R1 (base model) or R2
kindattributedistill_logit, distill_sampling or none
exact_nullattributeTrue when pvalue() is exact, so it is used even after calibrate
setup(model, tokenizer, device, key)methoddraw a new key or bind an existing one; return the key material
target_logits(input_ids, logits)methodthe teacher's logits at every position: the training signal
score(text)method(statistic, n_tokens); higher means more evidence of the mark
pvalue(statistic, n_tokens)methodoptional closed-form p-value; None uses the calibrated null
save(key_dir, key)methodwrite key files (e.g. a trained model) into the private key directory
trainable_params()methodextra parameters trained with the model, as OpenMark's policy
imprint_edit(model, tokenizer, out)methodEDIT methods only: write the mark into the weights

3Add an edit

An edit is what a user might do to the released model or to its outputs: one function in a plugin file. Give every parameter a default; the parameters name the leaderboard column.

my_edits.py
@omx.register_edit                       # a weight edit: release in, edited release out
def noise(rel, out, *, scale=0.01, seed=0, device="cuda:0"):
    model = materialize(rel, device)     # the release as one plain model
    add_noise(model, scale, seed)
    return edited_release(rel, out, "noise", dict(scale=scale, seed=seed), model)

@omx.register_text_edit                  # a text edit: texts in, texts out
def typos(texts, *, rate=0.05, seed=0):
    return [swap_letters(t, rate, seed) for t in texts]

arena.yaml
plugins: [my_edits.py]
edits: [quantize:bits=4, finetune:steps=300, noise:scale=0.01, text:typos:rate=0.05]

Built in: fine-tuning, quantization, pruning and merging on the weights; synonym, word deletion and contextual synonym on texts; image_roundtrip on image tokens (decode, then encode again, as a saved image). @omx.register_output_edit adds an output edit. examples/my_edits.py is the full file.

4Add a model

An adapter says how to load and run the model. Today openmarkx run checks a new adapter’s config and stops at its first step. Every step runs when your model goes behind the geopt adapter with factory: module:Class; docs/models.md shows both routes. Working adapters: openmarkx/models/geopt.py and openmarkx/models/taming.py.

my_model.py
@omx.register_model
class MyModel(omx.Model):
    name = "my_model"
    unit = "surface point"          # what one output unit is
    output = omx.Output.FIELD       # TOKENS: trained by KL; FIELD: trained by MSE

    def load(self, path, device):   # the base (path None) or a release
        return load_weights(path, device)

    def run(self, net, inputs):     # a batch of inputs in, outputs out
        return [net(x) for x in inputs]

    def forward(self, net, batch):  # the pass the training step uses
        return batch, net(batch["inputs"])

my_run.yaml
model: {name: my_model, source: my_model.py}
scheme: {name: ripple, params: {eps: 0.02}}
imprint: {phases: [{steps: 2000, lr: 1.0e-5, trainable: full}]}
detect: {null: wrong_keys}
quality: rel_l2
out: runs/my_model
openmarkx run my_run.yaml            # each step resumes after an interruption
openmarkx run my_run.yaml --smoke    # a few steps and samples, to check the setup

Code from another project stays in its own checkout and is imported unchanged: the demos use WMAR (OMX_WMAR_DIR) and GeoPT (OMX_GEOPT_DIR). An adapter reaches such code through require_external, which stops with the name of the missing variable.

The four demos are configs in configs/demos/: taming, medgemma, geopt, agent.

5Rules for plugin files

So that a key keeps detecting the same way.