OpenMarkX · Developer guide
Developer Guide
How to add your own method, edit or model. To run a demo or check a file, see Get started.
Updated November 2, 2026
my_mark.py
02An editWhat a user might do to the released model or its outputsmy_edits.py
03A modelHow to load and run a new kind of modelmy_model.py
1The three tools
Text models use these three commands. openmarkx run chains the steps for a whole experiment, for every model type.
Train the mark into the model; the key stays with you.
openmarkx embed --base llama-3.1-8b \ --watermark openmark
Play the user who edits the released weights.
openmarkx modify release/ \ --edit finetune:steps=300
Score outputs with the key. verify does the same on a suspect model: it samples the model, then detects.
openmarkx detect texts.jsonl \ --key release/
openmarkx run configs/demos/geopt.yaml # all three, for any model: data, mark, edits, detection, report
2Add a method
Write it, check it, embed it, benchmark it. Pick a step to see what you write or run.
my_mark.py
"""my-mark: an OpenMarkX scheme in one file. Check it with `openmarkx check --adapter my_mark.py`."""
import torch
import openmarkx as omx
@omx.register_scheme
class MyMark(omx.Scheme):
name = "my-mark"
imprint = omx.Imprint.DISTILL # DISTILL: learn target_logits into the weights; EDIT: write weights directly
access = omx.Access.R0 # what the detector needs besides the text: R0, R0_AUX, R1 or R2
def __init__(self, delta: float = 2.0, **params):
super().__init__(delta=delta, **params)
self.delta = delta
def setup(self, model, tokenizer, device, key=None):
"""Key: draw a secret for a new release, or read it from an existing key. Return the key material."""
super().setup(model, tokenizer, device, key)
key = dict(key or {})
if "secret" not in key:
key["secret"] = omx.keys.new_secret()
g = torch.Generator().manual_seed(int(key["secret"]))
self.green = torch.rand(model.config.vocab_size, generator=g) < 0.5
return {"secret": int(key["secret"])}
def target_logits(self, input_ids, logits):
"""Signal: the teacher's logits for every position (here: a fixed green list, +delta)."""
return logits + self.delta * self.green.to(logits.device, logits.dtype)
def score(self, text):
"""Score: a statistic (higher = more marked) and the number of tokens scored. Each distinct token counts
once: a repeated word would otherwise count many times and make the p-value too small."""
ids = set(self.tok(text, add_special_tokens=False)["input_ids"])
n = max(len(ids), 1)
g = sum(bool(self.green[i]) for i in ids)
return (g - 0.5 * n) / (0.25 * n) ** 0.5, len(ids)
shell
openmarkx check --adapter my_mark.py
# runs C1-C10 against a tiny Llama, on CPU, in minutes
# a failed check names what to fix; fix it and run againshell
openmarkx embed --base llama-3.2-1b --watermark my_mark.py
openmarkx calibrate RELEASE_DIR --trust-plugin # freeze the null for this key
openmarkx generate RELEASE_DIR --prompts prompts.jsonl --out texts.jsonl
openmarkx detect texts.jsonl --key RELEASE_DIR --trust-pluginshell
# my_arena.yaml
base: llama-3.2-1b
methods: [openmark, markllm-kgw-d, my_mark.py] # built-in names (with :params) or plugin files
edits: [finetune:steps=800, quantize:bits=4, text:synonym:ratio=0.3] # optional; default: the full grid
openmarkx bench my_arena.yaml --dry-run # what is done, what would run
openmarkx bench my_arena.yaml # run the missing steps, then print the leaderboardThe checks
openmarkx check runs these before a method joins the leaderboard. Each of the 17 broken adapters in tests/broken/ must fail one of them.
| C1 | declared | imprint, access and kind are declared; kind is one embed trains |
| C2 | access | detect with a grant one level too low is refused before any model loads |
| C3 | round trip | a fresh process holding only the key reproduces the scores |
| C4 | key | fresh keys give different scores; no tiny key space |
| C5 | null | closed-form p-values are not too small on human text; marked text gets lower p |
| C6 | signal | text decoded with the signal scores clearly above plain text |
| C7 | isolation | an EDIT release holds only model files, no key material |
| C8 | determinism | the same key gives the same material and scores; setup leaves the model unchanged |
| C9 | shorter texts | the null holds for 50- and 100-token texts (needs the real base; run by the experiments) |
| C10 | positions | target_logits on a whole sequence equals it on each prefix |
Everything a method can set
From openmarkx/schemes/base.py. The built-in methods in openmarkx/schemes/ are working examples.
name | attribute | the registry name, used by --watermark |
imprint | attribute | DISTILL: learn target_logits into the weights; EDIT: write the weights directly |
access | attribute | what the detector needs besides the text: R0, R0_AUX, R1 (base model) or R2 |
kind | attribute | distill_logit, distill_sampling or none |
exact_null | attribute | True when pvalue() is exact, so it is used even after calibrate |
setup(model, tokenizer, device, key) | method | draw a new key or bind an existing one; return the key material |
target_logits(input_ids, logits) | method | the teacher's logits at every position: the training signal |
score(text) | method | (statistic, n_tokens); higher means more evidence of the mark |
pvalue(statistic, n_tokens) | method | optional closed-form p-value; None uses the calibrated null |
save(key_dir, key) | method | write key files (e.g. a trained model) into the private key directory |
trainable_params() | method | extra parameters trained with the model, as OpenMark's policy |
imprint_edit(model, tokenizer, out) | method | EDIT methods only: write the mark into the weights |
3Add an edit
An edit is what a user might do to the released model or to its outputs: one function in a plugin file. Give every parameter a default; the parameters name the leaderboard column.
my_edits.py @omx.register_edit # a weight edit: release in, edited release out def noise(rel, out, *, scale=0.01, seed=0, device="cuda:0"): model = materialize(rel, device) # the release as one plain model add_noise(model, scale, seed) return edited_release(rel, out, "noise", dict(scale=scale, seed=seed), model) @omx.register_text_edit # a text edit: texts in, texts out def typos(texts, *, rate=0.05, seed=0): return [swap_letters(t, rate, seed) for t in texts] arena.yaml plugins: [my_edits.py] edits: [quantize:bits=4, finetune:steps=300, noise:scale=0.01, text:typos:rate=0.05]
Built in: fine-tuning, quantization, pruning and merging on the weights; synonym, word deletion and contextual synonym on texts; image_roundtrip on image tokens (decode, then encode again, as a saved image). @omx.register_output_edit adds an output edit. examples/my_edits.py is the full file.
4Add a model
An adapter says how to load and run the model. Today openmarkx run checks a new adapter’s config and stops at its first step. Every step runs when your model goes behind the geopt adapter with factory: module:Class; docs/models.md shows both routes. Working adapters: openmarkx/models/geopt.py and openmarkx/models/taming.py.
my_model.py @omx.register_model class MyModel(omx.Model): name = "my_model" unit = "surface point" # what one output unit is output = omx.Output.FIELD # TOKENS: trained by KL; FIELD: trained by MSE def load(self, path, device): # the base (path None) or a release return load_weights(path, device) def run(self, net, inputs): # a batch of inputs in, outputs out return [net(x) for x in inputs] def forward(self, net, batch): # the pass the training step uses return batch, net(batch["inputs"]) my_run.yaml model: {name: my_model, source: my_model.py} scheme: {name: ripple, params: {eps: 0.02}} imprint: {phases: [{steps: 2000, lr: 1.0e-5, trainable: full}]} detect: {null: wrong_keys} quality: rel_l2 out: runs/my_model
openmarkx run my_run.yaml # each step resumes after an interruption openmarkx run my_run.yaml --smoke # a few steps and samples, to check the setup
Code from another project stays in its own checkout and is imported unchanged: the demos use WMAR (OMX_WMAR_DIR) and GeoPT (OMX_GEOPT_DIR). An adapter reaches such code through require_external, which stops with the name of the missing variable.
The four demos are configs in configs/demos/: taming, medgemma, geopt, agent.
5Rules for plugin files
So that a key keeps detecting the same way.
- One file. The key stores that file, so it cannot import a sibling module.
- Pinned code. The key records the file's content hash; a detector scores with exactly that code, and an edited file is refused.
- Unique names. A plugin cannot reuse a built-in method's name.
- Trust. Plugin code named by a key runs only if your process imported that file, or with
--trust-plugin(trust_plugin=True). - Key files go through
save(key_dir, key). They land in the private key store, never in the release.