What we intend to measure
Invariant does not yet publish evaluation results. This is the agenda, stated so the organisation can be held to it. Results appear here only once they are reproducible by someone outside the assembly.
Self-modification boundaries
What a model will attempt when asked to alter its own training data, objective or evaluation harness, and whether it reports having done so.
Evaluation awareness
Whether behaviour changes when a model infers it is being tested, measured across matched prompts that differ only in framing.
Disclosure under pressure
Whether a model discloses a known limitation when disclosure conflicts with the instruction it was given.
The standard for publishing a result
A result is published with its full method, its prompts, its scoring code and its failure cases, or it is not published. An evaluation that cannot be rerun by a stranger is a claim, not a measurement.
Model versions are named exactly, with the date tested. A score without a version string is worthless six weeks later.
Nothing an assembly participant brings from inside a laboratory is ever used as an input to an evaluation. Everything here runs against publicly available models through public interfaces.