The benchmarking platform for physical intelligence.
Robot policies are graduating from demos to deployments — and their scores need to
mean something. pera runs every model on canonical, versioned protocols: same tasks,
same seeds, same rules. Every number is anchored to the authors' own harness on the
same machine, and every score links to episode videos and a checksummed evidence
bundle. Where the policy succeeds, where it fails, and why.
Each benchmark runs one canonical protocol: pinned task set, exact instructions,
derived per-episode seeds, fixed step limits, versioned simulator. Deviations would be a
different protocol version — never a silent change.
Method, briefly
Models talk to benchmarks through typed contracts: a policy declares its cameras, state format,
and action conventions; the engine resolves the pair as direct, adapted (explicit,
recorded transforms) or refused — a mismatched pair cannot silently produce garbage.
Episodes are seeded deterministically; infrastructure faults are excluded from scores and retried.
Every published run exports a bundle — seeds, action transcripts, per-episode outcomes, videos —
whose checksums can be checked against the root published with its result. Failure episodes are
shown with a fixed-taxonomy label or an explicit untagged state, and each result discloses how
many labels came from viewed episode frames versus task-pattern inference.
How failure labels are made
Labels use pera.failure-taxonomy.v1. In a full review, every failure is
inspected through an eight-frame contact sheet. For large failure sets, sampled review
inspects two episodes per task; remaining labels may be inferred from the viewed task
pattern, carry confidence 0.4, and retain the disclosure suffix
“(pattern-inferred from sampled episodes of this task)”.
Each result publishes structured coverage, per-episode method and confidence, the tagger
identity, and the separately hashed annotation sidecar. There are no separate saved tagger
prompt files.
What “env-verified” means here
Current env-verified rows were run by pera curators through the Environment API. The tier
attests environment execution, the declared protocol, scoring, action transcript, and
evidence. Curators know model identity because they operated the named models, but these
unsigned Phase 1 bundles do not independently or cryptographically prove platform or model
identity.
Conflict-of-interest disclosure: PulseVLA is verapulse’s own model. Its
four published rows use the same protocols, evidence surface, and disclosure rules as every
other model, including results that compare unfavorably.