Manifest

The run manifest

One JSON file per test run that lists every screenshot comparison, built upon CTRF, written by several tools and read by PR bots and review apps.

Every visual regression tool ends a run the same way: some comparisons passed, some failed, some had no baseline yet, and the images are somewhere on disk. What happens next, a comment on the pull request, a review screen, a button that approves a baseline, needs that information in a form a program can read. Today almost every tool leaves image folders and nothing else, and every review tool parses logs or walks directories to work out what happened.

The visual regression manifest is that missing file: visual-regression-manifest.json, one per run, listing every comparison with its status, diff ratio, threshold and the paths of the baseline, actual and diff images, plus where the run happened (CI provider, repository, commit, pull request) and what ran it.

The format

An annotated example, the status table and the rules every writer and reader follows.

Schema reference

Every field, generated from the JSON Schema at /schema/v1.json.

Converting

Make a manifest from Playwright, Vitest, jest-image-snapshot, cypress-visual-regression, BackstopJS, Roborazzi or plain image files; export to CTRF.

Viewer

Drop a manifest and its images into the browser and review them. Nothing is uploaded.

Built upon CTRF

CTRF, the Common Test Report Format, is the one cross-tool test results format with a growing ecosystem of reporters and CI integrations. It has no notion of a screenshot comparison: no baseline, actual and diff triple, no diff ratio, no "needs a baseline" outcome. Rather than invent a parallel vocabulary, the manifest uses CTRF's names wherever CTRF has a field (reportFormat, specVersion, tool, summary, commit, buildId, osPlatform, extra, attachments) and adds only what a visual comparison needs.

The result is a file that converts to a CTRF report and back without renaming anything: toCtrf writes one CTRF test of type: "visual" per comparison with the images as attachments, so every CTRF-aware PR summary or dashboard shows your visual results today, and fromCtrf reads it back. A proposal to document the visual fields upstream is open with the CTRF maintainers.

Implementations

The manifest is not a standard. It is an open format with one maintainer, a schema, a reference library and converters, and this table is the honest state of adoption. It becomes a standard the day a second tool writes or reads it without asking.

Writes it natively	Converted by `visual-regression-manifest`	Reads it
pixsame Cypress plugin	Playwright `toHaveScreenshot`, Vitest `toMatchScreenshot`, jest-image-snapshot, cypress-image-snapshot, cypress-visual-regression, BackstopJS, Roborazzi, any tool that leaves baseline/actual/diff images	the viewer, the pixsame GitHub App (in preparation), anything that reads CTRF via `toCtrf`

Adding your tool to the middle column is one file and one test in the repository; adding it to the first is ManifestWriter and a few lines in your plugin. Both are welcome.

Install

npm install visual-regression-manifest

The package has no runtime dependencies and ships the JSON Schema, TypeScript types, a reader with validation, a writer that keeps the file complete even when the run is killed, a merger for runs split across machines, and the converters.

Edit this pageorReport an issue

Compare the tools

Runner integrations, hosted review services, diff engines and mobile tools, compared on facts from their own docs.

The format

The normative description is the JSON Schema (draft 2020-12) at https://visual-regression.dev/schema/v1.json, shipped as schema.json (in the package, and in the repository under src/) and importable as visual-regression-manifest/schema.json or the manifestSchema export.