Converting to and from other formats

CTRF

CTRF is the test results format a growing number of reporters write and CI integrations read (PR summaries, Slack messages, dashboards). A manifest becomes a CTRF report with one test of type: "visual" per entry, named after the screenshot, with the baseline, actual and diff images as attachments and everything CTRF has no field for under extra.visualRegression:

import fs from 'fs';
import { readManifestFile, toCtrf, fromCtrf } from 'visual-regression-manifest';

const manifest = readManifestFile('visual-regression-manifest.json');
fs.writeFileSync('ctrf-report.json', JSON.stringify(toCtrf(manifest), null, 2));

// and back: a report made by `toCtrf` converts without loss
const same = fromCtrf(JSON.parse(fs.readFileSync('ctrf-report.json', 'utf8')));

fromCtrf reads reports made by toCtrf back without loss (apart from the entry order, which is re-sorted, and the summary, which is recomputed). With { lenient: true, projectRoot } it also converts the visual tests of reports from other reporters, taking the three images from attachments named or tagged baseline/expected, actual and diff, and the status from rawStatus or from which images exist.

Playwright's toHaveScreenshot

Run with the JSON reporter and convert the report:

npx playwright test --reporter=json > report.json

import fs from 'fs';
import {
  fromPlaywrightReport,
  writeManifestFile,
} from 'visual-regression-manifest';

const manifest = fromPlaywrightReport(
  JSON.parse(fs.readFileSync('report.json', 'utf8')),
  {
    threshold: 0.01, // your maxDiffPixelRatio, the report does not carry it
    // snapshotPathTemplate / snapshotDir: only when changed in playwright.config
    // browsers: { chromium: { name: 'chromium', version: '130' } } for richer `platform` blocks
  },
);
writeManifestFile(
  'test-results/visual-regression-manifest.playwright.json',
  manifest,
);

Only comparisons that left files behind can be recovered: failed (expected, actual and diff attachments), missing-baseline and created (actual only). A passing toHaveScreenshot leaves no trace in the report, and of a retried test only the last attempt counts. Baseline paths are computed from the snapshot path template because the report only lists the copies in test-results; the diff ratio is parsed from the assertion message.

Vitest's toMatchScreenshot

Vitest (browser mode) keeps references in __screenshots__/<test file>/ next to the tests and writes <name>-actual.png and <name>-diff.png under .vitest/attachments/<test file>/ for failed comparisons. The converter scans the project for both:

import { fromVitestScreenshots } from 'visual-regression-manifest';

const manifest = fromVitestScreenshots({
  projectRoot: process.cwd(),
  threshold: 0.01, // your allowedMismatchedPixelRatio
  // roots: ['src', 'tests'], screenshotsDirName, attachmentsDir: when changed
});

A reference without attachments passed, attachments without a reference are missing-baseline, and the browser and platform are read off the file name (button-chromium-darwin).

jest-image-snapshot and cypress-image-snapshot

Baselines are <id>-snap.png in __image_snapshots__/; failures leave <id>-diff.png (a baseline | diff | received composite unless onlyDiff) in __image_snapshots__/__diff_output__/, plus <id>-received.png when storeReceivedOnFailure is on:

import { fromJestImageSnapshots } from 'visual-regression-manifest';

const manifest = fromJestImageSnapshots({
  projectRoot: process.cwd(),
  runner: { name: 'cypress' }, // for cypress-image-snapshot; defaults to jest
  // snapshotsDirName, diffDirName, receivedDirName, snapshotPostfix, diffPostfix, receivedPostfix: when customised
});

The received image is the entry's actual when it exists; the diff composite is the entry's diff image either way.

cypress-visual-regression

Reads the base, actual and diff folders (cypress/snapshots/... by default), nested by spec file:

import { fromCypressVisualRegression } from 'visual-regression-manifest';

const manifest = fromCypressVisualRegression({
  projectRoot: process.cwd(),
  threshold: 0.05, // your errorThreshold
  // baseDirectory, actualDirectory, diffDirectory, generateDiff: when changed
});

Cypress writes an actual image for every comparison, so a diff image is what marks a failure (visualRegressionGenerateDiff: 'fail', the default); pass generateDiff when the run used another mode.

BackstopJS

With "report": ["json"] BackstopJS writes backstop_data/json_report/jsonReport.json. Its paths are relative to that directory:

import fs from 'fs';
import { fromBackstopReport } from 'visual-regression-manifest';

const manifest = fromBackstopReport(
  JSON.parse(
    fs.readFileSync('backstop_data/json_report/jsonReport.json', 'utf8'),
  ),
  { projectRoot: process.cwd() }, // reportDir: when json_report was moved
);

One entry per scenario and viewport (titlePath: [label, viewportLabel], variant: viewportLabel), the mismatch percentage as diffRatio, misMatchThreshold as threshold, a Reference file not found error as missing-baseline, and the scenario's url, selector and the raw diff block under extra.backstop.

Roborazzi

compareRoborazzi* and recordRoborazzi* write build/test-results/roborazzi/<variant>/results-summary.json with absolute paths:

import fs from 'fs';
import { fromRoborazziResults } from 'visual-regression-manifest';

const manifest = fromRoborazziResults(
  JSON.parse(
    fs.readFileSync(
      'build/test-results/roborazzi/debug/results-summary.json',
      'utf8',
    ),
  ),
  { projectRoot: process.cwd(), threshold: 0.01 }, // your changeThreshold
);

changed is failed (with diff_percentage as the ratio and the _compare composite as the diff image), added is missing-baseline, unchanged is passed and recorded is updated. Entries carry an emulated Android target and a robolectric renderer; context_data lands under extra.roborazzi.

Plain image files

For any tool that leaves a baseline and, on failure, an actual and a diff image next to it:

import { fromImageTriples } from 'visual-regression-manifest';

const manifest = fromImageTriples({
  projectRoot: process.cwd(),
  runner: { name: 'my-tool' },
  triples: baselines.map((baseline) => ({ baseline })), // actual/diff default to `.actual`/`.diff` siblings
});

The status is inferred from which files exist (inferStatus), image sizes are read from the PNG headers, and every field can be given explicitly (actual, diff, name, testFile, titlePath, status, diffRatio, ...).