Skip to main content

Cypress.io

Installation​

Install the package in your Cypress project:

npm install --save-dev @orangebeard-io/cypress-listener

Quick start (step-by-step)​

Connecting Cypress.io

This video is hosted on YouTube. Playing it may transfer data to YouTube and set cookies. Click 'Accept' below to accept and enable all YouTube videos on this site.

  1. Install @orangebeard-io/cypress-listener.
  2. Provide Orangebeard connection settings (recommended: orangebeard.json).
  3. Configure Cypress to use the Orangebeard reporter + plugin.
  4. (Optional) Enable browser-side command + cy.log(...) forwarding via registerOrangebeardCommands().
  5. Run cypress run.

Configuration​

This package consists of:

  • A Mocha reporter (configured via reporter / reporterOptions)
  • A Cypress plugin (registered via setupNodeEvents)
  • Optional browser-side helpers (registered in cypress/support/*)

Orangebeard connection settings​

The reporter uses @orangebeard-io/javascript-client autoconfiguration. Settings are resolved from:

  • orangebeard.json in your project directory (or any parent directory)
  • Environment variables (can override/extend orangebeard.json)

Create orangebeard.json in your Cypress project directory (or any parent directory):

{
  "endpoint": "https://<tenant>.orangebeard.app",
  "token": "00000000-0000-0000-0000-000000000000",
  "project": "my_project",
  "testset": "My Cypress run",
  "description": "A run from Cypress",
  "attributes": [
    { "key": "branch", "value": "main" },
    { "value": "smoke" }
  ],
  "referenceUrl": "https://ci.example/job/123"
}

Required fields:

  • endpoint
  • token
  • project
  • testset (required; if missing/empty, the listener will log an error and disable reporting)

Optional fields:

  • description
  • attributes
  • referenceUrl

Environment variables (optional)​

You can configure via env vars without orangebeard.json, or override values on top of it:

ORANGEBEARD_ENDPOINT=https://company.orangebeard.app
ORANGEBEARD_TOKEN=XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX
ORANGEBEARD_PROJECT=my_project
ORANGEBEARD_TESTSET="My Cypress run"

# Optional
ORANGEBEARD_DESCRIPTION="My awesome testrun"
ORANGEBEARD_ATTRIBUTES="key:value; value;"
ORANGEBEARD_REFERENCE_URL="https://ci.example/job/123"

Configure reporter and plugin​

In cypress.config.js (or .ts), configure the reporter and register the plugin:

const { defineConfig } = require('cypress');
const registerOrangebeardPlugin = require('@orangebeard-io/cypress-listener/plugin');

module.exports = defineConfig({
  reporter: '@orangebeard-io/cypress-listener',
  reporterOptions: {
    // optional; default is true (if Cypress produces a video file)
    reportVideo: true,

    // Parallel mode (Cypress Cloud): join a shared run and do NOT finish it
    // parallelMode: true,
    // testRunUUID: process.env.ORANGEBEARD_TEST_RUN_UUID,
    // runnerId: process.env.CI_NODE_INDEX,
  },
  e2e: {
    setupNodeEvents(on, config) {
      // Required: registers tasks + screenshot/video forwarding, and signals the reporter
      // when the run is done so it can finish the Orangebeard test run. Without this,
      // the test run is never marked finished (it stays open in Orangebeard).
      registerOrangebeardPlugin(on, config);
      return config;
    },
  },

  // Enable Cypress video recording if you want videos attached in Orangebeard
  video: true,
});

Configure optional browser-side helpers​

To forward cy.log(...) output as Orangebeard logs and capture Cypress commands as Orangebeard steps, call the commands helper from your Cypress support file:

const { registerOrangebeardCommands } = require('@orangebeard-io/cypress-listener/commands');

registerOrangebeardCommands();

// Disable Cypress command step capture if you only want cy.log forwarding:
// registerOrangebeardCommands({ captureCypressCommandSteps: false });

Options​

Reporter options​

These are passed under reporterOptions in cypress.config.*.

  • disabled (boolean, default: false)

    • If true, the reporter does nothing for the whole process: no Orangebeard client/config is created, no IPC server is started, and no test run is created. Can also be set via the ORANGEBEARD_DISABLED environment variable (true/1).

  • reportVideo (boolean, default: true)

    • If true and Cypress has video: true, the reporter will attach the per-spec video (*.mp4) to the suite for that spec file.
    • If false, videos are not attached.

  • cleanupOldLockfiles (boolean, default: true)

    • If true, the reporter will delete any existing orangebeard-*.lock files before creating its own lockfile.
    • This prevents Cypress from waiting on stale lockfiles left behind by previous runs that were killed/crashed.
    • Set to false to keep the previous behavior (leave existing lockfiles in place and only warn).

  • parallelMode (boolean, default: false)

    • When true, the reporter will not finish the test run (because each parallel worker can’t know when the overall run is done).

  • testRunUUID (string)

    • Required when parallelMode: true.

  • runnerId (string)

    • Optional identifier used to disambiguate top-level suites in parallel runs.

Note: Orangebeard connection settings (endpoint/token/project/testset/...) are expected to come from orangebeard.json and/or env vars. Providing them in reporterOptions is supported for backwards compatibility, but not recommended.

Plugin options​

The plugin is registered like:

registerOrangebeardPlugin(on, config, {
  // options...
});

  • waitForLockfiles (boolean, default: true)

    • If true, the plugin waits in after:run until orangebeard-*.lock files are gone.
    • If false, Cypress may exit before the reporter finishes async IO/flush.

  • lockfilePollIntervalMs (number, default: 500)

    • Poll interval for lockfile checks.

  • lockfileTimeoutMs (number, default: 300000 (5 minutes))

    • Maximum time to wait before continuing Cypress shutdown.

  • screenshotLogFn (function)

    • Customize the log message used when attaching screenshots.

Commands helper options​

Using the optional registerOrangebeardCommands() helper, you can enable or disable browser-side command forwarding and capture Cypress command steps as Orangebeard steps.

  • captureCypressCommandSteps (boolean, default: true)
    • If true, Cypress commands are reported as steps in Orangebeard.

Running​

Run Cypress as usual:

cypress run

Running a subset of specs​

cypress run --spec "cypress/e2e/somespec/*.cy.js"

The reporter estimates how many specs will run so it knows when the last one has finished, but that estimate can't always account for --spec filtering. To cover that case, the plugin's after:run hook (registered via registerOrangebeardPlugin, see Plugin options above) tells the reporter when Cypress is actually done, so the run is finished correctly either way - as long as the plugin is registered. Without it, the test run will never be marked finished in Orangebeard.

Parallel execution​

When running Cypress in parallel across multiple CI jobs/machines (whether you split specs yourself with --spec, or use a third-party spec-splitting tool), each job runs its own cypress run process, so the reporter cannot reliably know which runner is "last". Instead, a coordinator job starts and finishes one shared Orangebeard run, and every parallel job joins it.

1: Make the config parallel-aware​

Drive parallelMode/testRunUUID from an environment variable, so the exact same config works for a normal local/CI run (env var unset) and a parallel job (env var set):

reporterOptions: {
  ...(process.env.ORANGEBEARD_TEST_RUN_UUID ? {
    parallelMode: true,
    testRunUUID: process.env.ORANGEBEARD_TEST_RUN_UUID,
    runnerId: process.env.OB_RUNNER_ID, // optional; disambiguates suite names per job
  } : {}),
},

2: Connection settings for the CLI​

orangebeard-cy (used by the coordinator to start/finish the shared run) does not read orangebeard.json - unlike the reporter, it needs the connection settings as env vars:

export ORANGEBEARD_ENDPOINT="https://<tenant>.orangebeard.app"
export ORANGEBEARD_TOKEN="00000000-0000-0000-0000-000000000000"
export ORANGEBEARD_PROJECT="my_project"

3: Coordinator: start the Orangebeard run​

RUN_UUID=$(npx orangebeard-cy start-run --testset "My Cypress run")
info

If npx orangebeard-cy hangs in your environment (some setups prompt for installation confirmation with no TTY attached), call the binary directly instead: node node_modules/.bin/orangebeard-cy start-run --testset "My Cypress run".

Pass $RUN_UUID to every parallel job (CI artifact, job output, dotenv, etc.).

4: Each job: run subset with the shared UUID​

export ORANGEBEARD_TEST_RUN_UUID="$RUN_UUID"
export OB_RUNNER_ID="job-$CI_NODE_INDEX"   # or whatever your CI calls it

npx cypress run --spec "$MY_SPEC_SUBSET"

Note: the ORANGEBEARD_ENDPOINT, ORANGEBEARD_TOKEN and ORANGEBEARD_PROJECT values are required to run the test, this example assumes they are set in the environment (recommended for token) or in orangebeard.json.

Jobs never finish the run themselves in parallel mode - only the coordinator does, in the next step.

5: Coordinator: finish the Orangebeard run once all jobs are done​

npx orangebeard-cy finish-run --testRunUUID "$RUN_UUID"

Make sure this step only runs after every parallel job has completed (e.g. via job dependencies/needs), otherwise the run is finished before all results have arrived.

GitHub Actions example​

name: E2E tests

on: [push]

jobs:
  start-run:
    runs-on: ubuntu-latest
    outputs:
      run_uuid: ${{ steps.start.outputs.run_uuid }}
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: 20 }
      - run: npm ci
      - id: start
        env:
          ORANGEBEARD_ENDPOINT: ${{ secrets.ORANGEBEARD_ENDPOINT }}
          ORANGEBEARD_TOKEN: ${{ secrets.ORANGEBEARD_TOKEN }}
          ORANGEBEARD_PROJECT: ${{ secrets.ORANGEBEARD_PROJECT }}
        run: |
          UUID=$(node node_modules/.bin/orangebeard-cy start-run --testset "My Cypress run")
          echo "run_uuid=$UUID" >> "$GITHUB_OUTPUT"

  test:
    needs: start-run
    runs-on: ubuntu-latest
    strategy:
      matrix:
        spec:
          - cypress/e2e/foo/**/*.cy.js
          - cypress/e2e/bar/**/*.cy.js
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: 20 }
      - run: npm ci
      - env:
          ORANGEBEARD_TEST_RUN_UUID: ${{ needs.start-run.outputs.run_uuid }}
          OB_RUNNER_ID: job-${{ strategy.job-index }}
        run: npx cypress run --spec "${{ matrix.spec }}"

  finish-run:
    needs: [start-run, test]
    if: always()
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: 20 }
      - run: npm ci
      - env:
          ORANGEBEARD_ENDPOINT: ${{ secrets.ORANGEBEARD_ENDPOINT }}
          ORANGEBEARD_TOKEN: ${{ secrets.ORANGEBEARD_TOKEN }}
          ORANGEBEARD_PROJECT: ${{ secrets.ORANGEBEARD_PROJECT }}
        run: node node_modules/.bin/orangebeard-cy finish-run --testRunUUID "${{ needs.start-run.outputs.run_uuid }}"

GitLab CI example​

GitLab's dotenv artifact reports propagate variables to downstream jobs automatically, so the UUID doesn't need to be passed around manually:

stages: [start, test, finish]

variables:
  ORANGEBEARD_ENDPOINT: "https://<tenant>.orangebeard.app"
  ORANGEBEARD_PROJECT: "my_project"

start-run:
  stage: start
  script:
    - RUN_UUID=$(node node_modules/.bin/orangebeard-cy start-run --testset "My Cypress run")
    - echo "ORANGEBEARD_TEST_RUN_UUID=$RUN_UUID" >> run.env
  artifacts:
    reports:
      dotenv: run.env

test:
  stage: test
  needs: [start-run]
  parallel: 4
  variables:
    OB_RUNNER_ID: "job-$CI_NODE_INDEX"
  script:
    # Split specs across CI_NODE_INDEX/CI_NODE_TOTAL however your project does today.
    - npx cypress run --spec "$MY_SPEC_SUBSET"

finish-run:
  stage: finish
  needs: [start-run, test]
  when: always
  script:
    - node node_modules/.bin/orangebeard-cy finish-run --testRunUUID "$ORANGEBEARD_TEST_RUN_UUID"

What gets reported​

  • Suites and tests from Mocha
  • Hook failures (before/after)
  • Cypress test tags (see below)
  • cy.log(...) output (if registerOrangebeardCommands() is enabled)
  • Cypress command steps (if captureCypressCommandSteps: true)
  • Screenshots (via after:screenshot) attached to the correct failing test
  • Per-spec video files (via after:spec) attached to the suite for that spec file (when video: true and reportVideo: true)

Cypress tags to Orangebeard attributes​

If you add Cypress tags to a test, they will be sent as Orangebeard attributes on the test item.

Example:

it('adds 2 todos', {
  tags: [
    '@some-tag',
    '@requirement:REQ-123',
    '@issueURL:https://gitlab.com/project/issues/1234',
  ],
}, () => {
  // ...
});

Mapping:

  • @some-tag → { value: 'some-tag' }
  • @requirement:REQ-123 → { key: 'requirement', value: 'REQ-123' }
  • @issueURL:https://gitlab.com/project/issues/1234 → { key: 'issueURL', value: 'https://gitlab.com/project/issues/1234' } -> Will translate to a clickable tag named issueURL.
Ask Orangebeard