> ## Documentation Index
> Fetch the complete documentation index at: https://docs.stacknext.space/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> StackNext documentation is written for developers using StackNext in a Next.js App Router project. Use the getting-started and guides sections to install and operate the product, and the reference section for CLI, configuration, MCP and package surfaces. Version-matched package API reference ships inside each npm package under node_modules/@stacknext/<package>/docs/ and is preferred over this site for installed-version APIs. Diagnostic findings are evidence, not confirmed root causes; regression comparisons are deterministic and report-only. Contributor and repository-maintenance documentation is not part of this site.

# CI integration

> Run capture, comparison and evidence artifacts in a normal CI job.

StackNext regression checks fit into a normal CI job without a hosted StackNext
service. The CI runner is the machine: the collector binds to loopback, the
Playwright browser runs there, and run, baseline and comparison files stay in
the workspace. The example uses GitHub Actions, but the same steps work in any
runner that can run Node.js, Playwright and the app.

> [!NOTE]
>
> The default is report-only. A job fails only when `stacknext compare` is given a
> matching `--fail-on` option.

## What the job needs

* Node.js 20.9 or newer.
* pnpm or npm with the repository's lockfile.
* The app built and able to start in the runner.
* Playwright browsers installed (`pnpm exec playwright install --with-deps chromium`).
* One Playwright worker.
* A baseline whose recorded environment matches the candidate runner.
* At least three verified samples per scenario.

Do not run captured Playwright tests in parallel. Correlation is verified per
injected trace with one worker; a second worker can make the sample set
incomplete.

## GitHub Actions example

This example assumes the app directory is `apps/my-app`, the Playwright config
starts the app through `webServer`, and a baseline file is available from a
previous baseline job or artifact. Adjust paths and package manager commands.

```yaml theme={null}
name: runtime-regression

on:
  pull_request:
  workflow_dispatch:

permissions:
  contents: read

concurrency:
  group: runtime-regression-${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true

jobs:
  regression:
    runs-on: ubuntu-24.04
    timeout-minutes: 30
    env:
      CI: 'true'
      NEXT_TELEMETRY_DISABLED: '1'
    defaults:
      run:
        working-directory: apps/my-app
    steps:
      - uses: actions/checkout@v4

      - uses: pnpm/action-setup@v4
        with:
          version: 12.16.1

      - uses: actions/setup-node@v4
        with:
          node-version: 22
          cache: pnpm

      - name: Install dependencies
        working-directory: .
        run: pnpm install --frozen-lockfile

      - name: Install Playwright Chromium
        run: pnpm exec playwright install --with-deps chromium

      - name: Build
        working-directory: .
        run: pnpm build

      # Restore a baseline captured on the same OS/Node/Next/Playwright setup.
      - name: Restore baseline
        run: |
          mkdir -p .stacknext/baselines
          cp ci-baselines/base-0123456789abcdef.json .stacknext/baselines/

      - name: Capture candidate
        run: |
          set -o pipefail
          pnpm exec stacknext capture \
            --name "ci candidate" \
            --quiet --json \
            -- pnpm exec playwright test --workers=1 --repeat-each=3 \
            > candidate.json

      - name: Compare with the baseline
        run: |
          CANDIDATE_ID=$(jq -r .run.id candidate.json)
          pnpm exec stacknext compare \
            base-0123456789abcdef "$CANDIDATE_ID" \
            --fail-on regression \
            --json > comparison.json

      - name: Upload regression records
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: stacknext-regression
          path: |
            apps/my-app/candidate.json
            apps/my-app/comparison.json
            apps/my-app/.stacknext/runs/
            apps/my-app/.stacknext/baselines/
            apps/my-app/.stacknext/comparisons/
          if-no-files-found: ignore
```

`--fail-on regression` makes the compare step exit 1 only for a `regressed`
comparison. Use `--fail-on inconclusive` to fail when evidence is insufficient,
or `--fail-on any` to fail on either.

The capture step exits non-zero when the child test command fails, and exits
**3** when the child succeeds but telemetry is incomplete. Do not ignore that
exit code.

## Where the baseline comes from

The baseline must be produced by an earlier capture and saved with
`stacknext baseline save`. Pick one approach:

### Same-runner baseline job

Run both the baseline and candidate captures in one workflow on the same runner
type. This is the most reliable option because Node.js, Next.js, Playwright, OS
and database versions are identical.

1. Check out the known-good revision in a separate directory or worktree.
2. Install, build and start that revision.
3. Capture it and save a baseline.
4. Check out the candidate revision in the primary workspace.
5. Capture the candidate and compare.

Keep ports, app names and collector processes separate between the two
revisions. Do not compare runs from two matrix entries.

### Stored baseline artifact

Save a baseline from the main branch once, upload the JSON as an artifact, and
download it into later PR jobs. Downloading from a different workflow run needs
the source `run-id` and a token with `actions: read`:

```yaml theme={null}
permissions:
  contents: read
  actions: read

# In the baseline job:
      - name: Upload baseline artifact
        uses: actions/upload-artifact@v4
        with:
          name: stacknext-baseline
          path: apps/my-app/.stacknext/baselines/base-*.json
          if-no-files-found: error

# In a later candidate job:
      - name: Download baseline artifact
        uses: actions/download-artifact@v4
        with:
          name: stacknext-baseline
          path: apps/my-app/.stacknext/baselines
          run-id: ${{ vars.STACKNEXT_BASELINE_RUN_ID }}
          github-token: ${{ secrets.GITHUB_TOKEN }}
```

Re-capture the baseline whenever Node.js, Next.js, Playwright, the database setup
or the instrumentation flags change. A stale artifact is not a stable reference.

### Reviewed baseline file

A team can keep a reviewed baseline JSON in a directory outside `.stacknext`,
for example `ci-baselines/`, and copy it into the workspace at the start of the
job. Review it before committing; it contains redacted route, span, query, error
and environment data. Never commit `.stacknext` runtime state or tokens.

## Capture wiring

### App started by Playwright `webServer`

`stacknext capture` starts an ephemeral collector and passes the run id,
collector URL and token to the child. Playwright's `webServer` inherits the
environment and starts the app:

```ts theme={null}
export default defineConfig({
  testDir: './e2e',
  fullyParallel: false,
  workers: 1,
  retries: 0,
  webServer: {
    command: 'pnpm dev',
    url: 'http://127.0.0.1:4402',
    reuseExistingServer: !process.env.CI,
  },
  use: {
    baseURL: process.env.STACKNEXT_APP_URL ?? 'http://127.0.0.1:4402',
    trace: 'off',
    screenshot: 'off',
    video: 'off',
  },
})
```

Then:

```bash theme={null}
pnpm exec stacknext capture \
  --name "ci candidate" --quiet --json \
  -- pnpm exec playwright test --workers=1 --repeat-each=3 \
  > candidate.json
```

If the runner is slow to start the dev server, raise the fixture timeout in the
child environment:

```bash theme={null}
STACKNEXT_CAPTURE_TIMEOUT_MS=30000 \
STACKNEXT_CAPTURE_SETTLE_MS=1000 \
pnpm exec stacknext capture --name "ci candidate" --quiet --json \
  -- pnpm exec playwright test --workers=1 --repeat-each=3 \
  > candidate.json
```

### App started by `stacknext dev`

If the workflow starts the app separately, run `stacknext dev` in the app
directory and wait for both the collector and the app before capturing. Capture
attaches to the live collector when `.stacknext/run.json` is valid.

```bash theme={null}
pnpm exec stacknext dev --quiet > stacknext-dev.log 2>&1 &
echo $! > stacknext-dev.pid

for attempt in $(seq 1 60); do
  curl -fsS http://127.0.0.1:4400/api/ping > /dev/null && break
  sleep 1
done

# Warm route modules so one-time spans do not appear only on one side.
for path in / /projects/1 /api/n-plus-one; do
  curl -fsS -o /dev/null "http://127.0.0.1:4402$path"
done
```

Always stop the process with SIGINT or SIGTERM in a cleanup step so it closes
the collector and removes `.stacknext/run.json`:

```yaml theme={null}
      - name: Stop StackNext
        if: always()
        run: |
          if [ -f stacknext-dev.pid ]; then
            kill -TERM "$(cat stacknext-dev.pid)" || true
          fi
```

Check ports 4400-4425 before the capture step if the runner reuses processes.

## Thresholds and environment

Keep the defaults unless you have measured your own noise. The comparison
requires three verified samples per side by default; duration regressions must
cross both 20% and 25 ms. See
[comparison methodology](https://docs.stacknext.space/regression-testing.md#comparison-methodology) for the
exact ordering.

Pin the runner image and versions. The comparison records Node.js, Next.js,
platform, architecture, runtime, enabled adapters, instrumentation flags, SDK
version, Playwright version and project name, and defaults to `inconclusive`
when they differ. Do not build a baseline on macOS and compare it with a Linux
candidate unless you explicitly accept `--ignore-environment` and its warning.

## Artifacts and privacy

Upload the capture JSON, `.stacknext/runs/run-*.json`,
`.stacknext/baselines/base-*.json`, `.stacknext/comparisons/cmp-*.json` and the
Playwright report if your project produces one.

The records contain redacted scenario titles and file paths, normalized routes
and SQL shapes, bounded span/error evidence, timings and environment facts. They
do not contain request/response bodies, headers, cookies, URL query values, SQL
bind parameters, screenshots, video or Playwright trace archives. They are still
local debugging artifacts; review them before sharing artifacts outside the
team. Never upload `.stacknext/run.json` or any file containing the collector
bearer token.

## Concurrency and cleanup

* Use one capture per collector.
* Use one Playwright worker for captured runs.
* Give parallel CI jobs separate ports or separate ephemeral collectors.
* Always stop `stacknext dev` with a signal and check for orphaned `next dev`
  processes.
* Set a job timeout.
* Keep `CI=true` so Playwright does not reuse a developer's server.

## What CI does not do

* It does not upload telemetry to StackNext or any hosted service.
* It does not run an LLM or ask an AI provider to classify a regression.
* It does not take screenshots, record video or read Playwright trace archives.
* It does not prove statistical significance or causality.
* It does not make an `inconclusive` comparison a pass.
* It does not automatically delete old baselines or comparisons.

## Related

* [Regression testing](https://docs.stacknext.space/regression-testing.md)
* [Playwright integration](https://docs.stacknext.space/playwright-integration.md)
* [CLI commands](https://docs.stacknext.space/cli/commands.md)
* [Privacy](https://docs.stacknext.space/privacy.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.