Quick Start

This page covers the four-step canonical workflow, end to end:

novetest init — create the per-project store under .novetest/.
novetest test — run tests, derive coverage / regression / localization, synthesize a recommendation.
Read (human) or route on (agent) the recommendation.
(Optional) novetest inspect <run_id> — drill into one run.

Throughout, the running example is calc, a tiny Python + pytest package with three green tests:

calc-demo/
├── pyproject.toml          # [tool.pytest.ini_options] testpaths=["tests"] pythonpath=["."]
├── calc/
│   ├── __init__.py
│   └── arithmetic.py       # def add(a,b): return a+b   /   def subtract(a,b): return a-b
└── tests/
    └── test_arithmetic.py  # test_add_positive, test_add_zero, test_subtract

You need python >= 3.11 and pytest on PATH (the native engine Nove Test shells out to). The novetest binary bundles its own Python — see Installation.

Not a Python project?

The four steps below are engine-independent: the commands, the novetest/v1 envelope, the exit codes and the routing logic are the same for all six engines. Only the project skeleton and the toolchain differ. The JavaScript / TypeScript equivalent of calc:

cart-demo/
├── package.json            # the detection marker; jest in devDependencies
├── node_modules/           # created by `npm install`
├── src/
│   └── promotions.js
└── __tests__/
    └── promotions.test.js

json

{
  "name": "cart-demo",
  "private": true,
  "scripts": { "test": "jest" },
  "devDependencies": { "jest": "^30.0.0" }
}

Pin whatever jest version your project already uses — Nove Test reads the installed version from node_modules/jest/package.json and never installs anything itself. Readiness needs Node.js >= 18 with both node and npx on PATH, plus jest actually installed: jest declared in package.json but not installed reads as engine-misconfigured with the hint npm install, and jest absent from package.json altogether reads as engine-misconfigured with the hint npm install --save-dev jest. Then novetest init detects package.json, pins jest, and every step below works unchanged.

Go, Rust, Java and .NET projects follow the same pattern with their own marker file — the per-engine markers, toolchains, node-id forms and coverage availability are in Supported Languages.

Step 1 — `novetest init`

Run once from the project root.

For human

bash

cd calc-demo
novetest init

✓ Initialized .novetest/ at /home/you/calc-demo/.novetest
  engine readiness: ready — python/pytest 9.0.3

Nove Test detected pytest from pyproject.toml and pinned it — every later verb runs the pinned engine; nothing is re-detected at run time. ready means the engine resolved; the 9.0.3 is your pytest version. If you see engine-missing / engine-misconfigured, the next issue: line says what to fix (Troubleshooting).

For agent

bash

NOVETEST_OUTPUT=json novetest init

json

{
  "command": "init",
  "data": {
    "engine_readiness": {
      "ecosystem": "python", "engine": "pytest", "engine_version": "9.0.3",
      "evidence": ["pyproject.toml"], "issues": [], "state": "ready"
    },
    "initialized_at": 1782370092699,
    "pinned_engine": {"ecosystem": "python", "engine_name": "pytest"},
    "store_path": "/home/you/calc-demo/.novetest",
    "store_state": "ready"
  },
  "errors": [], "ok": true, "schema": "novetest/v1", "warnings": []
}

Gate "can I run tests?" on data.engine_readiness.state == "ready". The three real states are ready / engine-missing / engine-misconfigured (there is no engine-not-ready). init exits 0 even when an engine is missing — the store is still created.

init creates the .novetest/ Project Store:

.novetest/
├── store.json          # schema_version, initialized_at, store_state
├── blobs/
├── memory/{runs,tombstones}/
├── run/                # run/artifacts/ created on first run
├── coverage/           # coverage/facts/ on first coverage derive
├── regression/         # regression/pairs/ on first comparison
├── localization/       # localization/findings/ on first SBFL run
├── replay/             # replay/results/ on first replay
└── orchestration/      # reserved; recommendations are computed live, not stored

Each engine creates its leaf subdirectory lazily on first write, so right after init only the parent directories above exist. Nove Test finds the store by walking up from your current directory (like git finds .git/). Run a verb with no .novetest/ in any ancestor and you get uninitialized (exit 2).

Step 2 — `novetest test`

The headline verb: it runs the tests, stores a Run Record, derives coverage / regression / localization where it can, and synthesizes a recommendation. It always collects coverage (there is no --coverage flag on test; only the lower-level novetest run has --coverage / -c).

For human

bash

novetest test

1 recommendation · 1 category · run_id=01KVYRJJJ75ZRHC05GNKYRK99S

  ✓ [all_green] All tests green; no action recommended (passed 3, skipped 0, total 3).
      ↳ run_reference 01KVYRJJJ75ZRHC05GNKYRK99S

Exit code 0. The header gives the count + run_id; the block is a glyph + [category] + summary + ↳ citation. [all_green] is one of a closed set of seven categories, each with an integer priority (1 = most urgent … 7 = all green).

For agent

bash

NOVETEST_OUTPUT=json novetest test

json

{
  "command": "test",
  "data": {
    "recommendation_schema_version": 1,
    "recommendations": [
      { "category": "all_green", "priority": 7,
        "recommendation_id": "rec_01KVYRJJJ75ZRHC05GNKYRK99S_908389d6",
        "summary": "All tests green; no action recommended (passed 3, skipped 0, total 3).",
        "slots": { "passed": 3, "skipped": 0, "total_tests": 3,
                   "run_reference": "01KVYRJJJ75ZRHC05GNKYRK99S" },
        "evidence_citations": [ { "kind": "run_reference", "selector": {},
          "run_reference": { "run_id": "01KVYRJJJ75ZRHC05GNKYRK99S",
                             "created_at": 1782370093639, "schema_version": 1 } } ] }
    ],
    "run_reference": { "run_id": "01KVYRJJJ75ZRHC05GNKYRK99S",
                       "created_at": 1782370093639, "schema_version": 1 },
    "stage_eligibility": { "coverage": "available", "localization": "unavailable",
                           "regression": "available", "replay": "not_run" }
  },
  "errors": [], "ok": true, "schema": "novetest/v1", "warnings": []
}

Recommendations carry an integer priority (no severity field). stage_eligibility.localization is the SBFL mode when available; replay is always not_run. See the exit-code contract below.

Shortcut: novetest <path> ≡ novetest test <path> — any first argument that is not a known verb is treated as a test target. Bare novetest prints help; it does not run tests.

Exit-code contract

exit	`ok`	meaning
0	true	tests passed
3	true	tests failed (a result, not a crash) — read `recommendations[]`
2	false	usage / validation / `uninitialized` / unknown run_id
4	false	engine missing / adapter error
5	false	storage error
1	false	generic failure

A failing run is exit 3 with ok: true. Do not treat it as a tool error.

Step 3 — read / route on the recommendation

For human

On the happy path there is nothing to do — [all_green] means everything passed. When a test fails, test emits one or more [investigate_location] recommendations that pin the most suspicious code (and exits 3). The full green → bug → fix walkthrough is in Understanding Results.

novetest status summarizes the latest run's analysis availability:

latest run · 01KVYRJK97SSR5DR840PH26VQK · history: 4 runs

  — coverage      unavailable
  — regression    unavailable
  — localization  unavailable
  — replay        unavailable

— means "unavailable for a structural reason" (no coverage collected, no failing tests to localize, etc.) — not an error.

For agent

python

recs = env["data"]["recommendations"]
top = min(recs, key=lambda r: r["priority"])   # lowest priority int = top category
if returncode == 0 and top["category"] == "all_green":
    pass
elif returncode == 3:
    # Up to and including v0.2.1 this array is NOT score-ordered (position is
    # emission order). From v0.3.0 it IS ordered: rank (asc) then score_normalized
    # (desc) within each priority/category group. Either way, select on the
    # fields below rather than trusting position — that is correct on every version:
    locs = [r for r in recs if r["category"] == top["category"] and "rank" in r["slots"]]
    if locs:   # rank-bearing category (investigate_location, regression_with_localization)
        key  = lambda r: (r["slots"]["rank"], -r["slots"]["score_normalized"])
        best = min(locs, key=key)
        # A tie on BOTH rank and score is a real result, not noise: the analysis
        # could not separate those locations, and what is left of the order is
        # decided by file path. Take every tied entry as a co-equal suspect.
        tied = [r for r in locs if key(r) == key(best)]
        for r in tied:
            act_on(r["slots"])   # -> ["file"], ["primary_line"], ["symbol"]
    else:      # top category carries no rank — never sort; handle each entry
        for r in [r for r in recs if r["category"] == top["category"]]:
            act_on(r["slots"])   # keys differ per category — see below

Route on the lowest priority integer for the top category; within a rank-bearing category, rank by slots.rank/score_normalized (not array order). Only investigate_location and regression_with_localization carry slots.rank — investigate_regression routes by slots.test_id, coverage_gap carries slots.file plus a slots.lines list (no rank, no primary_line), and unavailable_analysis carries no location at all. That is why the if locs: guard above is load-bearing: drop it and min() crashes on an empty list exactly when one of those categories tops the array. When two or more entries tie on both slots.rank and slots.score_normalized, they are co-equal suspects — investigate all of them, preferring a non-test source file, rather than only the first. The full decision tree is in Understanding Results.

Step 4 — (optional) `novetest inspect <run_id>`

A pure read (executes nothing) that aggregates the four derived sub-reports for one run.

For human

bash

novetest inspect 01KVYRJJJ75ZRHC05GNKYRK99S

✓ 01KVYRJJJ75ZRHC05GNKYRK99S · passed · pytest (python) · target=<workspace>

  coverage      ✓ per-test · 13/13 statements (100.0%)
  regression    ✓ clean · regressed=0 fixed=0 still_failing=0
  localization  — unavailable (missing-derived-facts)
  replay        ? unavailable (missing-derived-facts)

Coverage is present (test always collects it); regression is clean (compared against the prior run); localization is unavailable (a passing run has nothing to rank); replay is unavailable until you explicitly run novetest replay <run_id>.

For agent

bash

NOVETEST_OUTPUT=json novetest inspect 01KVYRJJJ75ZRHC05GNKYRK99S

Each sub-report under data is a discriminated union — switch on kind ("fact-set" vs "unavailable"):

json

{ "command": "inspect",
  "data": {
    "coverage_outcome":   { "kind": "fact-set", "mapping_granularity": "per-test",
                            "summary": { "percent_covered": 100.0, "num_statements": 13, "...": "..." } },
    "regression_outcome": { "kind": "fact-set",
                            "summary": { "regressed": 0, "still_passing": 3, "...": "..." }, "...": "..." },
    "localization_outcome": { "kind": "unavailable", "reason": "missing-derived-facts", "...": "..." },
    "replay_outcome":     { "kind": "unavailable", "reason": "missing-derived-facts", "...": "..." },
    "run_summary": { "engine_name": "pytest", "status": "passed", "...": "..." },
    "sub_reports": { "coverage": "available", "localization": "unavailable",
                     "regression": "available", "replay": "unavailable" } },
  "errors": [], "ok": true, "schema": "novetest/v1", "warnings": [] }

A stale/unknown run_id → errors[].code == "not-found", exit 2.

Run the loop automatically — hooks and CI

Steps 1–4 are the manual form of the loop. Once it works by hand, wire it into your own workflow as a hook — post-edit, pre-commit, pre-push, or a CI step — so novetest test fires on every change without anyone (human or agent) remembering to run it. Hook Setup is the complete, agent-agnostic recipe: choosing the trigger point, a copy-as-is POSIX hook script, and the one interpretation rule any automation must keep — exit 3 is routed data, not a hook crash (Step 2's exit-code contract, applied from inside automation).

Quick Start

Not a Python project?

Step 1 — novetest init

Step 2 — novetest test

Exit-code contract

Step 3 — read / route on the recommendation

Step 4 — (optional) novetest inspect <run_id>

Run the loop automatically — hooks and CI

What to read next

Step 1 — `novetest init`

Step 2 — `novetest test`

Step 4 — (optional) `novetest inspect <run_id>`