Reference

Troubleshooting & FAQ

Nearly every problem with SpecTracer is one of five things, and four of them are tag mismatches. Work down this page in order.

Coverage reads 0% (or far lower than expected)

The report generated fine, the tests clearly ran, and the number is zero. Check these in order — the first two account for most cases.

Open the Unlinked Tests page

If your results are listed there, they were parsed successfully but their @scenario: values matched no @id:. That's a tag mismatch, not a config problem — jump to the next section.

If the page is empty and coverage is still zero, the results probably aren't being read at all. Check the pyramid counts: if a layer shows 0 tests, go to A layer shows 0 tests.

Check the two tag prefixes aren't swapped

Scenarios take @id:FC-42. Test results take @scenario:FC-42. Using @scenario: on the Gherkin scenario — an easy mistake, since it reads more naturally — links nothing.

Check the values match exactly

FC-42 ≠ FC42 ≠ fc-42 ≠ FC-042. Matching is exact string equality with no normalisation.

Check the tags are on scenarios, not the feature

Tags on the Feature: line are not inherited. Each scenario needs its own @id:.

Check `features` points where you think

If the configured feature paths resolve to nothing, there are no scenarios to cover and the denominator collapses. The Feature Breakdown page will be empty — a quick tell.

Everything lands in Unlinked Tests

The results parsed, the tags were found, and nothing matched. Almost always one of:

Cause	Fix
Prefix typo — `@scenarios:`, `@Scenario:`, `@scenario-`	The prefix is exactly `@scenario:`, lowercase, colon-terminated.
Value drift after a rename	A scenario's `@id:` was changed and its tests weren't. Search the codebase for the old value.
The scenario's feature file isn't in `features`	The tests are fine; SpecTracer just never loaded the scenario they point at. Widen the `features` paths.
The scenario has no `@id:` at all	Nothing can link to a scenario without an identity tag. Add one.
Framework mangles the tag in the test name	Some runners rewrite or truncate test names. Move the tag to a JUnit `<property>` instead — see where tags are read from.

Quick sanity check Pick one result from the Unlinked page, copy its @scenario: value, and grep your feature files for @id:<that value>. If grep finds nothing, the mismatch is on the scenario side. If grep finds it, the feature file isn't being collected.

A layer shows 0 tests

The pyramid shows an empty bar for a layer you definitely ran.

The core gotcha An empty or missing test-result path is silently ignored — that layer just contributes zero tests. This keeps a half-wired pipeline from crashing, but it means a typo'd path and "no tests yet" look identical in the report.

Work through:

Does the file exist where the config says? Relative paths resolve against the config file's directory, not your shell's working directory.
Right extension for the layer? Directory scans look for .xml under unit / integration, and .json under e2e. Pointing e2e at a directory of JUnit XML finds nothing.
Did the test job actually write the file? A suite that crashed before producing output leaves no file — and SpecTracer will report that as zero tests rather than failing. Make the test job itself fail loudly.
In CI, did the artifact arrive? With parallel jobs, the report job needs the results downloaded into the workspace before spec-tracer runs. See parallel test jobs.

Or: a layer shows twice as many tests as it should

You've pointed two layers at the same directory. Scanning ./reports for both unit and integration registers every XML file under both. Give each layer its own subdirectory.

A module-scoped requirement never gets satisfied

@require-unit:auth stays red even though the unit test exists, links correctly, and passes.

Module matching is strict, and this is the trap: an unscoped result never satisfies a scoped requirement. If your unit XML is registered under the "" key, no amount of @require-unit:auth tagging will be satisfied by it.

✗

Won't work

"unit": { "": ["./reports/unit.xml"] } paired with @require-unit:auth.

✓

Works

"unit": { "auth": ["./reports/auth-unit.xml"] } paired with @require-unit:auth.

Two ways out: split the result files by module in the config, or drop the :module suffix and use a bare @require-unit, which any unit result satisfies. Also check for a plain spelling mismatch — the config key and the tag suffix must be character-identical ("auth" vs @require-unit:Auth won't match).

Error messages

Situation	What happens
No config file found, none specified	Errors out. A config file is mandatory.
Config missing `features` or `output`	Errors out — both keys are required.
Invalid value in `fail_on`	Config error. Only `progress`, `pyramid`, and `e2e_runtime` are accepted.
Malformed JUnit XML	Aborts with a clear message naming the offending file.
Malformed Cucumber JSON	Aborts on the JSON decode error. The message is not contextualised with the file path — with several E2E files you may have to bisect to find the bad one.
Empty or missing test-result path	Silently ignored; zero tests for that layer.
Config path passed as `.\config.json` in Git Bash / WSL	The backslash is consumed as an escape character. Use `./config.json` or a bare filename.

Behaviour reference

Everything the tool does that might surprise you, in one table.

Input	Behaviour
Scenario with no `@require-*`	Defaults to a bare, unscoped `@require-e2e`.
Scenario with no `@id:`	Unlinkable. Reported as permanently incomplete.
Test matching no scenario	Listed under Unlinked Tests. Still counts toward pyramid and pass-rate stats.
Scenario matching no test	Shown as incomplete, with each declared layer flagged missing.
Scenario with `@require-*` but no matching test for that layer	That specific layer is flagged missing; other satisfied layers still count.
Two scenarios sharing an `@id:`	One matching result links to both. Coverage is inflated — keep ids unique.
One result tagged with several `@scenario:` values	Links to every scenario carrying any of those ids. This is intentional and supported.
Tags on the `Feature:` line	Ignored. Not inherited by scenarios.
`Scenario Outline:` with `Examples:`	Parsed as one scenario, named from the `Scenario Outline:` line. Examples rows are not expanded individually.
`Rule:`, `Background:`, non-English dialects	Not interpreted. SpecTracer understands only `Feature:`, tags, `Scenario:` / `Scenario Outline:`, and steps.
Unicode and special characters	Preserved, and HTML-escaped in the report.
Output file already exists	Overwritten. Parent directories are created if missing.

FAQ

Do I have to use Gherkin?

Yes — .feature files are how you tell SpecTracer what the scope is. But you don't have to run them. A feature file that no BDD framework executes is still a perfectly valid input: it becomes a machine-readable list of the scenarios you expect to have covered, with unit and integration tests pointing back at it.

Does this replace code coverage?

No. Line coverage measures which code executed; SpecTracer measures which specified behaviours have tests, and at which layers. They answer different questions and they don't overlap — 100% line coverage is entirely compatible with an untested checkout flow.

My project isn't Python. Can I still use it?

Yes. SpecTracer happens to be written in Python, but it only reads .feature files, JUnit XML, and Cucumber JSON. It never touches your source. You'll need a Python 3.12+ runtime on the machine that generates the report — usually just one CI step.

Can one test cover several scenarios?

Yes. Tag it with several @scenario: values and it links to all of them. Whether that's good testing practice is your call; the tool supports it.

Why is "scenarios fully matched" so much lower than "declared tests matched"?

Because they're different fractions. The first counts scenarios where every declared layer is satisfied; the second counts individual satisfied layer requirements. A scenario needing three layers and having two contributes 2/3 to the second number and 0 to the first. A wide gap means coverage is spread thin rather than completed.

Can I track coverage over time?

SpecTracer stores nothing between runs, deliberately. Set output_json and post summary to whatever metrics system you already run — one curl in CI. See Historical trends.

Can I use boolean tag expressions?

No. Matching is exact string equality — no and / or / not, no wildcards. Do selective execution in your test runner's own tag filtering, before the results reach SpecTracer.

Is the HTML report safe to publish?

It contains your feature names, scenario text, test names, and failure stack traces. That's usually fine for an internal artifact and worth a second thought before putting it on a public URL. It makes no network requests other than the CDN webfont.

Can I customise the report's appearance?

Not through configuration. The template lives in spec_tracer/renderers.py and the project is MIT-licensed, so forking is an option — but the theme, light/dark handling, and layout are not exposed as settings.

Something's still broken.

Open an issue on GitHub. A minimal reproduction — one feature file, one result file, and your config — makes it dramatically faster to diagnose.