Verifying with tests

A requirement is verified only by recorded test results, never by editing a file. rqtk records which tests passed against which version of each requirement. When the requirement changes afterwards, that evidence turns Suspect until the tests run again.

The loop

1. Link a test to a verification activity.
2. Run the tests with JUnit XML output.
3. Record the results with rqtk verify.
4. Check with rqtk coverage --strict, and commit .rqtk/evidence.toml with the code.

Linking tests

Put the activity ID directly above the test. Only blank lines, comments, other tags and attributes (#[test], @Test, @pytest.mark.parametrize(…), [Fact]) may sit in between; the first other line must be the test.

rqtk recognises these test declarations:

rqtk scan shows what each link is attached to. A tag with no test declaration below it can never match a result; rqtk lint reports it as an error (RQ030).

A test can verify several activities, and an activity can have several tests; it passes only when all of them pass. rqtk lint also reports links to unknown activities (RQ028).

One case of a table-driven test

Tag the case’s row, and name the case:

func TestIntact(t *testing.T) {
    cases := []struct{ name string; flip bool }{
        // rqtk: verifies VA-SYS-0003-01 case "flipped byte is corrupt"
        {"flipped byte is corrupt", true},
        // rqtk: verifies VA-SYS-0003-02 case "intact file passes"
        {"intact file passes", false},
    }
    // …
}

The link belongs to the enclosing test and matches the runner’s result for that case: Go’s TestIntact/flipped_byte_is_corrupt, or pytest’s test_x[case]. In Python and Rust, pass case to the annotation instead: @rqtk.verifies("VA-…", case="neg").

The same works for rows of a JS/TS it.each([...]) table, where the case is any part of the rendered title (case "1+1" for "adds 1+1"), and for GoogleTest value-parameterised tests, where the case is the parameter’s value: tag the TEST_P with // rqtk: verifies VA-… case "7".

Display names

JUnit’s @DisplayName("…") and @ParameterizedTest(name = "…"), and xUnit’s DisplayName = "…", change the name some runners report (Gradle does; Maven Surefire reports the method name). rqtk reads them from the annotations above the test and matches either name. A parameterised JUnit test without a name is reported by Gradle as [1] … with no trace of the method, so give it a name.

Choosing what is scanned

In .rqtk/config.toml:

[scan]
paths = ["."]                        # default; .gitignore is honoured
exclude = ["tests/fixtures/**"]      # gitignore-style globs

Running tests with JUnit output

rqtk reads JUnit XML, which nearly every test runner can write:

--results takes several files, so one per package or module works too.

How results are matched

A result matches a link by test name (or display name), together with the suite for GoogleTest and the case for a case link. When several tests share a name, rqtk narrows by the source file the runner reports (Bun, vitest, GoogleTest) or by what the link’s path says about the result’s class or module. If a linked test still matches several distinct tests, rqtk verify records nothing for it and says which tests it matched; rename one. Disabled and skipped tests leave an activity incomplete.

Recording results

rqtk verify --results junit.xml

rqtk matches each test case to the functions linked to each activity and records one entry per activity in .rqtk/evidence.toml: the outcome, the tests, the commit, and the content hash of the requirement at that moment.

• An activity passes only when every linked test ran and passed; a skipped test leaves it incomplete.
• A run that covers only some tests, such as the Python suite without the Rust one, leaves other activities’ evidence untouched. verify counts the linked tests it found no result for, so a runner naming tests differently doesn’t go unnoticed.
• Evidence names each test as <source file>::<name>, so the same test gets the same entry whichever runner reported it.
• verify exits 1 if a linked test failed, if a linked test matched several tests, or if the results matched no linked test at all (usually the wrong file, or tags rqtk can’t attach).
• verify --check writes nothing and exits 1 if the committed evidence doesn’t match this run. Use it in CI.

Coverage states

rqtk coverage puts every requirement in one state:

rqtk coverage --strict exits 1 unless every requirement is Verified and every need is satisfied. For requirements written ahead of their implementation, --allow planned (and --allow in-progress) accepts those states too.

Suspect

A requirement becomes Suspect when evidence recorded for it no longer settles it. rqtk coverage and rqtk context say why:

The content hash covers what the requirement demands and how it is verified: the statement, its structural links (parents, depends_on, derived_from, refines, satisfies), its parameters, and the verification method, level and phase. Editing the title, keywords, priority or notes changes nothing.

This is deliberate. A reworded requirement may no longer be what the tests check, so rqtk refuses to carry the old verdict over, and rerunning a test that was never updated proves nothing new. rqtk impact <base> lists every activity to re-run after a change. Change control covers rqtk review.

Activities no test can check

Inspections, analyses and demonstrations keep a hand-written status:

[[verification.activities]]
id = "VA-SYS-002-01"
name = "Thermal analysis"
status = "Passed"
executed_at = 2026-03-14
evidence = ["reports/thermal-2026-03.pdf"]

As soon as a test is linked to an activity, its status is ignored, and rqtk lint warns until you remove it (RQ029).