Test Management

Test management in Almirah follows the same principle as everything else in the framework: test cases are Markdown files placed under source control, right next to the requirements they verify. Two folders carry the whole concept — tests/protocols/ for what should be tested, and tests/runs/ for what was tested, when, by whom, and with which result.

Protocols: test cases, not yet executed

The tests/protocols/ folder holds the test protocols — one folder per test case:

my_project/tests/protocols/ ├── tp-001/tp-001.md ├── tp-002/tp-002.md └── tq-001/tq-001.md

A protocol is a Markdown document with a summary table and a test procedure — a controlled table whose steps link up to the requirements they verify with the > notation:

| Test Step # | Test Step Description | Result | Req. Id | |---|---|---|---| | 1 | Some preparation step | | | | 2 | Verification step for requirement "REQ-001" | | >[REQ-001] |

By design, protocols are not executed: the Result column and the summary (software version, tester name, date) stay empty. They answer one question — does a test exist for this requirement? From these links Almirah builds the coverage matrix, so an uncovered requirement is visible before anyone runs anything.

Runs: execution sessions with results

Each numbered folder under tests/runs/ represents one test run session. To execute tests, copy the protocols you are running into a new run folder and fill them in — the pass/fail result of every step, plus the software version, tester name, and date in the summary table:

my_project/tests/runs/ ├── 001/ # first session: tp-001 and tp-002 executed │ ├── tp-001/tp-001.md │ └── tp-002/tp-002.md └── 010/ # another session: tq-001 executed └── tq-001/tq-001.md
$ cp -r my_project/tests/protocols/tp-001 my_project/tests/runs/002/ # fill in the Result column and the test summary, then: $ almirah please my_project --run 002

The --run option builds the coverage matrix against that exact session, with its pass/fail results — instead of the non-executed protocols.

Because every run folder is an ordinary set of files in source control, the history of test sessions accumulates naturally: run 001 stays untouched when run 002 happens, each is traceable to the exact software version it tested, and a reviewer or auditor can open any past session as easily as the current one. No test-management tool, no export — the audit trail is the repository.

Where to go next

  • Traceability and Coverage — how protocols and runs feed the coverage matrices
  • Installation — scaffold an example project with protocols and runs already in place, via almirah create