Test strategy
Used in the playbook Quality and release.
Download test-strategy.md — the source, as in the repository.
How to use
The template's own instructions:
UASK Engineering Hub template · test-strategy 0.1.0 · playbook quality.quality-release (R3, R5, R6) · CC BY 4.0 Copy to the project as TEST-STRATEGY.md. One page is enough for a small project: fill in the tables, delete rows the project does not use, keep it true. It is the single place that says which checks exist, where they run and what they block; CI configuration implements it. Delete these comments in the copy.
Fields
Front-matter fields and sections of the document the template creates. A section marked Optional in the template may be deleted from a copy; every other field and section stays (tools/check_artifacts.py checks this for the kinds it knows).
| Field or section | Where | Required |
|---|---|---|
| 1. Quality goals | section | required |
| 2. Test levels | section | required |
| 3. Quality gates | section | required |
| 4. Pipeline | section | required |
| 5. Environments | section | required |
| 6. Critical user paths | section | required |
| 7. Test data and secrets | section | required |
| 8. Flaky and disabled checks | section | required |
| 9. Measures | section | optional |
Example
A filled-in copy for an invented product, templates/examples/test-strategy.md, in the Hub repository.It shows the level of detail to aim for, not text to reuse.
# Test strategy and quality gates — Fieldnotes
Version: 0.1.0 · Owner: Sam Roe · Last changed: 2026-03-01
## 1. Quality goals
| Characteristic | Why it matters here | How we check it ||---|---|---|| Reliability | a lost note is the worst failure | the kill test saves 1 000 notes and kills the app each time || Portability | notes must leave with the user | export test reads every note without the app |
## 2. Test levels
| Level | Covers | Tools | Runs | Gate ||---|---|---|---|---|| Component (unit) | parsing, file naming | pytest | local, every pull request | G1 || Component integration | store and index together | pytest | every pull request | G1 || System (end-to-end) | the app on an emulator | Playwright, Android emulator | release candidate | G1 / G2 || System integration | a real Git host in a sandbox | pytest | release candidate on staging | G2 || Acceptance | testers confirm the outcome | checklist | staging before approval | G2 |
Static checks (lint, type checks, schema validation): ruff, mypy — run in G1.Non-functional checks (performance, accessibility, security scans): benchmark and axe — run in G2.Regression: every gate re-runs the whole existing suite of its levels; every escaped defect adds a test.
## 3. Quality gates
| Gate | When | Checks | Command | Blocks ||---|---|---|---|---|| G1 Change | every pull request, locally and in CI | static, component, integration, build | `bash tests/run.sh` | merge to main || G2 Release candidate | before release approval | full suite, staging checks, dependency audit | the release checklist | release approval || G3 Post-release | right after deploy | critical paths (section 6), crash reports | the smoke checklist | closing the release; triggers rollback |
How merging is blocked: a required status check on the main branch.
## 4. Pipeline
1. Push or pull request → CI runs G1 on a hosted runner.2. Merge to main → CI runs G1 again on main.3. Tag `vX.Y.Z-rc.N` → CI builds artefacts, records checksums; deploy to the test channel by hand.4. G2 on the test channel; release checklist; approval by the owner.5. Tag `vX.Y.Z` → CI builds the release; promotion to production by hand in the planned window.6. G3 post-release verification; record kept in `releases/`.
## 5. Environments
| Environment | Purpose | Data | How it is refreshed ||---|---|---|---|| Local | development, G1 | fixtures | — || CI | G1 | fixtures | each run || Staging | G2, acceptance | fixtures | before each release || Production | users | real, on the users' devices | — |
## 6. Critical user paths
| Path | Check | Automated? ||---|---|---|| write a note offline, kill the app, reopen | the note is intact | yes || sync two devices | both have both notes | yes |
## 7. Test data and secrets
Fixtures live in `tests/fixtures/` and hold no real notes. The sandbox Git token is a CI secret; it never appears in logs.
## 8. Flaky and disabled checks
A failing check is fixed or quarantined in its own reviewed change with an issue (playbook R7).
| Check | Issue | Disabled since | Returns by ||---|---|---|---|| — | — | — | — |
## 9. Measures
Lead time and change failure rate are read from the release records each quarter.Completion checks
A copy is complete when:
- every required field and section above is present, in the same order, and no bracketed placeholder is left;
- the template's comments are deleted from the copy;
- the exit criteria of its stage of the playbook Quality and release are met;
- the owner has approved it, where the playbook asks for approval. Only the owner sets
approved.
Source
templates/test-strategy.md, as in the repository. Placeholders are in square brackets; comments carry the instructions.
<!-- UASK Engineering Hub template · test-strategy 0.1.0 · playbook quality.quality-release (R3, R5, R6) · CC BY 4.0 Copy to the project as TEST-STRATEGY.md. One page is enough for a small project: fill in the tables, delete rows the project does not use, keep it true. It is the single place that says which checks exist, where they run and what they block; CI configuration implements it. Delete these comments in the copy.-->
# Test strategy and quality gates — [project]
Version: 0.1.0 · Owner: [name] · Last changed: [YYYY-MM-DD]
## 1. Quality goals
<!-- Which quality characteristics matter most for this product and why (ISO/IEC 25010 names them: functional suitability, performance efficiency, compatibility, interaction capability (usability), reliability, security, maintainability, flexibility (portability), safety). Pick three to five. -->
| Characteristic | Why it matters here | How we check it ||---|---|---|| [for example: reliability] | [checkout must not fail] | [system tests of checkout; error-rate monitoring] |
## 2. Test levels
<!-- The five ISTQB levels. Most tests belong to the first two rows. "Runs" = where and when. -->
| Level | Covers | Tools | Runs | Gate ||---|---|---|---|---|| Component (unit) | [functions, classes, modules in isolation] | [framework] | [local, every pull request] | G1 || Component integration | [database, file system, framework hooks, modules together] | [tools] | [every pull request] | G1 || System (end-to-end) | [whole product through its real interface: browser, CLI, booted image] | [tools] | [pull request or release candidate] | G1 / G2 || System integration | [external services: payment, delivery, third-party APIs] | [tools, sandboxes] | [release candidate on staging] | G2 || Acceptance | [owner or users confirm the outcome] | [checklist, demo] | [staging before approval] | G2 |
Static checks (lint, type checks, schema validation): [tools] — run in G1.Non-functional checks (performance, accessibility, security scans): [tools] — run in [G1 or G2].Regression: every gate re-runs the whole existing suite of its levels; every escaped defect adds a test.
## 3. Quality gates
| Gate | When | Checks | Command | Blocks ||---|---|---|---|---|| G1 Change | every pull request, locally and in CI | [static, component, integration, fast system tests, build] | `<bash tests/run.sh>` | merge to main || G2 Release candidate | before release approval | [full suite, staging checks, dependency audit] | `<command or checklist>` | release approval || G3 Post-release | right after deploy | critical paths (section 6), error logs, monitoring | `<smoke command or checklist>` | closing the release; triggers rollback |
How merging is blocked: [required status check on the main branch / written rule plus pre-push hook].
## 4. Pipeline
<!-- From push to production in a few lines: what runs where, what is manual. -->
1. Push or pull request → CI runs G1 on [runner].2. Merge to main → CI runs G1 again on main.3. Tag `vX.Y.Z-rc.N` → CI builds artefacts, records checksums; deploy to staging [manual or automatic].4. G2 on staging; release checklist; approval by [name or role].5. Tag `vX.Y.Z` → CI builds the release; deploy to production [manual or automatic] in the planned window.6. G3 post-release verification; record kept in [releases/ or release page].
## 5. Environments
| Environment | Purpose | Data | How it is refreshed ||---|---|---|---|| Local | development, G1 | [fixtures] | — || CI | G1 | [fixtures] | each run || Staging | G2, acceptance | [anonymised copy of production / fixtures] | [before each release] || Production | users | real | — |
## 6. Critical user paths
<!-- What must work after every release. Checked in G3 and on staging in G2. Keep it short and concrete. -->
| Path | Check | Automated? ||---|---|---|| [for example: search → product → cart → checkout] | [order placed with test payment] | [yes / manual] |
## 7. Test data and secrets
<!-- Where fixtures live, how personal data is kept out of test environments, where CI secrets are stored. -->
## 8. Flaky and disabled checks
A failing check is fixed or quarantined in its own reviewed change with an issue (playbook R7).
| Check | Issue | Disabled since | Returns by ||---|---|---|---|| — | — | — | — |
## 9. Measures
<!-- Optional: DORA measures taken from release records — change lead time, deployment frequency, change fail rate, failed deployment recovery time, deployment rework rate. -->