Skip to content

Draft — not approved. A working draft by UASK. It may change without notice.

Quality attributes, quality goals and scenarios

Status
Review
Version
0.1.2
Authority
UASK standard
Applies to
Core
Markdown source

Summary

A quality attribute is how well a system does what it does: how fast, how safe, how easy to change, how well it recovers. This item gives a project one vocabulary for them, the nine quality characteristics of ISO/IEC 25010:2023, and one way to make them testable: three to five quality goals, each backed by a quality scenario with a source, a stimulus, an environment, a response and a measure. It adds an operability block for what ISO leaves to the people who run the system: availability, recovery objectives, restore tests, rollback, monitoring and the end of life of the runtime. The goals and scenarios fill sections 1 and 10 of the Blueprint (arch.architecture-description). The operability block is stated as requirements in the PRD, in its section Operability; section 7 of the Blueprint says how each value is met.

Rules

Vocabulary and goals

  • R1. A quality requirement MUST name the quality characteristic it belongs to, using the nine names of ISO/IEC 25010:2023: functional suitability, performance efficiency, compatibility, interaction capability, reliability, security, maintainability, flexibility and safety. A project MAY add a sub-characteristic in its own words, and MUST NOT invent a tenth top-level name for a quality that one of the nine already covers.
  • R2. A project MUST choose three to five top quality goals. Each goal MUST name its characteristic, say in one sentence what the goal means for this system, name the stakeholder who cares (arch.architecture-description#R2), and list the requirement ids it serves. Goals MUST be ranked, so that a reader knows which one wins when two collide.
  • R3. Where two goals pull against each other (speed against security, flexibility against simplicity, cost against availability), the Blueprint MUST say which one wins and why, and the choice MUST be recorded as an ADR (arch.adr) when it is hard to reverse.

Scenarios

  • R4. Each top quality goal MUST have at least one quality scenario. A scenario MUST state its source (who or what causes it), its stimulus (what happens), its environment (the state the system is in), its response (what the system does) and its measure (how well, in a number or a checkable yes or no). It SHOULD also name the part of the system that is stimulated.
  • R5. A scenario’s measure MUST say how it is checked: a test, a monitor, an inspection or a review. A requirement written only in words such as fast, secure, user-friendly or highly available MUST NOT stand without a scenario that gives it a measure.
  • R6. A scenario that states a non-functional requirement MUST carry that requirement’s id. Scenarios SHOULD be ranked by importance and by difficulty (high, medium, low), as arch.architecture-review#R5 does for a review.
  • R7. A cross-cutting concern with a measurable outcome (a security objective, a privacy limit, an accessibility level, observability) MUST be written as a scenario of the same form. Its concept text stays in section 8 of the Blueprint, tied to the same requirement ids (arch.architecture-description#R8).

The operability block

  • R8. From the standard tier, the PRD MUST carry an operability block, in its section Operability, that states, each as a number, a date, a named owner or the word “none” with its reason: the availability target and the hours it applies to; the recovery point and the recovery time objectives; how often the restore is tested and when it was last proved; how a release is rolled back and how long that takes; what is monitored and who is told; who runs the system; and the end-of-support date of each runtime and platform it depends on, with the plan for it. Section 7 of the Blueprint MUST say how each value is met, and where one is not met yet, say so.
  • R9. At the lightweight tier, a system that holds data the business cannot recreate MUST still state its recovery point, its recovery time and its restore test, because ops.backups#R7 and ops.backups#R6 apply at every tier. Everything else in the block MAY be omitted.
  • R10. Each operability value MUST be derived from what the stakeholders can bear, and MUST NOT be copied from what the hosting offers by default. Where a value is a guess, the block MUST say so and name the check that would settle it.

Tiers and change

  • R11. At the high-assurance tier, each top scenario MUST be traced to the check that proves it, in a test, a monitor or a rehearsal (mgmt.governance-tiers#R8). At the standard tier the trace MAY be left to the test strategy.
  • R12. A change that moves a goal or a measure MUST change section 10 of the Blueprint in the same pull request. A release that cannot meet a top scenario MUST say so in its release spec (process.release-spec) and MUST NOT ship as if it did.

Guidance

The nine characteristics

The Hub takes the nine names and none of the standard’s wording. The questions below are in the Hub’s own words.

Characteristic A stakeholder asks
Functional suitability Does it do the jobs it was built for, completely and correctly?
Performance efficiency Is it fast enough and light enough on time, memory and money for the load it will meet?
Compatibility Does it live alongside other systems, and exchange data with them, without harm?
Interaction capability Can the people who use it learn it, use it, and recover from their mistakes? That includes people with disabilities.
Reliability Does it stay up, and does it come back after a fault, with the data intact?
Security Does it keep out what it should and keep in what it should, and can we show who did what?
Maintainability How cheap and safe is it to change, test and repair?
Flexibility Can it move to a new platform, be installed elsewhere, or grow, without a rewrite?
Safety Can its failure hurt people or the environment, and is the harm bounded?

Most projects need only three or four of them as goals. Choose the ones whose failure the owner would notice first.

From a concern to a scenario

Work from the stakeholder to the measure:

  1. Concern. The owner says, “I lose sales if checkout is down in opening hours.”
  2. Goal. Reliability: checkout keeps working while the shop is open. Stakeholder: the owner. Ranked second.
  3. Scenario. One concrete event with all its parts, below.
  4. Check. How the measure is verified, and by whom, and how often.

A scenario has the six parts that the Quality Attribute Workshop method names. This item asks for five of them (R4) and recommends the sixth, the stimulated part:

Part What goes in it A poor entry and a good one
Source who or what produces the stimulus “a user” and “a shopper on a phone”
Stimulus the event that the system must react to “load” and “400 shoppers arrive within ten minutes”
Artifact the part of the system that is stimulated “the system” and “the checkout and the database”
Environment the state the system is in “normally” and “a Saturday in the sale, all nodes up”
Response what the system does “works” and “completes the order and sends the receipt”
Measure how well, and how it is checked “quickly” and “95 % of orders complete in under 4 s, from the weekly load test”

A scenario with a clear stimulus, a response and a measure is already useful. The environment settles whether a response is hard or easy to reach. The source and the stimulated part make the scenario traceable to the architecture.

Writing a measure

  • Put a number with its condition: a percentile, a count, a duration, a date. “95 % under 2 s on a mid-range phone on 4G” can be tested; “fast” cannot.
  • Say what is counted and where it is read: a synthetic check, a log, a monitor, a test run, a review checklist.
  • Prefer a measure the project already collects over one that needs new tooling. Add the tooling to the release spec if it is needed.
  • If there is no baseline, write the number as a first guess and say “to be measured in Mn”. R10 asks for that.

The operability block

The block is written as requirements in the PRD, in its section Operability, and section 7 of the Blueprint, next to the deployment, says how each is met. Each point has a value, and for a live system each value is something a person could read off a monitor or a calendar.

Point Say Example value
Availability the target, and the hours it applies to “99.5 % a month, 08:00 to 22:00 UK time”
Recovery point (RPO) how much data the owner can lose “at most 1 hour of orders”
Recovery time (RTO) how long the owner can wait for it to be back “within 4 hours”
Restore test how often, by whom, and the date of the last one “quarterly; last proved 2026-09-12, by the lead”
Rollback how a release is undone and how long it takes “restore the release folder and the database dump, 20 minutes”
Monitoring what is watched and who is told “uptime check every minute; the owner is mailed after three failures”
Who runs it the person or team, and the cover “the lead; the owner’s phone for a stop”
End of life the end-of-support date of each runtime and platform, and the plan “PHP 8.2 security fixes end in December 2026; move to 8.4 in the March release”

The availability target is a small service level: a measure of what users see, a goal for it and a window over which it is read. The Site Reliability Engineering book calls them an indicator and an objective. ops.service-levels covers them in full; the block states the target as one line.

Reading the cloud pillars in this vocabulary

The Well-Architected frameworks of the cloud providers organise their review questions in pillars. A project on a cloud platform can use them as review questions, and then maps them to this item’s vocabulary, so that the Blueprint speaks with one voice. The pillar names below are from the Google Cloud framework; AWS and Azure use nearly the same set.

Pillar Where it goes here
Operational excellence the operability block, and maintainability
Security, privacy and compliance security, and the legal constraints in section 2 of the Blueprint
Reliability reliability, and the recovery points of the operability block
Performance optimization performance efficiency
Cost optimization not a quality characteristic: the appetite and a constraint in section 2, or a goal ranked below the rest
Sustainability not a quality characteristic: a constraint or a goal in the project’s own words

Five characteristics (functional suitability, compatibility, interaction capability, flexibility and safety) have no pillar. A cloud review does not ask about them, so the Blueprint must.

Tailoring by tier

Tier Goals and scenarios Operability block
Lightweight three goals, one line each; one scenario per goal may sit in the goal table RPO, RTO and the restore line where data cannot be recreated (R9)
Standard three to five goals; a scenario for each, in section 10; ranked the full block (R8)
High-assurance as standard, with the check that proves each top scenario (R11) the full block, with a rehearsed rollback and a date for each restore test

What belongs elsewhere

  • What the system does (functional requirements) goes in the PRD and the requirements specification (req.requirements-specification). A quality scenario says how well, never what.
  • Acceptance is the criteria of req.acceptance-criteria. A scenario’s measure can be one of them.
  • A review grades how the architecture meets the scenarios; that is arch.architecture-review.

Explanation

Why scenarios. A quality word such as “reliable” means something different to each stakeholder, and no test can fail it. The Quality Attribute Workshop method of the Software Engineering Institute makes stakeholders write the event, the response and a measure. A response measure forces the architect to build for a particular, checkable criterion, as the report’s own example shows: “modify the system” is always possible given time and money, but “in two person-weeks” is a constraint on the design. The environment matters because the same response may be easy in a quiet hour and hard at the peak.

Why the ISO vocabulary. The nine characteristics are a common list of names that stakeholders, reviewers and tools already know. The Hub uses the names and the idea of sub-characteristics, and writes its own text, because the standard is paid and closed to copying. The 2023 edition renamed two characteristics (usability is now interaction capability, and portability is now flexibility) and added safety; projects that used the older names find them again here.

Why an operability block. Reviews of earlier projects found that the worst gaps were not in what the system did but in how it was run: no recovery objectives, a backup that had never been restored, no rollback, a runtime past its end of life. None of them is a separate quality characteristic, so none had a place in a Blueprint. The block gives them one. The word operability is used here for running and recovering the system, and not for how easy it is for a user to operate.

Why pillars are mapped and not adopted. The cloud frameworks are useful review checklists, but they leave out five of the nine characteristics, and two pillars (cost and sustainability) are goals or constraints and not qualities of the product. Mapping keeps one list in the Blueprint and lets a cloud review feed it.

Why arc42 by reference. The Blueprint’s section 10 follows arc42’s order, and arc42 also gives a way to write quality scenarios. The Hub cites its structure by reference (arch.architecture-description#R13) and writes its own text.

Examples

Good. A synthetic example: a small UK web shop that sells refurbished bicycles, at the standard tier. Four goals, in rank order.

# Characteristic Goal Stakeholder Serves
1 Security Customers’ card details never reach the shop’s own servers, and personal data stays protected shoppers, owner NFR-1, NFR-2
2 Reliability Checkout keeps working while the shop is open owner NFR-3
3 Performance efficiency Product pages feel instant on a phone shoppers NFR-4
4 Maintainability The part-time developer can ship a fix in a day owner, developer NFR-5

Goal 1 wins over goal 3: a third-party payment page is slower than a native form, and the owner accepts it. That is ADR 0004, marked owner-level.

Two of the scenarios:

Part NFR-3, Reliability NFR-4, Performance efficiency
Source shoppers a shopper on a mid-range phone
Stimulus 400 shoppers reach checkout within ten minutes opens a product page for the first time
Artifact checkout and database the storefront and its image delivery
Environment a Saturday in the sale, all components up 4G, one bar of signal, no cache
Response every order is taken and its receipt is mailed the page shows the product and price
Measure at least 99 % of orders complete in under 4 s; read from the weekly load test on staging largest content on screen in under 2.5 s at the 95th percentile; read from the weekly synthetic check
Rank high importance, medium difficulty high importance, low difficulty

The operability block, in the PRD:

Point Value
Availability 99.5 % a month, 08:00 to 22:00 UK time
RPO and RTO at most 1 hour of orders lost; back within 4 hours
Restore test quarterly; last proved 2026-09-12 by the lead
Rollback restore the previous release folder and the database dump, about 20 minutes
Monitoring uptime check every minute; the owner is mailed after three failures
Who runs it the lead; the owner’s phone for a stop
End of life PHP 8.2 security fixes end in December 2026; move to 8.4 in the March release; the guess for the traffic limit is marked “to be measured in M2”

Bad. A Blueprint that says “The shop must be fast, secure and always available”, with no stakeholder, no scenario and no number. Nobody can fail it, so nobody can build to it. It misses R2, R4 and R5, and no operability block says how long a restore takes or when the runtime ends its support (R8).

Sources