Skip to content

Draft — not approved. A working draft by UASK. It may change without notice.

Status: draft · Version 0.2.0

Evaluation and conformance

Summary

Version: 0.2

1. Principle

A system is not evaluated by whether it boots, nor by the number of features it contains. Evaluation answers:

  • can the system be reconstructed from its own record;
  • is everything in it accounted for and authenticated;
  • does it hold its security properties in its delivered configuration, not in a hardened variant;
  • does it protect the data of the people who run it;
  • can it be updated, rolled back and eventually retired safely;
  • are its claims bounded by evidence.

A pipeline that is green proves that the pipeline ran. It does not prove any of the above.

Rules

2. Evidence levels

Level Name What it establishes
E0 Declaration someone states the property holds. Not evidence.
E1 Specification review the specification, lock and policy have been read and are consistent with the requirement.
E2 Offline artefact verification the produced artefacts were inspected mechanically without booting: configuration, kernel options, signatures, bill of materials, scan results, licence and trademark checks.
E3 Emulated dynamic verification the system was booted in a virtual environment with firmware and a virtual security module, and the behaviour was observed.
E4 Reference-hardware verification the property was demonstrated on the declared reference hardware, including firmware interaction, storage encryption unlock, suspend, update and rollback.
E5 Field evidence the property is confirmed by operational data the producer is lawfully entitled to hold, support records, incident history or independent audit.

A property may only be claimed at the level at which it has been observed. A kernel option present in a configuration file is E2 evidence about the configuration, not E4 evidence about the running system.

The artefact is not the medium. A property verified on a mounted image is evidence about the image, not about what a recipient will boot. An image may rely on filesystem features the delivery medium does not have — symbolic links resolve inside ISO9660 and do not exist on FAT32 — so a defect can be invisible to every check that looks inside the image and still reach the user. Where the product is delivered on a medium, E2 MUST be established on the medium as written, not only on the image as mounted.

Required minimum levels:

Requirement class Minimum
specification, provenance, licensing, bill of materials E2
isolation, hardening, defaults, credentials E2, with E3 for anything that must be observed at run time
boot integrity, storage encryption, measurement E4
update, rollback, anti-rollback, interruption safety E4
recovery path E4
privacy behaviour of the delivered system E3 minimum, E4 for devices
accessibility of installer, first run and recovery per the applicable HIF profile

3. The verification corpus

Every product maintains a verification corpus. Each entry has the form:

CHECK-ID
requirement SCF-*
property what must be true
method the tool, command or procedure
environment offline | emulated | reference hardware
expected result
evidence level
owner

The baseline corpus below is mandatory for every profile unless a profile records an exception. It is intentionally expressed as properties and methods, not as tool invocations, because tools change faster than obligations.

The baseline is a floor, not a complete mapping of the Constitution. Where a requirement has no entry here, the product MUST add one to its own corpus before claiming that requirement met; a requirement claimed without an entry is an E0 declaration.

3.1 Specification and provenance

Property Method Environment
The lock resolves every input to a version and digest inspect the lock; fail if any input is unpinned offline
Every origin has an authenticated trust anchor under producer control inspect source configuration in the built image and in the build record offline
The build consumed only the lock compare build log inputs against the lock offline
The release is rebuildable rebuild from lock in a clean environment and compare offline
No trust-disabling flags anywhere in the delivered configuration search the image for options that disable signature or certificate verification offline

3.2 Artefacts and attestation

Property Method Environment
Manifest complete and consistent with artefacts validate manifest against produced files and digests offline
Bill of materials present, valid and covering top-level components schema validation plus coverage comparison against the closure offline
Provenance attestation present and verifiable verify the signature and the referenced lock offline
Every distributed artefact is signed and its digest published verify signatures with the published key offline
Verification instructions are published with the artefact inspect the delivery surface offline
Corresponding source is available from the same place fetch it as a recipient would offline

3.3 Boot integrity

Property Method Environment
Boots with firmware verification enabled boot with verification on emulated, then hardware
Boot components carry current revocation generation compare the shipped generation against current revocation policy offline
Boot chain components are signed by keys the chain trusts inspect signatures of loader and kernel offline
Measurement values are as documented, where the profile requires measurement read measurements after boot emulated, then hardware
System volume integrity enforced where the profile requires it verify at run time and attempt a modification emulated
Anti-rollback refuses an older version attempt to install and boot an older release hardware

3.4 Cryptography and credentials

Property Method Environment
Storage encryption uses parameters compliant with the algorithm policy inspect the header of an encrypted volume created by the installer emulated
Key binding to platform state is bound to a meaningful state inspect the enrolment and change the state deliberately hardware
No deprecated primitives in any trust decision inspect signature and protocol configuration offline
No shipped credentials, no empty passwords, no unintended privileged accounts inspect account and key material in the image offline
Automatic login and passwordless administration are off inspect display-manager and privilege-escalation configuration offline
Long-lived secrets are generated at first boot launch two instances and compare emulated

3.5 Isolation and attack surface

Property Method Environment
Mandatory access control enforcing, no unresolved denials on first boot read the enforcement state and the audit record after a clean boot emulated
No unexpected listening services enumerate listening sockets after first boot and scan from outside emulated
Firewall active with default-deny ingress read the effective ruleset emulated
Kernel hardening baseline met check the shipped kernel configuration and boot parameters against the baseline offline
Unsigned kernel code cannot load attempt to load an unsigned module emulated
Network-facing services are confined inspect the confinement of each such service offline
Installed surface matches the profile compare the installed set against the declared selection offline

3.6 Data and privacy

Property Method Environment
No network contact after first boot other than declared endpoints observe traffic on a clean instance for a defined period emulated
Telemetry off until an explicit choice inspect defaults and the first-run surface offline plus emulated
No stable device identifier transmitted without necessity inspect observed traffic emulated
Erasure and reset return the system to its as-delivered state perform the documented procedure and inspect the result hardware
Logging can be disabled and its consequences are documented perform the procedure emulated

3.7 Updates and vulnerabilities

Property Method Environment
Security updates apply automatically by default inspect configuration, then observe on a running instance emulated
Updates are verified before application attempt an update signed by an untrusted key emulated
An interrupted update leaves a bootable system interrupt at several points hardware
Rollback returns to the previous known-good state perform a rollback after a deliberately broken update hardware
Offline update path works with the same guarantees perform an offline update emulated
No known exploited vulnerability in delivered components scan the bill of materials against current vulnerability data offline
Exploitability statements exist for accepted findings inspect the published statements offline
Property Method Environment
Every component has an identified licence and required notices are delivered inspect the licence inventory and the image offline
No upstream mark subject to removal remains in the image automated search of filesystem, boot media, artwork, metadata and strings offline
Support period and end-of-life date are declared and visible inspect the product record and the delivery surface offline
Baseline conformance measured where the profile requires it run the conformance scan against the named baseline revision offline

3.9 Human surfaces

Verified per the applicable HIF profile, with at least: installation without data loss on a machine with existing data; first run completed with a keyboard only; first run completed with a screen reader; every destructive action correctly describing its scope; recovery entered and exited without a network.

4. Severity

  • S0 Blocker — the system cannot be trusted or used: it does not boot, an update bricks it, data is lost, a secret is shared across installations, an artefact is unsigned or unaccounted for, or an active exploitation report cannot be handled.
  • S1 Critical — a security or privacy property claimed by the product does not hold in the delivered configuration.
  • S2 Major — a required property holds only after manual configuration, or rollback and recovery work only in some cases.
  • S3 Moderate — evidence is weaker than the required level, or documentation contradicts behaviour.
  • S4 Minor — a local inconsistency with no effect on the properties claimed.
  • Observation — a hypothesis, not a confirmed defect.

Severity is determined by consequence for the people and organisations running the system, not by how visible the problem is.

5. Conformance levels

SCF-Declared

  • profiles chosen;
  • the traceability matrix exists;
  • known exceptions are registered.

This marks the beginning of a process. It is not a quality signal and MUST NOT be presented as one.

SCF-Conformant

  • every applicable MUST is met;
  • every SHOULD exception is justified and registered;
  • the whole baseline verification corpus has been executed at its required evidence level, including reference-hardware checks;
  • the release is rebuildable from its lock;
  • the bill of materials, provenance and signatures are complete;
  • the support period, update contract and disclosure policy are published.

SCF-Validated

  • SCF-Conformant, plus:
  • the release has been rebuilt independently and compared;
  • high-risk paths — boot integrity, update, rollback, recovery, erasure — have been demonstrated on reference hardware;
  • the vulnerability process has been exercised, not only described;
  • operational monitoring for the support period is defined and staffed.

A conformance statement MUST name the SCF version, the profiles, the release, the architectures and the reference hardware to which it applies. A statement without those bounds is void.

6. Traceability matrix

obligation (law, threat model, customer requirement)
→ SCF requirement
→ product requirement
→ specification field, component or policy
→ implementation
→ automated check
→ manual or hardware scenario
→ current evidence and its level
→ exceptions

A requirement with no path to evidence is incomplete and MUST NOT be counted as met.

7. Release gate

Before a release is signed:

  1. the scope of change is defined and the affected profiles listed;
  2. the lock is complete, and every input is authenticated and pinned;
  3. the build ran in a compliant boundary and its log is retained;
  4. all offline checks pass, including licence, trademark and vulnerability checks;
  5. the bill of materials, manifest and provenance are complete and valid;
  6. the boot chain generation is current against revocation policy;
  7. emulated dynamic verification passes;
  8. reference-hardware verification passes for boot, update, rollback, recovery and storage encryption;
  9. accessibility checks required by the applicable HIF profiles pass;
  10. the support period, channel and end-of-life date are set and published;
  11. every open exception has an owner and an unexpired review date;
  12. a named person approves, and the approval is recorded with the evidence.

The rollback plan and the criteria for withdrawing the release are part of the gate, not a later decision.

8. Exceptions

ID
SCF version and requirement
product and profile
reason
affected users, devices and data
risk and likelihood
compensating control
owner
expiry or review date
evidence

An expired exception blocks a new conformance statement until it is reviewed. An exception MUST NOT be used to carry a permanent architectural decision; if a requirement is wrong, propose a change to the standard.

9. After release

The support period is part of the product, so evaluation does not stop at release.

  • monitor upstream for changes to the inputs of every supported release, and be able to state how far behind each release is;
  • monitor vulnerability data against the bill of materials of every supported release;
  • monitor revocation and signing-authority policy;
  • record time from report to fix to published advisory, and review it;
  • re-run the verification corpus for every update that touches boot, kernel, cryptography, update logic or defaults;
  • record field failures against the corpus, and add a check for every failure the corpus did not catch.

10. Changing the standard

If a product cannot meet SCF:

  1. check whether the problem is local architecture rather than the requirement;
  2. check the alternatives, including a different platform family;
  3. register an exception with evidence;
  4. gather evidence over at least one release cycle;
  5. only then propose a change to the standard.

A widespread industry practice is not evidence that a requirement is wrong. The absence of an upstream facility is a constraint to be recorded, not a justification for a weaker claim.