Skip to content

Draft — not approved. A working draft by UASK. It may change without notice.

Status: review · Version 1.0.0

Backups before changes and restore tests

Summary

A backup is only as good as the last successful restore. Keep at least three copies of important data on two kinds of storage with one copy off-site (the 3-2-1 rule), make sure at least one copy cannot be reached or changed by an attacker who controls the live system, and take a fresh, verified backup before any change that could lose data. Prove all of this by restoring, on a schedule, into a separate environment.

Rules

  • R1. Every system that holds data the business cannot recreate MUST have an automated, scheduled backup of that data and of the configuration needed to run it.
  • R2. Backups MUST follow the 3-2-1 rule: at least three copies, on at least two different kinds of storage, with at least one copy off-site.
  • R3. At least one backup copy MUST be offline or immutable: the live system and its normal credentials MUST NOT be able to delete or overwrite it.
  • R4. Before a change that can destroy or rewrite data — a migration, upgrade, bulk import, schema change or manual edit on a live system — a fresh backup MUST be taken and its completion and size checked before the change starts.
  • R5. The pre-change backup MUST be recorded with the change (where it is, when it was taken, how to restore it) so that the rollback plan does not depend on memory.
  • R6. Restores MUST be tested on a schedule, at least quarterly and after any change to the backup set-up, by restoring into a separate environment and checking that the system works with the restored data.
  • R7. Backup frequency and retention MUST be derived from agreed recovery objectives: how much data may be lost (RPO) and how long recovery may take (RTO).
  • R8. Backups SHOULD be encrypted in transit and at rest, and the keys SHOULD be stored separately from the backups so that losing one does not expose or lock the other.
  • R9. Retention SHOULD keep several generations over time (for example daily, weekly, monthly), so that a problem discovered late can still be rolled back to a clean state.
  • R10. Backup jobs SHOULD alert a person when they fail or when the backup size changes unexpectedly.

Guidance

  • What to include. Databases (as consistent dumps or snapshots, not copies of live data files), uploaded files and media, configuration, secrets in a secure store, and the exact application version. Code in Git is not a backup of the data.
  • Off-site and offline. Cloud storage counts as off-site but is always online; make it effectively offline with separate credentials, write-only or object-lock (immutable) access for backup clients, and multi-factor authentication for people.
  • Restore test, not “the job ran”. A green job proves a file was written, not that it can be restored. Measure the restore time and compare it with the RTO.
  • Pre-change checklist. Take the backup; check it finished and its size is plausible; note its location in the change record; know the restore command; then start the change.
  • Retention and law. Personal data in backups is still personal data: retention periods and deletion duties apply to backups too. Agree the retention with whoever is responsible for data protection.
  • Small systems. For a single site on shared hosting, a nightly provider backup plus a weekly copy to separate storage that the site’s own account cannot delete already covers R2 and R3.

Explanation

Most data loss comes from ordinary events — a failed upgrade, a wrong DELETE, a broken disk — and the rest increasingly from ransomware, which deliberately searches for and encrypts or deletes reachable backups. The 3-2-1 rule protects against device and site failure, but on its own it does not require any copy to be offline, which is why the UK National Cyber Security Centre adds an explicit offline rule and a separate recovery rule. CISA likewise recommends scheduled recovery tests to verify backup integrity.

Backups before changes matter because changes are when data is most at risk and when the latest scheduled backup may be hours old. A backup taken minutes before the change, and recorded with it, turns a failed change into a short rollback instead of an incident.

Examples

Good — a change record for an upgrade:

Change: upgrade shop platform 3.0.3 -> 3.0.4
Backup: db dump shop-2026-11-20T09-05.sql.gz (412 MB, gzip -t OK)
files files-2026-11-20T09-05.tar.zst (3.1 GB)
stored on backup host and in object storage (object lock 30 days)
Restore: documented in runbook section 4; last restore test 2026-10-14, 22 min
Rollback trigger: checkout error rate > 1 % for 10 min after release

Bad — “The host makes backups.” Nobody knows where they are, how far back they go, whether the account that runs the site can delete them, or whether a restore has ever worked.

Sources