Deployment Plan and Operations Runbook

Proof required

Submit: (1) an architecture diagram showing all components with their runtime environments and inter-component dependencies labelled (arrows with protocol and direction — e.g. 'HTTPS/443', 'PostgreSQL/5432'); (2) a component inventory table (component name, runtime environment, config sources, dependencies, health signal); (3) a brief scope statement (100–150 words) confirming whether this is a real system you operate or a realistic reference architecture, and why the system has been bounded where it has.

What gets checked

• Architecture diagram labels every connection with the protocol and port — 'arrow from app to DB' without protocol/port is not an engineering diagram
• Component inventory table is complete for all components in the diagram — every box in the diagram has a row in the table; missing rows indicate components not understood well enough to document
• Scope statement addresses the system boundary explicitly — what is inside the runbook scope and what is treated as external (e.g. 'the CDN is treated as external; only the origin servers are in scope')

Common mistakes

• Selecting a system so simple it has only one component — a single static website with no backend cannot demonstrate the deployment coordination and dependency management that a runbook exists to address
• Producing a conceptual diagram without operational details — a diagram that shows 'Frontend', 'Backend', 'Database' as boxes without runtime environments, protocols, or ports is a concept diagram, not a deployment architecture diagram

Resources

What a verifier looks for

• Engineering Design Triad: M1 produces a design artifact (architecture diagram + component inventory) — this is the foundation that M2 (runbook procedures) and M3 (incident simulation + report) build on.
• Every connection in the diagram must be labelled with protocol and port — check that arrows are not bare lines.
• Component inventory must match the diagram exactly — every box in the diagram has a row; reviewer should cross-check both artefacts.
• Scope statement must address what is outside the boundary and why — this demonstrates that the author understands the system's operational context, not just its internal structure.
• Reviewer should be a software engineer or SRE with deployment/operations experience — the operational correctness of the inventory cannot be evaluated without this background.

Proof required

Submit the runbook document containing all three sections. Each step must include: the exact command (or UI action if no CLI is available), the expected output or confirmation signal, and an explicit statement of what to do if the expected output is not seen (escalate, retry, or trigger rollback). If any step requires credentials or secrets, the runbook must reference them by name only (e.g. '`$DB_PASSWORD` from the `prod/database` Vault path') — never inline the actual value.

What gets checked

• Every step has an expected output — 'Run `kubectl rollout status deployment/app`' without stating the expected output ('Waiting for deployment... successfully rolled out') is an incomplete step; an on-call engineer cannot verify success without the expected signal
• Rollback procedure starts from the failed deployment state, not from a healthy state — a rollback that assumes you are starting clean has not accounted for the partial-deployment scenario it is intended for
• No credentials or secrets are inlined — only references to their storage location (Vault path, AWS Secrets Manager ARN, environment variable name) are permitted

Common mistakes

• Writing the deployment procedure without actually executing it against a test environment and verifying the expected outputs — a runbook written from memory without validation will have wrong expected outputs, which is worse than no runbook because it misleads the on-call engineer
• Writing rollback as 'revert the deployment' without specifying how — the rollback section must name the exact mechanism (e.g. `kubectl rollout undo`, `git revert` + re-deploy, feature flag toggle + traffic shift) with the specific commands

Resources

What a verifier looks for

• Engineering Design Triad: M2 produces an analysis artifact (the runbook as an operational analysis of the deployment process) — combined with M1's design artifact and M3's documentation artifact, the triad is complete.
• Every step must have an expected output — reviewer should check a random sample of five steps to confirm this; a step without expected output fails the standard.
• Rollback must start from a failed/partial deployment state — check that the rollback section explicitly states its starting assumption.
• Credentials must not be inlined — check for any string resembling a password, API key, or token; only references to secret stores are permitted.
• Reviewer must have software engineering or SRE/DevOps experience — the procedural correctness of the deploy and rollback steps requires operational domain knowledge.

Proof required

Submit: (1) the revised runbook (with tracked changes visible — use GitHub diff, strikethrough in Markdown, or a 'Changes' appendix); (2) a runbook improvement report (300–500 words) covering: the two failure scenarios simulated; at least three specific gaps or ambiguities found; the exact changes made to address each gap; and a statement of what remains out of scope and why.

What gets checked

• At least three specific gaps are documented with the exact step number and the nature of the ambiguity — 'Step 4 did not specify what to do if the database migration fails mid-run' is a valid gap; 'the runbook could be clearer' is not
• Changes to the runbook are traceable — the improvement report's gap descriptions must map to visible changes in the revised runbook; a report that lists three gaps but shows no corresponding runbook changes fails the traceability check
• Out-of-scope statement is explicit and reasoned — 'multi-region failover is out of scope because the system runs in a single region' is an acceptable rationale; omitting an out-of-scope statement implies the runbook covers all scenarios, which no runbook does

Common mistakes

• Self-facilitating the incident simulation without an external participant — the value of the simulation is that the facilitator knows things the runbook author does not; self-simulation cannot produce the 'follow-up condition' that exposes a gap
• Producing an improvement report that lists gaps without showing the corresponding runbook changes — a gap finding without a fix is an observation, not an improvement; both parts are required

Resources

What a verifier looks for

• Engineering Design Triad check: M1–M3 together produce a design artifact (architecture diagram + component inventory), an analysis artifact (runbook procedures — an operational analysis of the deployment process), and a documentation artifact (runbook improvement report with traced gap-to-fix mapping) — confirm all three are present.
• Improvement report must map each gap to a specific step number in the original runbook — check that gap descriptions are locatable in the original document.
• Runbook changes must be visible and traceable — use GitHub diff link or explicit 'before/after' in the report; invisible changes cannot be verified.
• Simulation must have had an external facilitator — the submission must name the facilitator by role (e.g. 'senior SRE at my team', 'peer engineer familiar with the system', 'technical colleague who had not read the runbook before the session').
• Reviewer must have software engineering or SRE/DevOps experience — evaluating the realism of the failure scenarios and the completeness of the gap analysis requires operational domain knowledge.