Build a Distributed System

Proof required

Submit: (a) the architecture document (minimum 1,500 words) naming the real problem, the consistency model chosen with rationale, the fault tolerance approach with specific failure mode analysis, the data partitioning strategy with explanation of rebalancing, and a component diagram, and (b) a critical path complexity analysis: the operations with the highest latency impact, their Big-O time and space complexity, and a target SLA (p99 latency, throughput in requests/second) with justification. The analysis must reference specific CAP theorem trade-offs made.

What gets checked

• Architecture makes specific trade-off decisions — 'we chose eventual consistency because our use case tolerates stale reads for up to 10 seconds and we need partition tolerance for geographic distribution' not 'we use eventual consistency'
• Critical path analysis includes specific latency estimates with breakdown by component — not 'fast enough'
• CAP theorem trade-offs are explicitly stated — which is being relaxed and in what failure scenario

Common mistakes

• Architecture describes what the system does without explaining why specific design choices were made — the design is defensible only when trade-offs are explicitly articulated
• Consistency model stated without explaining user-visible behaviour — 'we use eventual consistency' without explaining what a user experiences during partition is incomplete

Resources

What a verifier looks for

• You are reviewing a distributed system architecture. The key test is specificity of trade-off reasoning: does the submitter explain WHY each major design decision was made, not just WHAT the decision was?
• Probe the consistency model: 'Walk me through what happens to a user's data if a network partition occurs between these two services while a write is in flight.' The answer reveals whether the submitter understands the actual behaviour of their chosen consistency model.
• Check the CAP analysis: does it identify which specific failure scenario triggers the CAP trade-off? 'We sacrifice consistency during network partitions' is correct; 'we use eventual consistency for performance' is a common misconception to challenge.

You'll sign in first, then come straight back here.

Proof required

Submit: (a) a link to the public code repository with all service code, infrastructure-as-code files, and a README with setup instructions, (b) a deployment screenshot or recording showing three services running simultaneously, a request being processed end-to-end, and the observability dashboard showing the trace of that request, and (c) a demonstration video (minimum 3 minutes) showing: starting the system, sending test requests, triggering a simulated node failure, and showing how the system handles the failure according to the stated fault tolerance approach.

What gets checked

• Code repository contains infrastructure-as-code — the deployment must be reproducible, not manual
• Observability shows distributed traces — not just container logs
• Failure demonstration shows the specific failure mode from milestone 1 and the actual system response

Common mistakes

• Deployment only runs locally with docker-compose on a single machine — a distributed system must run across multiple compute instances
• Failure mode demonstration shows the system crashing without recovery — fault tolerance means graceful degradation and recovery

Resources

What a verifier looks for

• Clone the repository and run the setup instructions — does it actually deploy? A distributed system proof requires a working system, not just a well-written README.
• Ask the submitter to trigger the failure scenario live: 'Kill one of the services now and show me what happens to requests in flight.' The real-time response is the proof of fault tolerance.

You'll sign in first, then come straight back here.

Proof required

Submit: (a) a load test report showing latency distribution (p50, p95, p99) and throughput under the target load, compared against the milestone-1 SLA targets with analysis of any gaps, (b) the production runbook (minimum 1,500 words covering all four sections: deployment, scaling, top-3 failure troubleshooting, and backup/restore), and (c) a Q&A log from the architecture review (minimum 500 words documenting at least three challenging design questions and your responses).

What gets checked

• Load test results show actual latency percentiles (p50/p95/p99) under load — not just success rates
• Runbook covers failure troubleshooting with specific diagnostic commands — not just descriptions of failure modes
• Q&A log shows the reviewer challenged specific design decisions with alternatives and the submitter provided reasoned responses

Common mistakes

• Load test run with too few concurrent users to stress the system — the target SLA must be validated at the stated throughput, not at 10% of it
• Runbook written as a tutorial not an operational reference — a runbook's audience is an on-call engineer at 3am; it must be specific, command-line-first, decision-tree-structured

Resources

What a verifier looks for

• You are a distributed systems practitioner. Your most valuable questions challenge architectural choices: 'Show me a sequence diagram of what happens during a network partition under your chosen consistency model.' 'Your p99 latency doubles under 2× load — where is the bottleneck and what would you change?'
• After the session, provide written confirmation (minimum 200 words) of your engineering background, the three design decisions you challenged, and your assessment of the submitter's ability to reason about distributed systems trade-offs.

You'll sign in first, then come straight back here.