← InfrastructureDisaster Recovery

Disaster Recovery Plan

BasicStack maintains a comprehensive disaster recovery plan with documented procedures, regular testing, and defined recovery objectives to ensure business continuity.

Recovery Objectives

  • Recovery Time Objective (RTO): 4 hours - Full service restoration within 4 hours of a disaster event
  • Recovery Point Objective (RPO): 24 hours - Maximum acceptable data loss is 24 hours (one daily backup cycle)
  • Service Priority: Authentication (Pocket ID) → CMS (Directus) → Documentation (BookStack)

Disaster Scenarios

Our DR plan covers:

  • Infrastructure Failure: Complete Hetzner region outage
  • Data Corruption: Database corruption or accidental deletion
  • Security Breach: Compromise requiring complete infrastructure rebuild
  • Human Error: Accidental configuration changes or deletions

Recovery Procedures

  1. Assessment: Identify disaster scope and impact (15 minutes)
  2. Communication: Notify stakeholders and display maintenance page (15 minutes)
  3. Infrastructure: Provision new Kubernetes cluster in alternate region (1 hour)
  4. Data Restoration: Restore databases from latest backup (1.5 hours)
  5. Service Deployment: Deploy all services from Infrastructure as Code (1 hour)
  6. Verification: Test all services and data integrity (30 minutes)
  7. DNS Update: Point DNS to new infrastructure (5 minutes + propagation)

Testing

We test our disaster recovery procedures:

  • Quarterly DR Drills: Full recovery simulation in isolated environment
  • Monthly Restore Tests: Database restoration verification
  • Annual Tabletop Exercises: Team walkthrough of disaster scenarios

Documentation

Detailed runbooks are maintained in BookStack with step-by-step recovery procedures, emergency contact information, and decision trees for different disaster scenarios.