Disaster Recovery Runbook Table of Contents RTO & RPO Backup Procedures Database Backups PV Snapshots Restore Procedures Full Cluster Recovery Database Restore Disaster Scenarios Complete Cluster Failure Database Corruption Procedures for backup, restore, and disaster recovery scenarios. RTO & RPO RTO: 4 hours RPO: 1 hour Backup Procedures Database Backups # Manual backup kubectl exec mysql-0 -n dev -- \ mysqldump -uroot -p --all-databases | \ gzip > mysql-backup-$(date +%Y%m%d).sql.gz PV Snapshots Use Velero for PV snapshots: velero backup create dev-backup --include-namespaces dev Restore Procedures Full Cluster Recovery Provision new K3s cluster Install ArgoCD Restore Velero backups Sync ArgoCD applications Restore database from backup Database Restore # Restore MySQL kubectl exec -i mysql-0 -n dev -- \ mysql -uroot -p < mysql-backup.sql Disaster Scenarios Complete Cluster Failure Provision replacement cluster Install core services (ArgoCD, sealed-secrets) Restore from Velero backup Verify all services Database Corruption Stop applications Restore from last known good backup Replay transaction logs if available Restart applications Full runbook includes detailed recovery procedures for all scenarios.