How to test your MU Online server's disaster recovery plan
Learn to build, and above all actually test, a disaster recovery plan (DRP) for your MU Online server, with simulations of database failure, host failure, and data corruption, plus RTO and RPO metrics.
Every MU Online server administrator says they "have backups." Few have actually tested restoring that backup from scratch, under time pressure, and bringing the server back online without losing critical data. A disaster recovery plan (DRP) is only worth something after it's been tested — on paper,
Every MU Online server administrator says they "have backups." Few have actually tested restoring that backup from scratch, under time pressure, and bringing the server back online without losing critical data. A disaster recovery plan (DRP) is only worth something after it's been tested — on paper, it's just a list of good intentions. This tutorial shows how to structure a DRP for a MU server and, above all, how to simulate real disasters to validate that the plan works before you actually need it.
Why "having a backup" isn't the same as "having a plan"
A backup is a file. A disaster recovery plan is a process: who executes it, with what access, restoring what, in which order, and how to validate everything came back correctly. Servers that only have the file discover during the crisis that they're missing the hosting provider's password, that the restore script is outdated, or that the database backup doesn't include the most recent character tables. Testing the plan turns those assumptions into verified facts.
Defining RTO and RPO before any test
Before simulating any disaster, define two numbers:
| Metric | Definition | Example for a mid-sized server |
|---|---|---|
| RTO (Recovery Time Objective) | Maximum time until the server is back online | 2 hours |
| RPO (Recovery Point Objective) | Maximum acceptable data loss, in time | 15 minutes |
These numbers guide technical decisions: an RPO of 15 minutes requires incremental database backups every 15 minutes (or replication), not just a daily dump. An RTO of 2 hours requires the full restore process (database + GameServer + ConnectServer + web panel) to be documented and timed — if it currently takes 6 hours, the plan needs adjustment before any real disaster.
Inventory of what needs to be recovered
A MU server has several pieces that need to be recovered together and in the right order:
| Component | Restore priority | Backup source |
|---|---|---|
| Database (accounts, characters, items) | 1 (critical) | Incremental dump/backup from SQL Server or MySQL |
| GameServer configuration files | 2 | Versioned repository or disk snapshot |
| ConnectServer and JoinServer | 2 | Disk snapshot or container image |
| Web panel and site database | 3 | Separate backup, can have a looser RPO |
| Client assets (for redistribution) | 4 | File repository/CDN |
Without this documented inventory, it's common to restore the database first and then discover the GameServer version isn't compatible with the restored schema — an avoidable error with a proper order checklist.
Disaster scenarios worth simulating
Don't just test "the server went down." Simulate specific scenarios, each with a different response:
| Scenario | What to test | Success signal |
|---|---|---|
| Total host failure (VPS wiped) | Provision a new host and restore everything from scratch | Server online within the defined RTO |
| Database corruption | Restore from backup without the most recent dump | Data loss within the defined RPO |
| Accidental table deletion | Restore only the affected table without full downtime | Point-in-time restore without affecting the rest of the database |
| Ransomware/malware attack | Restore in a clean environment, without reintroducing the vector | Clean environment confirmed before reconnecting |
| Human error in deploy (broken config) | Fast rollback to previous version | Full rollback in minutes, not hours |
Setting up the isolated test environment
Never test a restore on top of the production environment. Set up a test VPS or local VM with the same stack (SQL Server/MySQL, GameServer, ConnectServer) and run the full restore there. This lets you time the process without risk and document every real step — exact commands, time spent on each stage, required credentials.
# Example database restore in a test environment (MySQL)
mysql -u root -p mu_test < backup_2026-07-15_0300.sql
# Example for SQL Server via sqlcmd
sqlcmd -S localhost -Q "RESTORE DATABASE MuOnline FROM DISK = 'D:\Backups\MuOnline_full.bak' WITH REPLACE"After restoring the database, bring up the GameServer pointing to that test instance and confirm login, character creation, and item persistence — restoring the database without validating that the server can read it correctly is an incomplete test.
Timing the real process
During the test, log the time for each step in a table like this:
| Step | Estimated time | Actual test time | Owner |
|---|---|---|---|
| Provision new host/VM | 30 min | — | Infra |
| Restore database | 20 min | — | DBA/Admin |
| Bring up GameServer and ConnectServer | 15 min | — | Admin |
| Validate login and character integrity | 15 min | — | QA/GM |
| Communicate status to the community | 10 min | — | Community |
If the actual time exceeds the defined RTO, the plan needs adjustment — either by automating manual steps or by pre-provisioning an already-configured standby host (hot standby).
Validating data integrity after restore
Restoring without validating is risky. After any restore test, run specific checks:
- Account and character counts match expectations (compare with the metric monitored before backup).
- Login of a known test account works and reflects the expected item and Zen state.
- GameServer logs show no schema errors or missing tables.
- Rankings and leagues load without error (an indicator of related table integrity).
Communication during a real disaster
Part of the plan is how you communicate while restoring. Prepare a template message in advance for Discord/the site informing players there was an incident, an estimated return time, and a channel for updates — this reduces duplicate tickets and protects the server's reputation even during a real outage.
Test frequency and retest triggers
Set a minimum cadence plus triggers that force an out-of-cycle test:
| Trigger | Action |
|---|---|
| Quarterly schedule | Full from-scratch restore test |
| Monthly schedule | Quick isolated database restore test |
| Hosting provider switch | Full retest before trusting the new environment |
| Emulator version update | Retest restored backup compatibility |
| Team change (admin departure) | Revalidate plan credentials and access |
Documenting the plan for someone who isn't you
A disaster recovery plan only works if another team member can execute it without you available. Document every step in plain language, with exact commands, file paths, and where to find credentials (ideally in a shared password vault, never in plain text on Discord). Test this literally by asking another team member to follow the document without your help.
Common errors and fixes
| Symptom | Likely cause | Fix |
|---|---|---|
| Restore test has never been done | Plan exists only on paper | Schedule the first full test in an isolated environment |
| Restore takes much longer than the RTO | Manual process without automation | Automate steps with scripts and consider a hot standby |
| Restored backup won't start on GameServer | Version/schema mismatch | Validate compatibility on every emulator update |
| Restored data doesn't match expectations | Incremental backup failed silently | Add backup failure alerts and integrity checks |
| Only one person knows how to execute the plan | Lack of documentation and testing with someone else | Document and test with a second team member |
Disaster recovery test checklist
- RTO and RPO defined and documented.
- Component inventory and restore order documented.
- Isolated test environment provisioned.
- Disaster scenarios (host, corruption, deletion, attack) simulated individually.
- Actual time for each step logged and compared to the RTO.
- Post-restore integrity validation performed.
- Plan documented and tested by a second team member.
- Retest schedule defined with additional triggers.
A tested plan is what separates a few-hour incident from a permanent loss of community trust. After validating your disaster recovery, also review the infrastructure foundation described in the MU Online server creation guide to make sure your backup and monitoring basics are already aligned with your plan.
Frequently asked questions
Is having automated backups already enough as a disaster recovery plan?
No. Backup is just one component. A disaster recovery plan also defines who acts, in what order, with which credentials, and how quickly — and that's only validated by testing a full restore, not just confirming the backup file exists.
How often should I test the recovery plan?
The recommended minimum is one full test every 3 months and a quick test (isolated database restore) monthly. After any major infrastructure change (host switch, new emulator version), redo the full test before trusting the plan again.
What are RTO and RPO, and why do they matter so much?
RTO (Recovery Time Objective) is the maximum acceptable time until the server is back online. RPO (Recovery Point Objective) is the maximum acceptable amount of data you can lose, measured in time. Defining both before a real disaster prevents panic decisions during the crisis.
Do I need a secondary server to test the plan?
Ideally yes — an isolated environment (test VPS or local VM) prevents the restore test from affecting the production server. Testing by restoring over the production database is risky and can cause the very outage you're trying to prevent.
What should I do if the restore test fails?
Document exactly where it failed, fix the cause (corrupted backup, outdated script, expired credential), and repeat the test until it succeeds. A plan that fails during testing is far better discovered now than during a real incident with players online.