HomeWorkInfrastructure · Databases

Infrastructure · Databases

High-availability infrastructure for a field-operations app

A field-operations application used by teams out in the field needed a production home that stays up when a server fails. I built the infrastructure across six-plus servers: load-balanced web tier, replicated database, monitoring, alerting and a written way back from disaster.

Role
Systems & DevOps Engineer
Where
Hoboetech · production
Scale
6+ servers
Stack
Apache2, MySQL, Grafana
Field usersmobile · webApache2load balancerAPP SERVERSApp node 1Apache2App node 2Apache2MYSQLPrimaryread / writeReplicastandbyGrafanaalertsBackup serverDR runbooks

Results at a glance

  • 6+servers built and configured for production
  • 2app nodes behind an Apache2 load balancer
  • Primary → replicaMySQL replication for high availability
  • Runbooksbackup servers and documented recovery workflows

The brief

Field staff depend on the app all day, so an outage stops real work. The infrastructure had to survive a failed web or database server, show problems early, and be recoverable by anyone on the team, not just the person who built it.

What I built

Web tier

Apache2 serves the application and balances traffic across the app servers, with tuned virtual hosts, sensible timeouts and per-site logging so problems can be traced back to the exact request.

Database

MySQL master–slave replication keeps a live copy of the data on a second server. If the primary fails, the replica can be promoted instead of restoring from backup.

Monitoring and alerts

Grafana dashboards cover server health, web traffic and replication status, with automated alerts on the conditions that need a human: a server down, disk filling up, replication falling behind.

Disaster recovery

Dedicated backup servers, scheduled backups and documented recovery workflows, written step by step so the restore doesn't depend on one person's memory.

Decisions and trade-offs

  • Apache2 as the load balancer. The app already ran on Apache, so using its proxy balancer kept the stack small and familiar for the team. HAProxy would be the next step if traffic grew a lot.
  • Replication and backups, not one or the other. A replica protects against hardware failure; backups protect against mistakes, because a bad query replicates instantly.

Outcome

The application runs on a load-balanced, replicated and monitored platform, with a tested path back if something goes wrong.

Contact

Got a server problem? Let's fix it.

Tell me what you're running and what's going wrong, or what you want built. I usually reply within a day (Pakistan time, UTC+5).

Ameer Hamza

Ameer Hamza

Linux Systems Engineer · Lahore, PK

Available · usually replies within a day