Ameer HamzaLinux Systems Engineer Hire me

HomeBlog

· 6 min read

Why I wrote my own Prometheus exporter for SMS gateways

Kannel, smppbox and KSMPPD only show a status page. Here's how I turned them into metrics, and the counter-reset bug that nearly made the graphs useless.

If you run an SMS business on Kannel, smppbox or KSMPPD, you know the status page. It tells you what is happening right now, and nothing else. No history, no "which customer", and no alert when an operator link drops in the middle of the night.

I wanted answers to questions like these:

  • Which customer is close to their TPS limit?
  • Which operator link has been flapping all week?
  • Why did delivery reports drop at 2 a.m.?

So I wrote exporters for Prometheus, built Grafana dashboards on top, and open-sourced the whole thing: kannel-smpp-ksmppd-prometheus-exporter.

Three rules I set myself

  1. Don't touch the gateways. Production SMS gateways are not where you experiment. The exporter only reads the status page the gateway already serves.
  2. One exporter can watch many gateways. Installing an agent on every box is more work to keep up to date, so central mode is the default.
  3. Keep it small. One Python file per gateway type and two dependencies. Anyone on the team can read it.

The bug that almost broke everything

Prometheus counters are supposed to only go up. When a counter drops, rate() assumes it was reset and compensates.

smppbox resets a session's counters every time a customer reconnects. Customers reconnect all the time, and every reconnect looked like a reset. With several sessions per customer coming and going, the graphs showed spikes of billions of messages that never happened.

The fix was to have the exporter remember each session's last values and publish its own running totals that never go down. It's a few lines of code, but without it the dashboards would have been useless and people would have stopped trusting them.

Dashboards people actually open

Every dashboard follows the same path: all gateways → one server → one customer or link. The panels have hover help, because the person looking at 3 a.m. may not be the person who built them.

Alert rules ship with it: gateway down, SMSC link down or flapping, queues growing, customer disconnected, TPS limit reached. That's 29 rules across the three exporters.

Why open-source it?

Because I couldn't find one that did this when I needed it. There's also a Docker demo with a fake gateway that generates traffic, so you can see the dashboards in a minute without owning an SMS gateway.

If it saves you a night of debugging, a star on GitHub helps other telecom engineers find it.

Written by Ameer Hamza, Linux systems engineer in Lahore. Need help with something like this? Get in touch.