Stripe made webhooks normal for a lot of developers when it started pushing payment notifications straight to merchants’ servers, and I still think the idea is one of the cleaner ones in web software. Instead of your code asking a tracking platform over and over whether anything has changed, the platform calls a URL you gave it when something does. For fleet and field team events that sounds like a perfect fit, and mostly it is. The parts that go wrong are rarely the parts people worry about up front, though, and I managed to hit most of them in one afternoon.
The project was a proof of concept for a client with field technicians to keep track of, built on a third-party tracking platform with a long menu of event types, and I ticked every box. Arrivals, departures, zone entries and exits, connection drops, reconnections and plain location updates all went to one endpoint, and the endpoint forwarded each one to my email so I could “see what came through.” About 2,000 emails came through before I noticed, which was roughly when my phone buzzed itself off the edge of the desk.
I switched everything off and then spent a weekend writing filters I mostly didn’t need.
Pick events by what you’ll do with them
Platforms that offer webhooks group their events into similar families even when the names differ. You’ll usually find arrival and departure events tied to a destination or a saved place, boundary events for when someone crosses a zone an admin has drawn on a map (these are what vendors call geofences), status events for a device going offline or coming back, and location updates that push raw coordinates on a timer. Most also have job or trip lifecycle events for when a job is created, started, finished or cancelled.
Location updates were what buried me. A phone reporting its position every few seconds is fine when a single app on a single screen is moving a dot around, but once each of those pings becomes its own POST to your server, the request count is roughly team size multiplied by pings per minute, and a modest crew becomes a stream your handler has to keep up with all day. If you really need the raw coordinates, a bulk export or a polling API usually suits that job better than a webhook.
Lifecycle events are where I’d start now, because they’re rare and each one means something. A dispatch system that hands a technician the next job once the current one is marked finished needs exactly one event type, and on a busy day it might see a few dozen of them, which you can still read by eye while debugging. Arrival events are the natural second subscription if a customer is waiting on a “your technician is here” message. I’d leave boundary events until later, since a zone drawn a little too tight around a supplier’s yard will fire every time someone parks across the street for lunch.
Anyone can POST to your endpoint
A webhook is an HTTP POST to a URL you control, which means the URL has to be reachable from the internet, and anyone who finds it can send it whatever they like. If your handler trusts whatever arrives in the body, someone can fake a job completion and your system will dispatch the next job off the back of it.
HTTPS is required, since these payloads carry people’s names and where they are. The more interesting protection is signature checking. You and the provider share a secret, the provider computes an HMAC of the raw request body using that secret and puts the result in a header, and your handler computes the same value and compares the two. When they differ, you return an error and do nothing else. Compute it over the raw bytes as they arrived rather than over a JSON object you’ve parsed and serialized again, because a framework that reorders keys or changes whitespace will hand you a different digest and you’ll lose an hour convinced the secret is wrong. Use a constant-time comparison for that final check too.
Providers use HMAC instead of a plain hash of the secret glued to the front of the body, and the reason is a bit of cryptographic history I enjoy more than I probably should. Hash functions like SHA-256 are built so that the output is essentially the internal state of the function after it has processed the last block of input. If an attacker sees hash(secret + message) and can guess the secret’s length, they can load that output back in as the starting state and keep feeding data, producing a valid hash for the message with extra data appended, without ever learning the secret. That’s a length extension attack, and researchers used it in 2009 to forge signed Flickr API calls. HMAC avoids the problem by hashing twice, with the key mixed in separately on the inner and outer pass, so the value you see is never a state anyone can resume from. In practice you call the HMAC function from your language’s standard library and leave it there.
Providers also usually wait only a few seconds for a success response, and if your handler is writing to the database and calling a CRM inside the request, the provider gives up, marks the delivery as failed and schedules a retry, even though your code finished the work a moment later and you now hold the same event twice. Accept the payload, put it on a queue or in a table, return success, and let a background worker do the real processing.
Every provider I’ve used retries, usually with exponential backoff, where the wait roughly doubles after each failure until some cutoff after which the event is dropped or parked in a failed-deliveries list in the provider’s dashboard. When your endpoint comes back after an outage, that backlog tends to arrive in a burst.
Assume every event arrives twice
Webhook delivery is at-least-once with almost every provider, so you will get duplicates during perfectly normal operation. Each payload should carry an event ID, and you deduplicate on it with a unique constraint on the column where you record processed events, so a second insert fails and the worker skips it. Skip this and a customer gets the “your technician has arrived” text twice, and they will mention it.
Order isn’t guaranteed either, so a departure can land before the arrival it follows. Compare the event timestamps inside the payload instead of trusting the order requests reached you.
One URL or several?
You can register a URL per event type, or one URL with a small router that reads the event type field. I lean toward one URL, mostly so all the logging ends up in one place.
| Approach | Good for | Costs you |
|---|---|---|
| Separate URL per event type | Turning one event type off without touching the others | More entries to keep in sync across environments |
| Single URL with routing | One place to log and one secret to rotate | The routing logic lives in your code and needs its own tests |
Whichever you choose, store every payload raw before you parse it. I learned this when an arrival handler quietly threw away payloads after the provider added a field we didn’t expect, and we had no copy of what the original data had looked like. Logging the parsed object wouldn’t have helped, since the parser was the thing that broke.
Testing without touching production
The first time I tested webhooks against production, I didn’t mean to. I had pasted the production URL into what I believed was the staging configuration, and real technicians’ locations started scrolling past in a terminal my coworker was reading over my shoulder. Nothing broke, but I didn’t enjoy the conversation afterwards.
For early development, a request inspector like webhook.site or a tunnel such as ngrok gives you a public URL that logs each request with full headers and body, so you can see what the provider really sends before writing any parsing code.
Staging gets its own registration with the provider and its own secret. My production mix-up happened because I reused a URL and planned to swap it later.
Once raw payloads are being stored, write a small script that pushes any stored payload back through your handler on your own machine. When a payload exposes a bug, save it as a test fixture so that shape of data gets checked on every build.
Last, give the receiver a plain status page listing the most recent events and whether processing succeeded for each. A webhook that has quietly stopped arriving looks exactly like a quiet afternoon, and some providers disable an endpoint after enough failed deliveries without sending anyone an email about it.
When delivery stops altogether
Outages happen on both ends, so plan for webhooks stopping entirely. A scheduled job that polls the provider’s API for recent jobs and compares them with what you’ve recorded will find the gaps. Questions asked after the fact, like where a technician actually drove before arriving late, are better answered from stored location history than from whichever events happened to get through, and a trip replay is a good way to look at that history.
The client from that first afternoon ended up subscribing to two event types, and I’m fairly sure nobody has looked at the filters I wrote that weekend since.