How it works
One event comes in and fans out into one delivery per destination. From there, each delivery has a life of its own.
One signup. Four systems that need to know. Send it here once, get an answer in milliseconds, and let delivery sort itself out in the background.
Ada hits Create account. Before you can show her the dashboard, four other systems are supposed to hear about it: Mailchimp starts her onboarding emails, HubSpot creates a contact for sales, Slack posts to #signups, and the warehouse records it for analytics. Four APIs you do not own, sitting in the middle of your signup.
That sliver is the whole wait. All four deliveries happen behind her.
It is what happens when Slack returns a 500. Do you fail Ada's signup over a chat message? Do you swallow the error and lose it forever? Whatever you pick, you write that same retry loop four times, once per service, and it lives in your signup handler.
One call, 202 Accepted in milliseconds. The event and one delivery row per
destination are written in a single transaction, so it can never be half accepted. A
worker delivers each one and retries it on its own schedule, until it lands or gives up.
One event comes in and fans out into one delivery per destination. From there, each delivery has a life of its own.
Eight decisions carry most of the design.
Not per event. Every delivery keeps its own status, attempt count and next attempt time, so a broken destination retries alone while the rest finished long ago.
The event and its delivery rows go in together, in one transaction. There is no gap where an event is accepted but has nowhere to go. That gap is how data quietly disappears.
Postgres with FOR UPDATE SKIP LOCKED. Two workers never grab the same row
and neither one waits. No broker to run, and the ledger and the queue share a
transaction.
202 means we took responsibility, not that we delivered. A duplicate you
can recover from, a lost event you cannot. Send an Idempotency-Key and
repeats are deduped.
Each retry waits twice as long as the last, up to a cap. Jitter spreads them out, so a destination coming back up does not get hit by everything at once.
After the last attempt a delivery is marked dead and stops burning retries. It stays on disk. Fix the destination, hit replay, it goes back in the queue.
Concurrency caps globally and per destination, plus a circuit breaker that backs off anywhere failing repeatedly, so it cannot starve everyone else of workers.
Each destination picks the events it cares about with a glob like user.*,
then reshapes the payload with JMESPath. Same event, whatever shape each system wants.
None of this shows up on the happy path. One request, one 202, nothing to see.
So this breaks something on purpose. It creates a source, wires up three destinations, and
sends one event to all of them: a working endpoint, an unreachable one, and
a batched warehouse sink. Then it polls until each settles. Watch the three rows drift apart.
Real writes against the live database, then polls GET /v1/events/{id} every
two seconds. Around thirty seconds to settle.
What it is doing
Register the app once, add a destination per system, then send one event per signup. Adding a fifth system later is a config call, not a deploy.
# 1. register your app. the write key is shown exactly once
curl -X POST /v1/sources -d '{"name": "web-app"}'
# 2. HubSpot. it wants a bearer token, so put one in headers.
# the transform reshapes your payload into what HubSpot expects.
curl -X POST /v1/destinations -d '{
"source_id": "src_...", "type": "http", "filter": "user.signed_up",
"config": {"url": "https://api.hubapi.com/crm/v3/objects/contacts",
"headers": {"Authorization": "Bearer pat-na1-..."}},
"transform": {"email": "user.email", "plan": "plan"}
}'
# 3. Slack. an incoming webhook needs no auth header at all.
curl -X POST /v1/destinations -d '{
"source_id": "src_...", "type": "slack", "filter": "user.*",
"config": {"url": "https://hooks.slack.com/services/T00/B00/xxx"}
}'
# 4. now every signup is one call, and it is the only one Ada waits for
curl -X POST /v1/track \
-H "Authorization: Bearer wk_..." \
-H "Idempotency-Key: signup:u_123" \
-d '{"type": "user.signed_up",
"payload": {"user": {"email": "ada@example.com"}, "plan": "pro"}}'
# 5. later, ask what happened to it. per destination, with attempt counts.
curl /v1/events/evt_...
Mailchimp, Stripe, Segment and most other APIs work the same way as HubSpot above: an
http destination with a token in headers. Nothing about the
retry logic, the queue or the worker changes when you add one.
| POST /v1/sources | Register a source. Returns a write key, shown once. |
| POST /v1/destinations | Where to deliver, with filter, transform and batching. |
| POST /v1/track | Send an event. The only call in your hot path. |
| GET /v1/events/{id} | Status and attempt history for every destination. |
| POST /v1/destinations/{id}/replay | Dead deliveries back to pending, after a fix. |
| GET /v1/destinations/{id}/stats | Counts by status, and average attempts. |