architecture

Webhooks vs Event-Driven Architecture: Reliability, Retries and Idempotency

Choose between webhooks and brokered events by making delivery guarantees, retries, idempotency, security and recovery explicit.

Ler em português
A webhook HTTP path compared with a brokered event flow showing retries, queues and independent consumers

Webhooks and event-driven architecture are often presented as competing options. They are better understood as patterns at different levels.

A webhook is typically an HTTP callback: when something happens, one system sends a request to an endpoint owned by another. Event-driven architecture describes systems that publish and react to events, often through a broker or streaming platform. A webhook can carry an event, but it does not automatically provide the durability, fan-out, replay, ordering, or consumer isolation associated with an event platform.

Choose based on the integration contract

Webhooks work well when the producer knows the subscriber endpoint, the number of consumers is manageable, near-real-time notification is useful, and HTTP delivery is operationally acceptable. They are widely accessible and simple for external partners to adopt.

A brokered event approach becomes attractive when many independent consumers need the same event, producers should not know consumer locations, traffic arrives in bursts, replay matters, or consumers must progress at different speeds.

Neither option removes distributed-systems failure. Networks time out, consumers become unavailable, and acknowledgements get lost.

Design for at-least-once delivery

Most practical delivery systems can send the same event more than once. Consumers must therefore be idempotent. Assign every event a stable identifier and store enough processing state to recognize a retry. Where a business operation has its own natural idempotency key, preserve it across the entire workflow.

Do not treat an HTTP timeout as proof that the receiver did nothing. The receiver may have committed the action while its response was lost. Retrying without idempotency can duplicate payments, messages, or state transitions.

Retries should use bounded exponential backoff with jitter. After the retry policy is exhausted, move the event to a recoverable failure path such as a dead-letter queue or operator-visible delivery record. Infinite, invisible retries hide incidents and consume capacity.

Secure the boundary

Webhook receivers should verify a signature computed over the original payload, enforce a timestamp tolerance to reduce replay risk, and use constant-time signature comparison. Rotate secrets safely and document how subscribers can test verification.

Authentication does not replace input validation. Treat every event as untrusted, limit payload size, validate the schema, and authorize the event’s effect on the receiving system.

For internal event platforms, apply identity, encryption, topic-level authorization, schema controls, and tenant boundaries with the same care.

Make evolution and ordering explicit

Version event schemas and prefer backward-compatible changes. Consumers should ignore fields they do not understand, while producers should not remove or reinterpret fields without a migration plan.

Global ordering is expensive and often unnecessary. Define the smallest ordering domain the business requires—such as events for one order—and partition accordingly. Consumers must still handle late, duplicated, and occasionally missing data.

Observe delivery as a product

Track delivery latency, attempts, success rate, queue age, dead letters, consumer lag, and processing outcomes. Give operators a safe way to inspect and replay failed events. Include correlation identifiers so an event can be traced from producer to consumer.

The architectural question is not simply “webhook or events?” It is which delivery contract the business needs—and whether the chosen design makes duplication, delay, failure, security, and recovery explicit.

Reference

The AWS Builders’ Library article on timeouts, retries and backoff with jitter explains why retries need bounds and randomized delay in distributed systems.

Related articles

How Modern Systems Integrate: APIs, Webhooks, Events and QueuesStructured Logging Done Right: Trace Context, PII Redaction and Retention

Ready to improve your platform?

Share the current context and the outcome you need. We start with the engineering problem and define a practical path forward.