Message queues architectures

Event-driven architecture

Web-queue-worker architecture

Event-driven architectures:

• Events are delivered in near real-time and the application instances respond to them as they occur, without blocking each other.
• They can use a pub/sub or streaming architecture.
• In a pub/sub architecture, the infrastructure tracks subscriptions and sends events to each subscriber. After the message is received, it can’t be replayed. New subscribers don’t see the event.
• In a streaming architecture, events ordered and written to a log. Instances don’t subscribe to the stream, they can read any part of it and advance through it. New clients can join at any point in time and replay events.

The consumer can process events in at least three ways:

1. Simple processing: every event triggers an action in the consumer, e.g. sending an email.
2. Complex processing: the consumer processes a series of events, looks for patterns and does an action, e.g. aggregate transactions over a time window, generate a notification if the average crosses a certain threshold.
3. Event stream processing: the consumer ingests a feed of events and sends it to stream processors that process and transforms the stream, e.g. analytics and IoT workloads.

Web-Queue-Worker architectures:

• Common in front-ends sending client requests to workers that perform resource-intensive, long-running background or batch jobs, e.g. a single-page web app calling an email or sms service.
• This is a relatively simple architecture, with clear separation of concerns, decoupled and easy to scale independently. However, without careful consideration, both the front-end web application and the backend workers can easily become monoliths difficult to maintain.
• For scalability, the workers can share state in a distributed external cache.