Definition
A message queue is a software infrastructure component that enables asynchronous communication between separate processes, applications, or services by storing messages in a buffer (the “queue”) until they are retrieved and processed by a consumer.
Overview
Message queues are employed to decouple producers (senders) from consumers (receivers), allowing each to operate independently of the other’s availability and processing speed. Typical implementations provide features such as guaranteed delivery, ordering of messages, persistence to disk, and support for multiple concurrent consumers. They are widely used in distributed systems, micro‑service architectures, and event‑driven applications to improve scalability, reliability, and fault tolerance. Prominent examples of message‑queue systems include RabbitMQ, Apache ActiveMQ, Amazon Simple Queue Service (SQS), and Microsoft Azure Queue Storage.
Etymology/Origin
The term combines “message,” referring to a unit of data transmitted between communicating entities, and “queue,” derived from the French word queue meaning “tail.” In computing, a queue denotes a first‑in‑first‑out (FIFO) data structure, a concept formalized in early algorithmic literature of the mid‑20th century.
Characteristics
| Characteristic | Description |
|---|---|
| Asynchronicity | Producers can place messages on the queue without waiting for immediate processing by consumers. |
| Decoupling | Producers and consumers do not need to know each other’s runtime state or location. |
| Ordering | Most queues preserve FIFO order, though priority or custom ordering mechanisms may be supported. |
| Durability | Messages can be persisted to disk or replicated across nodes to survive failures. |
| Reliability | Systems often provide acknowled‑gement protocols (e.g., at‑least‑once, exactly‑once delivery) to avoid loss or duplication. |
| Scalability | Multiple consumers can read from the same queue (competing consumer pattern) to share workload. |
| Security | Access control, encryption, and authentication mechanisms are typically incorporated. |
| Transport Protocols | Commonly use TCP/IP, AMQP (Advanced Message Queuing Protocol), MQTT, or proprietary protocols. |
Related Topics
- Message broker – middleware that routes, transforms, and delivers messages, often built on top of a queue.
- Publish–subscribe pattern – a messaging paradigm where messages are broadcast to multiple subscribers, sometimes implemented via topic exchanges on top of queues.
- Event sourcing – a design approach that records state changes as a sequence of events, often stored in an append‑only log rather than a traditional queue.
- Distributed streaming platforms – systems such as Apache Kafka that provide persistent, ordered logs for high‑throughput event processing.
- Inter‑process communication (IPC) – broader category of mechanisms (pipes, shared memory, sockets) for data exchange between processes, of which message queues are one method.