9/17/2023 0 Comments Message queue example![]() ![]() Any non-trivial process needs a way to move information between each stage of the process any workflow needs a way to move the intermediate product between the stages of that workflow. We need message queues because no system exists or works in isolation-all systems need to communicate with other systems in structured ways that they both can understand, and at a controlled speed that they both can handle. The contents of each message will be driven entirely by the architecture of your application and its purposes-so for the rest of this guide, we don't need to be concerned about what's inside a message-we're more concerned with how the message gets from the system where it originates (the producer, source, publisher or sender) to the system where's it's supposed to go (the consumer, subscriber, destination or receiver). The kinds of messages we transfer today might be a note that something technical happened, like CPU usage exceeding a limit or a business event of interest, like a customer placing an order or a signal, like a command that tells another service to do something. The messaging systems in the physical world that come closest to what we have in computing are probably the pnuematic tubes that moved messages through buildings and cities using compressed air until a few decades ago (and are still used in some places today). The idea of a messaging system has been around a very long time, from the message boxes used for moving information between people or office departments (literally where the words inbox and outbox come from), to telegrams, to your local postal or courier service. The systems that are sending and receiving messages could be processes on the same computer, modules of the same application, services that might be running on different computers or technology stacks, or entirely different kinds of systems altogether-like transferring information from your software into an email or an SMS on the cellphone network. This information-a message-can be data, metadata, signals, or a combination of all three. Message Queues are a way to transfer information between two systems. Notes on the pros and cons of many popular systems available today.Patterns for fan-out and fan-in: delivering one message to many systems or messages from many systems into one.Ordering and FIFO guarantees and how they effect sequencing, parallelism and performance.Delivery guarantees that the queuing systems make (at-least-once, at-most-once, and exactly-once semantics).Why they're useful and what mental models to use when reasoning about them.What message queues are and their history.But what are they? Why are they useful? And how do we use them effectively? Which implementation do we pick? Does it even matter which one we use? And do we need to learn each of them individually, or are there more general concepts that apply to all message queues? In a way, they've been here all along-just without as many names. Whether we use them by themselves as-is to move data between parts of our application, or as an integral part of the architecture (like event driven systems), message queues are here to stay. ![]() If only there were fewer simple services that could help with publishing and subscribing, it would be so much easier to make a zero-effort choice □ Message Queues are now fairly prevalent-there are so many of them showing up so fast you'd think they were rabbits with an unlimited supply of celery, resulting in an kafkaesque situation where making a decision is like trying to catch a stream in your hands. A guide to the fundamental concepts that underlie message queues, and how they apply to popular queueing systems available today. ![]()
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |