I haven’t spent that much time looking at how RabbitMQ really works, recently. Sure, I’ve been using it a lot but I was using rabbit.js on top of it, which hides a lot of the details on how it works. I was happy with rabbit.js for this reason – but I recently switched one of my apps over to LeanKit’s “wascally”. In the process of doing that, I realized that I still had no clue how RabbitMQ was really working with regard to queues, exchanges and binding them.
So I dug in to the wascally specs, pulled out my RabbitMQ In Action book again, and started cramming information in to my feeble brain, hoping something would click. And just now, sitting in an airport on my way to a conference, something did click. Suddenly, the relationship between exchanges, queues and binding them with routing keys, makes sense. That’s not to say I perfectly understand the right combinations of things, yet… but that I at least have the right base line to work from, which will make the rest of it relatively easy to figure out.
The spark that revealed the relationships to me, was a slow but sure realization that I was creating and binding my exchanges and queues in the wrong place. It started as a confused, “what?” and “why would I … ???” which ultimately turned in to me realizing I was doing things backward.
RabbitMQ Exchanges and Queues
Exchanges are the place to which you publish a message. They are responsible for routing the message to the appropriate queue, through the use of an exchange -> queue binding.
Think of an exchange like the postal service. You specify an address on your envelope when you want to mail something, but you do not drive to that person’s home and insert the envelope in to their mailbox, yourself. Rather, you then take your envelope to your local postal service center – the “exchange”. Once the envelope is in the hands of the postal service, you don’t worry about it anymore. You expect the postal service to figure out how to get your envelope to the destination.
RabbitMQ works the same way. You “address” the “envelope” of the message with information about the exchange to which you are publishing, and with an optional routing slip (both the exchange and routing slip can be considered part of the “address”) and a few bit of other information. Once you send it to the exchange, you expect the exchange will route the message to the correct queue – the “inbox” or “mailbox”. Code on the other side is subscribed to the queue in question, and receives the message.
When you publish a message to RabbitMQ, you are doing through an exchange. Even if you think you are publishing directly to a queue, you are actually publishing through the “default” exchange, which is bound directly to the queue that you specified.
Exchange -> Queue Bindings
The “routing” that happens inside of an exchange is done through bindings. Having an exchange and a queue in place is simply not enough. If you don’t have them bound together appropriately, your message will never go anywhere. You can drop your envelope in an out-bound mailbox, but if the postal service never comes to collect the outbound mail, it will never get to the intended destination.
When working with RabbitMQ, then, you need to have both an exchange and a queue, and these need to be bound together with routing information. The routing information used depends on the type of exchanges, routing keys and other things. I won’t go in to detail here, because I’m still figuring these things out. But in general, the binding is there to ensure the messages move from the exchange to the correct queue.
Where Are They Defined And Bound??
In my experience, this default exchange has been one of the causes of my confusion about exchanges. I started my RabbitMQ experience by publishing directly to a queue – or so i thought – because that was how I did things with WebsphereMQ, 6 years ago. I thought I was publishing directly to a queue in RabbitMQ, and I got confused. I didn’t understand the point of exchanges, because I was always defining my exchanges and my destination queue within the publisher – exactly the wrong place to do it!
Having declared both my exchange and queue, and bound them together, within the publisher is a bit like me driving to my post office, handing them an envelope, then getting in a postal delivery truck and driving to the address on my envelope so that I can deliver it. It completely bypasses all of the benefit of having exchanges separated from queues.
It turns out, the exchange should be declared within the publisher, but the destination queue and binding should not. It is a much better idea to have the subscriber bind the queue to the exchange. By doing this, the publisher does not care at all about the destination. It only cares about the exchange to which it publishes. I only care about getting to my local post office. I don’t care how they get my envelope to the mailbox of my intended recipient.
On the subscriber side, things are a bit different and there is some debate here. I generally think the easiest way to set up the subscriber is to have it declare the exchange, the queue and the binding all together.
I think it’s possible to have the subscriber only declare the queue and exchange binding, but I think it is safer to have it declare the exchange as well. I’m not sure what would happen if the client declared a binding to an exchange that did not exist. It probably wouldn’t do anything, or would throw an error.
The debate, then, is whether or not the subscriber should declare the exchange and binding. It can be argued that the binding between exchange and queue is a completely separate concern from both the publisher and subscriber. I agree with this in principle, but this introduces other difficulties in RabbitMQ. Since there is no need for any kind of configuration or administration tool within RabbitMQ, and since the RabbitMQ protocol itself defines the ability to create exchanges, queues and bindings, I find the introduction of a 3rd tool to manage the bindings to be inconvenient at this point. I prefer to let my subscriber code manage this binding relationship. That does not mean I won’t change my mind in the future, but right now, this is my preferences.
Forward Progress, Finally
I’ve felt like I was missing a huge piece of the RabbitMQ puzzle, for the last year. I was declaring my exchange and queue inside of the publisher, wondering why I should even bother with exchanges – what was the point of them, if my publisher had to declare the exchange, queue and binding, after all?
With this one realization – that only the exchange should be declared within my publisher – I seem to have turned around, mid-stream, and started to swim with the flow of RabbitMQ, instead of trying to work against it. I am hopeful that this revelation will help to further enhance my productivity with RabbitMQ, and increase the rate at which I learn more about the details and configuration of RabbitMQ, and ultimately the system design around RabbitMQ.
Update: RabbitMQ For Developers
Having worked with RabbitMQ for a much longer period of time now, I have built a complete training package to help get you up to speed.
The RabbitMQ For Developers bundle is the ultimate resource package with 12 screencasts, 2 ebooks, and a series of interviews with experts from around the world. You’ll be up and running with RabbitMQ in minutes, with this complete bundle!