Messaging and Madness: Sending Messages with AMQP and Amazon MQ

Originally posted on my personal blog

Table Of Contents

• Introduction
• AMQP
• AMQP and Amazon MQ
• Serialization
• Conclusion

Introduction

How do software systems talk to each other? Back-end systems can scale into giant melted together Cronenberg monsters, often making up different tools and languages. So, communicating between these services can become an untenable challenge without some shared vocabulary. We can communicate in many ways, but today I wanted to talk about asynchronous messaging protocols and figure out how AWS can help.

AMQP

AMQP stands for Advanced Message Queuing Protocol. Ive been working to implement it for some back-end software suites Im building out to enable them to talk to each other. AMQP utilizes these things called brokers to publish messages on, then on the other end, a receiving service subscribed to the same channel that we posted to can pick up that message.

via Rabbit MQ Tutorials

Lets dive a little further down; the publisher service publishes a message to an exchange on a broker. This exchange has routes that lead to queues, or channels, where the payload is published. We make sure to include the sending information with our message to be routed to the correct queue. The broker cannot see the message, although it might look into any metadata attached to the message from the publisher. This workflow asynchronously sends messages. Imagine a server version of a mail sorting machine shooting letters into the correct mail slot based on the address.

When referring to a publisher, I mean some code that we utilize to connect and send a message. AMQP is programmable, so I can shape it to fit most situations. In this case, we need to send messages to our different software suites to trigger actions to happen. Learning this took some time, but its been simple to implement.

There are different types of exchanges that we can use to make these services fit our needs. Im going to explain what we use briefly.

We use a direct exchange utilizing routing keys to bind queues to exchanges. Our code can use direct exchanges to distribute tasks to many different endpoints, but we used these direct exchanges to make direct routes between our services. Other types of exchanges can be used to broadcast messages. More information can be found here. For now, were going to focus on direct exchanges.

AMQP and Amazon MQ

We touched on all that because I wanted to talk about Amazon MQ. Amazon MQ is a fully managed platform for setting up message brokers. Amazon MQ utilizes both RabbitMQ and Apache Active MQ for creating brokers. Were sticking with Rabbit MQ for the time being.

Here above, you can see you can easily set up a broker in just a few clicks. I left most of the settings on default, except for choosing RabbitMQ for our broker engine and setting some security up for accessing our management console.

Once we get that, we have access to the RabbitMQ dashboard Amazon MQ created and is managing. Now that we have a broker set up, we can play with some code.

Above I use the library Kombu to create some connections and send some stuff. I started by setting up our environment variables. Then created exchange and queue objects. Finally, I made our connection object and the producer object, and then we sent a simple Hello message.

Serialization

Serialization is another blog post, but I chose to use JSON to serialize the payload. In the production software, I use a combination of JSON and Pickle to serialize things like image data.

Now we can see our message published on the queue I declared in our publisher service. An identical receiving service would be set up on the other side to read out messages sent to that queue.

Conclusion

In conclusion, using Amazon MQ allows us to set up managed brokers for us to send messages. With AMQP as the broker engine, we have a lightweight message-sending workflow. Thanks for reading.

-George