ZeroMQ and NodeJS: HTTP Message Queue with Axios Client. An event driven program

What is ZeroMQ? Sockets on steroids. If you are interested in the queue, (https://zeromq.org/). In fact, I can vouch for this personally. That's why I am writing out how to do this little tutorial in how to make the message queues work.

What are we building? We are going to build a simple http logger with axios that sends a message to a queue through an event driven system that then can process the message.

Now create folder, mine, if you want to follow my github repo is "ZeroMQTutorial". Now change directories into the ZeroMQTutorial directory and run the following commands:

we need axios->

npm install axios

Second we need to install ZeroMQ (we are using version 5 as of 20/July/2021)
Note This command may take a bit of time ->

npm install zeromq@5

Finally, we are going to use a npm lib called "flatted" to handle the axios circular return object:

npm install flatted

Now create a file called "app.js"

In this file, we will be creating the actual axios call. In app.js, input the following:

app.js

const axios = require("axios").default    axios({    method: "get",    url: "/",    baseURL: "https://google.com",    responseType: "stream",    }).then(response => {        console.log(response), response)    }).catch(error => {        console.log(error), response)    })

now, run the following in the ZeroMQTutorial directory:

node app.js

Did you get anything other than an error? Message me if you got an error and we'll get it sorted out.

Now, we need to make a logger. An "event-driven" logger. In the same directory, make a file called "HTTPLogger.js". In this file we will be handling the actual http call to the queue. Why not send to the queue directory? Well, say you'd like to do some type of data aggregation prior to queueing.

One last note, we are "flattening" or converting a circular object aka "object that includes itself". This is where the "flatted" library comes into play.

So, we can add some type of stuff like that. Go ahead and copypasta this:

HTTPLogger.js

const Events = require("events")const zmq = require("zeromq")const {stringify} = require('flatted');let pusher = zmq.socket("push");pusher.bindSync("tcp://127.0.0.1:3000");let emitter = new Events()class HTTPLogger extends Events{    logHTTP(data) {        data["thisIsATimestamp"] = Date.now()       var msg = stringify(data)        pusher.send(["httpLogQueue", msg] )    }}let httpLogger = new HTTPLogger()emitter.on("logHttp", httpLogger.logHTTP)module.exports = emitter

What we are doing here is basically binding a socket and a port and push to topic queues. Now, we have one more file to create and that is the "puller". Basically, using a computer port and a "socket" zeroMQ doesn't establish a "queue" but, really a lean mode of communication.

So lets make the "HTTPMessageQueue.js" file.

HTTPMessageQueue.js

const zmq = require("zeromq")class HTTPMessageQueue {    constructor() {        console.log("we are in logger")        this.puller = zmq.socket("pull");        this.puller.connect("tcp://127.0.0.1:3000");        this.puller.on("message", function (topic, msg) {            console.log(topic.toString())            console.log(msg.toString())        })    }}module.exports =  HTTPMessageQueue

What this is doing is basically taking a topic and a message from the array that was sent by the logger after a simple piece of data aggregation. This is NOT BOUND. Meaning, one pusher can have a lot of listeners. Only the pusher of messages is bound to the port.

I digress...

Last, he need to now again change our app.js file to

app.js

const axios = require("axios").defaultconst httpLogEmitter = require("./HTTPLogger")const HTTPMessageQueue = require("./HTTPMessageQueue")//Instan-new HTTPMessageQueue()//-tiateaxios({    method: "get",    url: "/",    baseURL: "https://google.com",    responseType: "stream",    }).then(response => {        httpLogEmitter.emit("logHttp", response)    }).catch(error => {        httpLogEmitter.emit("logHttp", error)    })

now, just for fun, in the ZeroMQTutorial, run:

node app.js

And that is it. You, in 3 files, were able to set up a message queue through an event driven logger.

Kick back, relax and phone the rest of it in because you've earned it my friend.

Peep the files @ https://github.com/yugely/ZeroMQTutorial

chiao