I recently received a question, via email, about how to get up and running with microservices in Node.js.
The person asking was interested in my Microservices w/ RabbitMQ Guide over at WatchMeCode, but was having a hard time wrapping their head around some of the aspects of microservices, discovery, shared resources, etc.
From the person asking:
I do have a question about something. Curious if it is covered in your rabbitMQ/Services section.
How do you make that leap to node micro-services. That seems so foreign to me. I have no idea what they are… and how to set up multiple micro-services that may need to “share resources”.
Like, having a node-sdk for your application etc.. then each of your micro-services registers with it… then other microservices can ‘request’ those other micro services.. by bringing in the sdk, then just calling of the registrations you want. How is that even possible? Is that even the right way to do it?
Honestly, there’s not much of a secret or trick to this.
Most of the hype and hand waving around microservice SDKs, service discovery, etc. is just that – hype and hand waving.
It makes people look good because it’s “big” or “at scale”, to talk about microservices and the surrounding tools, SDKs and services.
Microservices are a mostly simple thing.
They typically don’t need anything special, or complex.
And chances are, you already know how to build microservices.
But our industry is so fixated on companies like Netflix, AWS, Heroku, Docker, etc. that we can’t see past the hype of “web scale” and Docker (which is a great tool – I love it, but it’s way overhyped).
When it comes down to doing the work, the idea of a “microservice” is just a renaming of existing concepts – concepts that have been around for a very long time.
Call it “Service Oriented Architecture” or whatever you want, the purpose is to keep an individual service small and focused, allowing it to be used by larger systems.
A microservice is small, focused.
It will do one thing, and do it well.
If you’ve read much about the unix philosophy, this is just an extension of that idea.
A single unix / linux command line tool will do one thing, taking text as input and producing text as output. A microservice will do one thing, typically taking a message as input and producing a message as output.
The exact format and protocol of those messages can vary greatly – HTTP, SOAP, AMQP (rabbitmq) and other protocols, with JSON, XML or other common (or proprietary) data formats.
There’s nothing new or different, here. These are tools and technologies that you’ve been using for years, as a developer.
When it comes to “shared resources”, there are a number of different aspects to consider.
But once again, it’s usually easier than most people want to admit.
Do you have a database server that multiple apps use? That’s a shared resource, already.
Did you build a back-end service that handles all of your authentication needs, and it’s just an HTTP call away? That’s a shared resource.
The notion of shared resources goes well beyond the infrastructure and plumbing, though. And frankly, a database is a terrible integration layer.
But when it comes down to it, sharing resources amongst services is once again, just passing messages around like we have always done.
Where it gets interesting / complicated, is the point where we start talking about scaling services and making them discoverable.
There are a lot of very fancy “secret management” and “service discovery” systems around.
Again, this isn’t a new problem. We’ve had “service discovery” in one form or another for many, many years now.
Remember WSDL? WCF-Discovery? Aspects of CORBA, DCOM, and countless other distributed systems architecture are all built around the idea of service discovery.
These tools make scalable microservices much easier – and there are a lot of new players in this realm, taking advantage of the current technology trends.
There’s no magic in this, other than the “discovery service” providing the correct connection string for you (basically – it’s a bit more complex than that).
Instead of using JSON documents or environment variables to store API keys for a service, you connect to the “secret store” and ask for the API keys that you need.
The are great tools to use when you have need to centrally manage services and API keys.
Most of the time, you won’t need these things.
I run a few systems built with a microservice architecture, in a very large healthcare network. These are critical systems that run nightly batch processes for the financial data warehouse.
I don’t bother with these large scale system architectures and scalability problems because I don’t need to, at this point.
I point my web app, my individual services, my scheduling system, and everything else at a known RabbitMQ and MongoDB instance, and that’s it.
I use JSON configuration files to store connection strings, and I let RabbitMQ be the central hub of communication for me.
It may not be “web scale” and won’t win me any awards, but I sleep well at night, knowing my system works.
The examples I show in the Microservices w/ RabbitMQ Guide are what I use in my production apps.
I built a simple Express web app that sends a message across RabbitMQ.
I built a simple Node.js “back-end” service that receives the message from RabbitMQ, and uses the data in the message to run some other process.
Sometimes there is a return message, sometimes there’s an update to the database.
Sometimes the database in question is “shared” across mutliple apps (though I try to avoid this).
I don’t get into fancy “orchestration” and “service discovery” because I don’t need it. I just need to make sure my “node web/bin/www” command line is run when my server reboots, or when the node process crashes.
Building a system with a microservice architecture is not a quantum leap.
It’s just building small apps that do something useful, and coordinating them with HTTP, AMQP, etc.
The “trick” is figuring out which parts of your apps can and should be put into a separate node executable process.
That knowledge only comes from experience… a.k.a trial and error.
For example, stop sending email from your web app. Instead, push a message across RabbitMQ and have another node process read that message and send the actual email.
Find simple places where the web app can stop being responsible for “simple” things, like sending email, and you’ll be well on your way to building a proper microservices architecture.
Getting Started Is As Simple As Learning RabbitMQ
A part of WatchMeCode, the Guide to Building Microservices w/ Node.js and RabbitMQ will get you up and running fast.
You’ll start with an introduction to RabbitMQ, and how to install and configure it. You’ll learn the basics of messaging with RabbitMQ, and see the most common application design patterns for messaging.
You’ll learn how to integrate RabbitMQ into your own services, your web application, and more. And you’ll do it with simplicity and easy, reducing lines of code while increasing features and capabilities in your applications.
From there, this guide will show you how to build a simple, production-ready microservice using Express as a front-end API and RabbitMQ as the communication protocol to and back-end service.
It’s code that comes straight from my production needs, used by thousands of developers around the world as part of my WatchMeCode services infrastructure.
There’s no better way to get started, and no better resource than real-world code that exemplifies simplicity, running in a production environment.