It’s 2020. If I must tell you what Microservices are, this is probably not for you and feel free to spare your precious few minutes elsewhere. But if you’re that person who is overwhelmed with the success stories of microservices, and compelled to get your hands dirty with the ‘panacea’; read on. Let me have the pleasure of disappointing you as well, but in a couple of minutes.
Though the thought had been buzzing for a while, the need to pen it down was stamped after a recent, gripping conversation I have had with a couple of people. I was invited to be in a quorum, to find an answer, to an intriguing question. “What is Microservices and should we follow this architecture for our solution?”
whilst the first part of the question was an easy one to tame, the second one stood tall and sensitive. A couple of minutes into the conversation, and a few facts were clear.
- The beneficiaries were to apply the microservices architecture to their upcoming product, and they sought an affirmation.
- The quorum had a high number of non- technical presence. And the more ‘technical’ the conversation progressed, the irrelevant it became.
- The long pauses and the absence of queries implied an unfamiliarity with web services; let alone microservices.
I wouldn’t condemn them for not knowing what a web service does, or how microservices can benefit or damage them. After all, they were far superior at a job I was awful at. And they were to jump into the microservices wagon; not knowing the awaiting repercussions!
The first time I ever heard the word “microservices” was back in 2013, in a YouTube video explaining the architecture of Netflix services. It was overwhelming and I skipped them without much of a hesitation. It was too much for someone who was trying to get into the field of design principles back then. But it soon became an obsession when the new project proposal announced the adoption of microservices. The design of the project was fascinating, and it still remain as one of the best codebase I have ever laid my hand on.
I will be honest. The broad possibilities of modular design kept the liabilities far away from me. And me being just an ignorant developer, hiding away from the devops, added an additional layer of density. Fast-forward half a decade and I was working with a brand-new product and a brand new set of people. And my days were filled with the problems arose by poorly designed microservices, coupled with amateur devops tactics. The dense cloud was cleared soon enough, and I was exposed the fragility of microservices. It also made me look at the entirety of the architecture, empirically. Pretty late, but better late than never!
Having seen microservices playing the role of both the protagonist and the antagonist, I exhort myself to be the devil’s advocate. If you’re an architect or a designer who is leaning towards microservices as a default architecture, I urge you to bite the bullet and ask a few questions to yourself.
Let’s admit it. Not all applications are large enough to be broken down into smaller services. Microservices as the name implies, is a collection of smaller, individual services with a purpose. On an ideal world, we would expect each service to be a complete application in itself.
Here is a hypothetical representation of “Cost per Line of Code” comparison between microservices and a monolithic service. Microservices incur a higher cost, even at its lighter form, due to the minimum resources it requires in terms of manpower and computational cost. And cost is everyone’s concern, and if not, you probably shouldn’t be taking the decision at all.
Sure, your codebase will grow in future and it may as well add a new domain itself. But you should always remember; a well designed codebase can always be switched to microservices, as and when you approach the threshold.
Let’s assume. Your product owner approached you with an idea of a HRMS application, to handle an organization with an employee size of 10k. The technology enthusiast in you had an instant solution: A Microservice architecture.
Sure, this is an extreme example, but you get the point!!
One of the major advantage of using microservices architecture is the simplicity in scaling an individual component. We may find tons of application where we will need the components to be individually scaled, but does your application really need this?
Now, this could probably be one of the hardest and strategic choice to make. Transactions spanning across multiple services are a liability to the whole architecture. Solving transactions across services means, inter-locking between the services, series of deadlocks which are hard to trace, and race conditions which can cripple the health of your services; and sometimes even the engineer’s.
REST services are stateless by definition. And they shouldn’t be participating in transactions where the boundary exceeds more than a single service. In the world of high performance, two phase commits [ 2PC ] is an unnecessary evil. And the SAGA pattern will just add another layer of complexity you weren't prepared for.
Microservices introduce eventual consistency issues because of their laudable insistence on decentralized data management. With a monolith, you can update a bunch of things together in a single transaction. Microservices require multiple resources to update, and distributed transactions are frowned on (for good reason). So now, developers need to be aware of consistency issues, and figure out how to detect when things are out of sync before doing anything the code will regret — Martin Fowler.
Is it possible to have transactions spanned across services?
But, is it worth implementing a chain of actions throughout stateless services?
On a traditional, monolithic service, each microservice instances are represented by modules within the system. Communication between modules are in-memory and the latency is near zero. The introduction of microservices means, the communication has just shifted from in-memory transactions to instructions through wire.
There are numerous amounts of tried & tested solutions available, but they all come with a price - Latency. Moving away from an in-memory transaction to network based communication will change your latency unit from nanoseconds to microseconds. Imagine having three different services talking to each other over network. Assuming each service call takes 100 ms [ which isn’t quixotic under load ], you will end up spending 300 ms just on the network alone.
And some of the applications are tightly integrated with components and services by nature. The added layer of communication can cause a catastrophic result in applications that process real-time data. Imagine having a communication delay in a surgical suit or on an airline traffic control!
- The added complexity — Certainly, complexity cannot be quantified and it can only be compared in relative terms. Though the microservices are originally designed to reduce the complexity by breaking down the application into smaller pieces, the architecture in itself, is complex to deploy and maintain.
- The cost of distribution — Microservices are distributed systems with molecularity. But the same distribution does come with a price. Your monolithic service would have been deployed over a large VM or on a preferred container. But the services needs to independently deployed [on an ideal world] with multiple of VMs or containers, in case of microservices. Sure, they can be small in size, but you do the maths. And remember, I haven’t even talked about the cost involved in the service orchestration and maintenance yet.
- Adaptation of Devops — Depends on the angle you look from, this could be beneficial or detrimental. Devops is a widely accepted and a proven operational solution. But if you’re part of a smaller organization, building a devops team can cause more damage than progress. But if one thing is for sure, you cannot maintain and monitor microservices without the presence of a dedicated devops team.
- Tight integrations — Some applications are tightly coupled by nature. Decoupling them to ‘fit in’ within the architecture will be catastrophic.
- Lack of experience — Lack of experiences is critical to any problem and they’re not just limited to SOA. But when it comes to microservices, it can enhance the damage due to the abstract definition it holds. If your microservice deployment push you onto the backseat by having an order of deployment, or crashes when one of the dependency services goes down, you’re already too late.
- End to End testing — A typical monolithic application would let you launch and run your test almost instantaneously. Having multiple services with interdependency will delay the testing without a viable orchestration.
- Chaotic data contracts — Devising and holding the data contracts within the team is tremendously different from sharing it among teams. And when you’re working with microservices, your team may not be in the same region; let alone they all follow the same programming language. Carving out data contracts for special needs will cost you time and space.
- Legacy codebase — Let’s be honest. For most of us, handling legacy codebase is a day to day activity. It is the bread an butter of most of the organizations. The rapidly changing technology advancements are putting us ahead of game, and the same time, it’s also isolating ourselves further away from the legacy codebase.
Are you sure the RabbitMQ framework you just developed, works well with your legacy application hosted in an IBM AIX server ?
- Debugging distresses — Each service will have its own set of log files to go through. More services = more log files.
Am I here to tell you “Not to use microservices”?
Microservices has maintained a well deserving notoriety from time to time. They have solved the problems that we all thought are insoluble. The Netflix story of adapting microservices has been an inspiration for many. And the list just doesn't stop at Netflix. Uber, SoundCloud, and the giant Amazon itself is some of the examples you will find in real time. And don’t think the success stories are limited to the consumer applications only. I have had my first-hand experience with a US healthcare giant and have been fascinated by the design possibilities, every single time I open up the source code.
I wouldn’t condemn your credulity if you have jumped the wagon half a decade ago. The time was different, and all we can do now is to be honest about it. But it’s 2020. We burnt our hands enough and we have too many roughhouses around us. By infusing microservices architecture gratuitously, you’re just going to turn your bad code into a bad infrastructure
I love an enthusiastic programmer. I once was and still am one. They worship what they do and goes beyond the expectations to solve one’s problem. But you cannot have the same energy in a decision making, that can cost you and the organization a fortune. I’m sorry to disappoint you. Microservices shouldn’t be your default application architecture. They’re not the silver bullet you were looking for. Keep yourself steady with KISS and YAGNI.
As a technology advocate and enthusiast, you’re entitled to have your favorites. What makes you prominent, however, is the ability to choose pragmatically, when the options are between “the right choice” and “your favorite choice.”