Migrating Monolith to Microservices

Step-by-Step Guide to Migrating a Monolith to Microservices

The microservices architecture deployment can be a good solution to this challenge. It is based on the idea of extracting large components into multiple, self-sufficient, and deployable functional entities grouped by a purpose. Each of these components is responsible for its own specific functions and interacts with other components through an API.

According to Gartner Peer Community, 85% of enterprises, including such giants as  Amazon, Netflix, Uber, and Spotify, have already implemented microservice architectures. This widespread microservices adoption is driven by many advantages such as improvements in resilience, high scalability, faster time to market, and smooth maintenance.

However, migration from a monolith to a microservices-based ecosystem may turn to be quite challenging and time-consuming, especially for organizations that operate large and complex systems with a monolithic architecture. The fact that microservices can peacefully coexist with a monolithic app gives a good trade space for such cases.

First of all, it allows you to stretch the microservices migration process (and corresponding investments) in time thus reducing immediate cost & effort loading, and stay fully operational through the whole process. Moreover, you don’t necessarily need to move the whole solution to microservices. It may be a good idea to go with a hybrid strategy when you extract only those parts that become hard to handle inside the monolith and keep the rest of the functionality unchanged.

Sigma Software team regularly helps our clients with such assignments and we have already experienced first-hand that with common sense, proper skill, and competent planning refactoring to microservices becomes quite a feasible task that can substantially simplify further development and maintenance activities.

Why Migrate from Monolith to Microservices?

Well-designed and implemented microservices can help address typical monolithic architecture problems such as system scalability, slow delivery process, defect detection & verification difficulties. With software components tightly interrelated inside the monolithic architecture, changes to a single line of code affect the whole application.

Thus, even minor system modifications require re-deployment of the entire system and turn new features release and bug fixing into a difficult time-consuming operation. Same with system scalability – you would need to scale the whole monolith application, even if just a small part of the system with specific functionality needs scaling.

A microservices architecture addresses these problems by breaking your large application down into the smallest possible parts segmented by specific domains that communicate with each other through lightweight APIs and look like a single application to end-users.

Microservices are stored independently of each other, which means that each of them can be created, deployed, tested, or updated separately (and by different teams). If one of the components fails, the development team can promptly switch to another version of this component and the application will continue to run effectively.

From an architecture perspective, the major difference between the monolithic and microservices approaches is the following:

As you can see, the Microservices approach turns your app into a set of independent components providing more flexibility to your development process and resulting in such benefits as:

• easier deployment and maintenance
• reduced downtime through fault isolation
• flexibility in choosing tech stack and easier scalability
• simplified cross-team coordination

However, once you replace the monolith with a set of independent components, integration between those becomes a great challenge. This aspect requires special attention to both the design, implementation, and testing part of the migration process.

General Approach to Migrating from Monolith to Microservices

As with any project related to architectural changes, migrating from monolithic architecture to microservices requires thoughtful and careful planning. Our core recommendation here would be to gradually cut off parts of the functionality from the monolith one after another, using an iterative approach and keeping those parts relatively small. This will result in reducing migration risks, more accurate forecasting, and better control over the whole project progress.

An excessive surge of optimism and desire to gain all the promised benefits as soon as possible may prompt to move towards a seemingly faster Big Bang rewrite when functionality for all microservices is being rewritten from scratch. However, we recommend thinking twice and go with the approach suggested above. It may take a bit longer, but is definitely less risky and more cost-efficient in the long run. Also, it prevents the complexity and potential issues of an all-at-once migration. By breaking it down into manageable chunks, you gain better insights into the interdependencies of your system and can course-correct more easily.

Of course, there can be cases when parts of the functional entities inside the monolith software are so tightly interrelated that the only option to run those as microservices is rewriting. However, such cases rarely cover the full functionality. So, we usually suggest extracting everything that can be isolated first and only after that taking the final decision on the remaining functionality. It can either be rewritten into separate microservices from scratch or left within the monolith which will become far easier to maintain after most of the functionalities have already been detached.

How To Convert Monolithic To Microservices One By One

With the suggested approach, a typical process would include the following steps:

Planning

• Identifying microservice candidates, creating a backlog
  Analyze the monolith to identify coupled functionalities that can be extracted into independent services. Further, organize the identified candidates into a prioritized backlog based on candidates’ business value, complexity, and interdependencies. This will help shape a clear roadmap for the migration.

Iterative implementation (this set of steps is repeated for each microservice from the backlog)

• Choosing the refactoring strategy among the following:
  Strangler Fig Pattern – gradually replaces the monolith’s functionality with new microservices.
  Branch by Abstraction – allows new functionality to be developed independently while maintaining compatibility with the existing monolith.
  Anti-Corruption Layer – isolate the new microservice from the monolith’s legacy code, preventing the “corruption” of new design patterns by old ones.
• Designing the microservice and changes according to CI/CD & testing process
  Clearly define your new microservice’s role and how it interacts with others. Build it directly into your CI/CD system and set up a testing process for it right away.
• Setting up CI/CD
  Adapt your CI/CD pipeline to automatically build, test, and deploy the new microservice to enable rapid and low-risk iterations.
• Implementing microservice
  Develop the microservice, focusing on its functionality and ensuring it can operate independently.

Competent planning and design are keys to smooth migration, as they significantly increase the chances of success. At the same time, it is also important to pay reasonable attention to the implementation itself and make sure you have available specialists who will take care of your entire migration process including integration of the dedicated functionality with the monolith and thorough testing of each release.

How to Choose Microservice Candidates

First of all, you need to create a backlog for microservices adoption. This requires identifying candidates (parts of the functionality that should be turned into separate microservices) and prioritizing those to form a queue for further migration. Ideally, those candidates should be functional groups that can be isolated and separated from the monolith with less effort and solve some pressing problems your app already has.

Of course, the most obvious candidate is an area that has certain performance and resource utilization issues, or a domain area that will unblock other microservices’ separation from the monolith (e.g. some functionality that is widely used by other parts of the monolith).

However, if you are looking for a more inclusive approach to determining what can be decoupled from the monolith, here is a list of strategies you can apply during your migration to microservices:

Dividing the Application into Layers

A standard application includes at least three various types of elements:

Presentation layer – elements that manage HTTP requests and use either a (REST) ​​API or an HTML-based web interface. In an application with a complex user interface, the presentation layer is frequently a significant amount of code.

Business logic layer – elements that are the nucleus of the application and apply business rules.

Data access layer – elements that access infrastructure parts such as databases and message brokers.

In this strategy, the presentation layer is the most typical candidate that can be transformed into a microservice. Such decoupling of the monolith has two main advantages. It allows you to create, deploy, and scale front- and back-end functionality independently from each other. Besides, it also allows the presentation-layer developers to quickly iterate the user interface and smoothly carry out microservices testing.

Nevertheless, it makes sense to split this layer into smaller parts if the application interface is big and complex.

Same works for the business logic layer and data access layer.  Any of those is often too large to be implemented as a single microservice as it would still be too big, complex, and pretty similar to the monolith you already have. So, you would need to split those further into smaller functional groups using any of the following approaches.

Dividing the Application into the Domain Areas

In case some functionality inside the application layers can be grouped into certain domain areas (e.g. reporting or financial calculations), each of those domain areas can be turned into a microservice.

Since the functionality inside one domain area often has more connections inside the area than with other domain areas, this approach quite accurately guarantees a good degree of isolation. This means you will be able to separate this functionality from the monolith with fewer difficulties.

Dividing the Application into the Modules

In the case of grouping, dividing functionalities into domain areas does not look like a fair alternative, it makes sense to consider grouping the functional entities into a microservice by the non-functional characteristics they have in common.

If a part of functionality calls for a fully independent lifecycle (which means committing code to a production thread), then it should be decoupled into a microservice. For example, if the system parts grow at different rates, then the best idea is to split these components into independent microservices. It will enable each component to have its own life cycles.

The load or bandwidth can also be the characteristics by which you can distinguish a module. If the system parts have different loads or bandwidth, they are likely to have different scaling needs. The solution to this is splitting these components into independent microservices, so they can scale up and down at different speeds.

Looking for professionals to help you migrate your software to microservices?

Let’s discuss what our Microservices Team can do for you.

One more good example is the functionality that should be isolated to prevent your application from certain types of crashes. For example, functionality that depends on an external service that is not likely to comply with your accessibility goals. You can turn it into a microservice to insulate this dependency from the rest of the system and embed a corresponding failover mechanism into this service.

Once you have identified the list of candidates to separate into independent microservices, you need to carefully analyze not only the candidates but also the interdependencies between those. As a result of this analysis, you will be able to create the backlog and define the order in which you will migrate the candidates. In some cases migrating certain functionality into a microservice can greatly simplify and speed up the process of migrating subsequent components. This should also be taken into account.

A good example of an early microservice migration is User Authentication and Authorization. It’s often a great first choice because it’s a self-contained unit with clearly defined inputs (like a username and password) and outputs (like a login token or a permission check). It typically manages its own data (user profiles, roles) and doesn’t usually need to dig deep into the complex, changing data of other core business processes.

On the other hand, migration of the functionality that is deeply connected with almost every part of your system can be tough. It often shares and modifies many critical database tables used by multiple other features. Trying to extract this component early would require disentangling a vast number of complex, transactional dependencies, risking data inconsistencies, breaking fundamental business flows, and potentially hindering operations. It’s a high-stakes, high-complexity target that should be approached much later in the migration process.

So, when developing a monolith to microservices migration strategy, it also makes sense to plan the availability of resources required for migration – either in-house or outside the existing development organization.

Migrating from monolithic architecture to microservices. 4 main steps

Once you have figured out which components and in which order to extract, it is time to pick up the first candidate in your queue and project further actions – define the refactoring strategy, design the microservice, estimate the efforts, and plan the iteration. It is also important to remember that the new approach would require certain changes in your CI/CD and testing processes, so it makes sense to agree on those changes with your team and adjust your processes accordingly.

Step 1: Choosing a strategy for refactoring a monolith to microservices

The next step is to select the right strategy to migrate the chosen functionality into a separate microservice. Two proven strategies we normally rely on in our projects are:

Both strategies have proven their efficiency. However, they both also have one vulnerability in common. Once the migration process is done in parallel with active development, new dependencies may emerge between the monolith and candidate functionality while the new microservice is being implemented.

In this case, you would need to carefully track and resolve those through new API endpoints or through shared libraries with common parts of a code. And on each occasion, this would take more time and effort than if the functionality was still a single whole with the entire monolithic application.

Step 2: Designing microservices architecture and changes to CI/CD & testing processes

First of all, you need to design the future microservice and define changes that should be brought to your CI/CD & testing processes. This important step forms the baseline for further successful migration of the microservice.

To design your microservice, you need to:

• determine the exact code in your app that will be attributed to the microservice
• design API interfaces through which the microservice will interact with the monolith and other microservices>
• choose a stack of technologies that will be used for the implementation of communication with microservice (in certain cases, operational demands dictate the choice of a specific technology, but most commonly, your development team can opt for their favorite tech stacks).

The new microservice will require a corresponding CI/CD (Continuous Integration/Continuous Delivery). Thus your existing process may require significant architectural shifts. In case you did not use CI/CD processes before, it may be the right time to introduce those as they are crucial for any considerable refactoring of existing code. CI/CD helps automate and accelerate all phases from coding up to deployment as well as reduce error detection time.

You also should bear in mind that with the transition to microservices, you also will need to make some changes to your testing strategy for microservices. When a microservice is launched, testers should take into account that now they can interact directly with each microservice. The test team would now need to look for issues at the levels of microservices interactions and infrastructure (containers/VM deployment and resource use) as well as at the overall application level.

Each of these levels requires its own set of testing tools and resources. Test automation can be of much help in this case.  With the right microservices testing strategy in place, your team can seamlessly migrate to microservices at the optimal time and without hassle.

Step 3: Setting up CI/CD

The process of changing CI/CD can be started concurrently with microservice development, or even before its implementation so that you can successfully maintain microservice during its development and after it is launched.

Continuous Integration will allow you to easily integrate your code into a shared repository. Continuous Delivery will allow you to take the code stored in the repository and continually deliver it to production.

There is an even more effective way. CI can transform code from a repository to a compiled package (ready to be deployed) and store it in Artifactory. CD can deploy compiled packages from Artifactory to the destination environment. In this case, CI and CD processes are independent of each other, don’t have intersection steps, and could be changed/improved independently of each other.

Step 4: Microservice implementation & testing

The time has come to implement everything that has been planned. At this stage, the development team encapsulates the functionality into an independent microservice, cuts off the tightly-coupled connections with the parent application, and implements the API gateway that becomes a single point through which the microservice can communicate with the parent application and other microservices.

Thorough testing should be an integral part of the microservice implementation as you need to safeguard that the microservice works exactly as expected and all possible errors or problems are tackled before the new environment is launched. Test coverage development can start in sync with microservice implementation (or even in advance when it goes about tools and test types selection and planning) to monitor the results of the microservice as early as possible (this can make test results more accurate). Testing should be considered completed when the microservice operates flawlessly in the new environment.

The final result of the implementation step is a fully-functional independent microservice that operates in isolation from the monolithic application. With proper and detailed planning, deployment of microservices is completed as forecasted and does not create any delays or hindrances in the overall development process.

Migrate to Microservices with Sigma Software

The transition from monolithic to microservices is a complex task, however, it can be simplified if done sequentially. Microservices have their advantages over monolithiс systems, but there must be good reasons for moving from monolith to microservices, as it requires great efforts and highly skilled specialists, who can plan and execute the migration.

We recommend migrating from monolith architecture to microservice if your existing architecture has become too difficult to operate and manage, and put your best efforts into strategy and design. Today scalability and agility are keys to success – microservices provide both.

We have already helped several clients migrate to microservices and have experienced first-hand that this task is achievable. With the proper planning, design, and competent implementation, investments into migration will pay off quite quickly.

If you have any questions or need help regarding microservices architecture refactoring, please feel free to contact us, we would be happy to share more insights and guide you through the process of choosing the most efficient strategies tailored to your specific needs.

• #Cloud & DevOps
• #Information Technology
• #Software Product Development