Skip to main content

Continuous Delivery & Microservices, Part 2

May 3, 2018 by TC Currie

Here at Armory, we believe that obtaining the value of microservices is a critical component of continuous delivery. A lot of companies are moving from monolith to microservices architecture, but they often end up duplicating effort in their systems which is an inefficient use of resources and leaves the promise of microservices, unfulfilled.

The value of microservices is evident in its ability to add a feature to one component without affecting another. The assumption is that the software development lifecycle (SDLC) cost-per-feature is reduced.

Every organization moving toward this type of architecture has a different understanding of microservices and best practices regarding them. Any instance of an individual microservice is constantly changing making it hard to navigate this transition. The architecture transition, make no mistake, is vast.

This two-part article shares some of the lessons we’ve learned while helping our customers through this journey. In Part One, we shared tips on breaking into using microservices, tooling, and determining service boundaries. Part Two will discuss security, testing, teams, and the most significant challenges we’ve faced.

New Security Requirements

The use of microservices significantly increases the need to secure data. Data is accessed from many different services and locations, and through a number of different APIs, therefore teams have to give access to data on more discrete datasets.

A lot has been written about segregating data and responsibility. For example, one service may be responsible for password data while another is responsible for sensitive user data.

Depending on the size of your organization this can look very different, because the services requesting access to your data may be in different parts of the country. There are three ways to solve this problem:

  • Token Based Authentication: this is a great place to start because it’s simple to implement
  • Mutual TLS: Two-way authentication is when two parties authenticate each other simultaneously. The Mutual TLS method requires that the service which is accessing the data be granted certificates which makes it a secure option. The downside of this methocanard is managing certificates for both parties. Netflix created an open source service called “Lemur” to help with certificate management.
  • Security Groups: these are effective, but are more difficult to implement. Although it becomes harder to apply across VPC and account resources in a security group, the extra security and work are considered to be worth it.

How Testing Changes with Microservices

Integration testing is very error prone and consumes a lot of person-hours, which makes it very costly. Each integration test brings limited coverage of use cases. If you consider all the code path combinations to your application coupled with another application’s code paths, the number of tests written quickly explodes to a number that is unachievable. As a result, managers feel the ROI isn’t there.

As an alternative, we instruct our customers to focus on unit test coverage. A handful of well-selected integration tests can demonstrate the critical points: total failure to essential areas of the application, additional metrics such as SLA/SLO, additional alerting, canary deployments and one-click rollbacks.

Inter-service integration tests defeat the purpose of microservices. If you’re testing how the system works as a whole, then conceptually you’ve done something wrong since these services should be loosely coupled. Essentially, you’ve just created a monolith that’s more complicated to debug.

Instead, consider building a “contingency plan” into the code for when a sub-service fails. For example, Netflix uses a service to predict what you will see in your feed but when that service fails it falls back on a cached version of the feed where no failures exist.

Managing the Biggest Challenges

The most frequent challenges we have come across are dependency management, deployments, monitoring, alerting, SLA/SLO, security, inter-team communication, logging, and debugging practices.

The way our customers solve these problems is to turn operations into an actual “DevOps shop.” Creating a “DevOps shop” is critical because microservices bring overhead in the form of duplication. Building great tooling is the only way to make microservices work.

It is important to start treating the application developers (the users of the tool) like customers. The result of doing so will not only improve the tooling for the internal developers, but it will also motivate your team towards a success metric.

For one of our customers, we measured tooling/service adoption as a way to measure success. Measuring success doesn’t have to be complicated.  Simply asking “are they using the internal tool?” was a good enough metric to drive better internal tooling.

Auditing data sources have simplified since the footprint of code that needs access to a data source gets smaller.

Changing How Teams Work

Instead of communication driving software design, software architecture drives communication. Traditionally, businesses have followed Conway’s Law, which is the inverse of what microservices need. The law states “organizations which design systems are constrained to produce designs which are copies of the communication structures of these organizations.”

Not only do teams operate and organize differently, but it requires a new set of tooling and process to support this type of architecture, i.e., DevOps. As monoliths get broken into smaller components, the need for smaller teams and their roles become more apparent.  Engineers best skilled for (or who have a strong desire to work on) a distinct type of problem can now be assigned to those microservices.

Building and deploying pipelines is very different from managing a monolith, and your team will need to learn how to work in this new environment.

We’ve just touched the surface of these topics, but hopefully, this has given you an excellent place to start.

Recently Published Posts

October 20, 2021
by Jason McIntosh

Monitoring Spinnaker: Part 1

Overview One of the questions that comes up a lot is how you monitor Spinnaker itself.  Not the apps Spinnaker is deploying, but Spinnaker itself and how it’s performing.  This is a question that has a lot of different answers. There are a few guidelines, but many of the answers are the same as how […]

Read more

October 18, 2021
by David Morgenthaler

The Importance of Patents: Interview with Nick Petrella, Head of Legal

    In honor of Armory’s recent acquisition of a patent for continuous software deployment, we sat down with Nick Petrella, Head of Legal, for a casual conversation covering a wide range of subjects, from patent law to Nick’s background as a software engineer and why he made the leap to the law. Check out […]

Read more

October 11, 2021
by Carl Timm

Armory Agent for Kubernetes Simplifies K8s Complexity

I’ll be upfront with you, I’m a sucker for a good origin story. It’s one of the reasons I spent hours engrossed in the Marvel Cinematic Universe not too long ago.  Rooting for incredibly flawed individuals with an outsized sense of duty and superpowers to back it up….What’s not to love?  My partner has a […]

Read more