Are You Logging Your Deployments Correctly?

May 17, 2017 by Armory

So our CTO, Isaac Mosquera, grabbed me and said, “Hey. We should talk about auditing trails.”

My reaction was: “What? We need to talk about this?!”

It appears that talking about being organized and accountable in your organization is one of those topics that people love to talk about but have difficulty committing to (shipping small diffs is another culprit). As companies migrate to the cloud for all the benefits it comes with, they are forgetting to lay down strong groundwork and infrastructure for gathering information regarding their own activities, a necessary step for responsibility and accountability. This becomes an unfortunate source of management debt with some technical debt that companies accrue due to their desire for velocity, which in turn becomes a strangling bottleneck as the lack of standardization kills efficiency.

Current companies generally rely on some form of ticket tracker (like JIRA) as a form of auditing trail; the thinking may be along the lines of “Well, if everyone’s logging their activities with tickets then we should have no problem, right?” However, this method relies on human diligence, and humans are prone to making mistakes. These mistakes can be anything from incorrectly reporting what was done, forgetting to report what was done, or even worse: compromising security with clandestine activity, something Uber and Waymo is currently in a legal battle over.

“It’s not a good system of record, because JIRA has nothing to do with deployments,” says Isaac.

The additional dependency on 3rd-party tools like JIRA for tracking tickets and changes involved means that your organization’s auditing trail lives and dies on the dependability of these tools (not to mention, the security). At best, this seems to be a fragile, temporary measure that will need to be addressed as management debt accumulates, introducing unnecessary overhead with each line of code that is committed or erased. JIRA tickets treat small and large tasks in an equal manner and further complicates the auditing trail when organizations conduct reviews of their logs.

Occasionally companies do something worse: they cut off developer access to production, and gate the access through specific engineers that are tasked with deploying to production. This makes superficial sense because the responsibility becomes centralized to several people, but makes no sense in the pursuit of speedy deployments and faster iteration because you’ve introduced a bottleneck. Additionally, this method introduces reliance and dependency on those people; if they have a sick day, quit, or go on vacation, deploying to production becomes halted.

But Spinnaker solves the root of these problems right out of the box, with an auditing trail easily set up for concise logging of Who, What, When, Where, and How in JSON format. Spinnaker is a tool that can be used for deploying software to multiple deployment targets already, and so engineers won’t even need to switch UIs or worry about making sure the new process is being logged. By using Spinnaker’s sub-service Echo you can literally set up an auditing trail in seconds in any secure environment your administrator wants. Since the auditing logs are in an easily searchable JSON format and do not contain security-sensitive information like passwords, a company can easily present the logs to auditors for quick and easy auditing. Additionally, if your company is used to using tools like Splunk for auditing, the JSON format can be easily imported without hassle.

Here is an example of the JSON format and Event logging that Spinnaker produces:

{ "details": { "source": "orca", "type": "orca:task:complete", "created": "1495056070817", "organization": "armory-io", "project": null, "application": "armoryspinnaker", "_content_id": null }, "content": { "standalone": true, "context": { "asgName": "armoryspinnaker-prod-v001", "credentials": "prod-aws-account", "deploy.account.name": "prod-aws-account", "deploy.server.groups": {}, "kato.last.task.id": { "id": "10203" }, "kato.task.id": { "id": "10203" }, "kato.tasks": [ { "history": [ ], "id": "10203", "resultObjects": [] } ], "notification.type": "enableasg", "regions": ["us-west-2"], "targetop.asg.enableAsg.name": "armoryspinnaker-prod-v001", "targetop.asg.enableAsg.regions": ["us-west-2"], "user": "[email protected]", "zones": ["us-west-2a", "us-west-2b", "us-west-2c"] }, "executionId": "a7ca5574-4629-119a-c9ac-ga873aa165b2", "taskName": "a72239a7-c57a-308d-9d72-1a77484e050c.enableAsg.monitorAsg.758e6e5-3c37-4599-9e93-g62118adc7c6" } }

Responsibility and Accountability are important no matter how big or small your organization is.

We’re not attempting to introduce the concept of auditing trails but the current evolution of auditing trails as companies move into the cloud – especially with regards to the current ecosystem of companies caught mid-transition with wrangled-together methods of logging their deployments. Everyone talks about having good audit trails, but in the end it’s always a good idea to check within your organization and ensure you’re practicing what you preach.

Learn More

Recently Published Posts

3 Common Spinnaker Challenges (and Easy Ways to Solve Them)

Sep 27, 2022

Spinnaker is the most powerful continuous delivery tool on the market.  DevOps engineers and developers recognize this power and are looking to use Spinnaker as a foundational tool in their Continuous Integration and Continuous Delivery (CI/CD) process for hybrid and multi-cloud deployments. Such a powerful, expansive open source tool needs expertise within your organization to […]

Read more

Streamline Advanced Kubernetes Deployments from GitHub Actions with New Armory Service

Sep 23, 2022

Today, Armory is excited to announce the availability of the GitHub Action for Armory Continuous Deployment-as-a-Service. GitHub is where developers shape the future of software. After a developer writes and tests their code in GitHub, it must be deployed. Armory’s GitHub Action for Continuous Deployment-as-a-Service extends the best-in-class deployment capabilities to Kubernetes. CD-as-a-Service enables declarative […]

Read more

When everyone is facing the same headwind, go on the offensive

Sep 12, 2022

Call me Pollyanna, but what a great time to be a Platform or DevOps engineer. If you’re working in a public company, the S&P is off ~20% year over year, so the value of your RSUs has wilted. If you’re working in a private company, venture funding and M&A velocity are anemic, making expansion capital […]

Read more