Git Pull Support in Spinnaker

Aug 31, 2021 by German Muzquiz

Imagine you have a git repository where you store helm charts, kustomization files, and other files that Spinnaker uses to make deployments. Git repositories are usually small in size because only source code is stored in them, right?

The reality is that we often don’t have control over what is stored in a git repository, and they can grow pretty big. Maybe they contain several years of history. All of that makes cloning them a time-consuming operation. In Spinnaker versions prior to 1.26, every time a pipeline with a Bake (Manifest) stage that uses a gitrepo artifact to fetch files to deploy (either helm charts or kustomization files) runs, the following occurs:

  1. The Clouddriver service clones the git repository.
  2. The repo is packaged as a tarball and sent to the Rosco service
  3. The clone is deleted immediately.

Imagine git repositories that are several megabytes in size and the performance hit, leading to slow or failed deployments (because of timeouts).

User-uploaded Image

In Spinnaker 1.26, there’s a new feature that supports git pull behavior, so that the repository only gets cloned the first time (on demand). Subsequent requests for the same artifact just pull the latest changes from the git server. This feature improves performance dramatically for pipelines that reference large git repositories because Clouddriver no longer clones repositories every time it gets referenced. This feature is disabled by default.

To use the feature, you need to provide the following configuration in clouddriver-local.yml (Halyard) or in spec.spinnakerConfig.profiles.clouddriver (Operator):

artifacts:
  git-repo:
    clone-retention-minutes: 10          # (Default: 0). How much time to keep clones. 0: no retention, -1: retain forever.
    clone-retention-max-bytes: 104857600 # (Default: 100 MB). Maximum amount of disk space to use for clones.
    clone-wait-lock-timeout-sec: 60      # (Default: 60). How long to wait for other threads to unlock the clone if several threads try to access it at the same time.
    clone-retention-check-ms: 60000      # (Default: 60000). How often to check if there are expired cloned that should be deleted.

Internally, clones are stored under /tmp with a folder name created by the SHA256 hash of this string: <repository-url>:<branch>. This makes it possible to locate existing clones for the same artifact since the name is composed of the URL and branch. If the branch is changed to something different, a new clone with a new hash gets created the first time that repo is used.

2021-08-17 17:04:59.022 DEBUG 1 --- [      MvcAsync3] c.n.s.c.a.gitRepo.GitRepoFileSystem      : Trying filesystem timed lock for https://github.com/german-muzquiz/spinnaker-scratchpad.git (branch master), hash: 52d32666691e5ed374c4672bb46f95b528c27cbc2f8a37010ba919090c16004c for 60 seconds
2021-08-17 17:04:59.023 DEBUG 1 --- [      MvcAsync3] c.n.s.c.a.gitRepo.GitRepoFileSystem      : Lock  acquired for https://github.com/german-muzquiz/spinnaker-scratchpad.git (branch master), hash 52d32666691e5ed374c4672bb46f95b528c27cbc2f8a37010ba919090c16004c, lock instance: [email protected][Locked by thread MvcAsync3]
2021-08-17 17:04:59.023 DEBUG 1 --- [      MvcAsync3] c.n.s.c.a.gitRepo.GitJobExecutor         : Executing command: "sh -c git symbolic-ref HEAD"
2021-08-17 17:04:59.032  INFO 1 --- [      MvcAsync3] c.n.s.c.a.gitRepo.GitJobExecutor         : Pulling git/repo https://github.com/german-muzquiz/spinnaker-scratchpad.git into /tmp/gitrepos/52d32666691e5ed374c4672bb46f95b528c27cbc2f8a37010ba919090c16004c/spinnaker-scratchpad
2021-08-17 17:04:59.033 DEBUG 1 --- [      MvcAsync3] c.n.s.c.a.gitRepo.GitJobExecutor         : Executing command: "sh -c git pull"
2021-08-17 17:04:59.773  INFO 1 --- [      MvcAsync3] c.n.s.c.a.g.GitRepoArtifactCredentials   : Creating archive for git/repo https://github.com/german-muzquiz/spinnaker-scratchpad.git
2021-08-17 17:04:59.773 DEBUG 1 --- [      MvcAsync3] c.n.s.c.a.gitRepo.GitJobExecutor         : Executing command: "sh -c git archive --format tgz --output /tmp/gitrepos/52d32666691e5ed374c4672bb46f95b528c27cbc2f8a37010ba919090c16004c/spinnaker-scratchpad.tgz master kustomize"
2021-08-17 17:04:59.782 DEBUG 1 --- [      MvcAsync3] c.n.s.c.a.gitRepo.GitRepoFileSystem      : Unlocking filesystem for https://github.com/german-muzquiz/spinnaker-scratchpad.git (branch master), hash: 52d32666691e5ed374c4672bb46f95b528c27cbc2f8a37010ba919090c16004c
2021-08-17 17:04:59.783 DEBUG 1 --- [      MvcAsync3] c.n.s.c.a.gitRepo.GitRepoFileSystem      : Unlocking filesystem for hash 52d32666691e5ed374c4672bb46f95b528c27cbc2f8a37010ba919090c16004c, lock instance: [email protected][Locked by thread MvcAsync3]

Recently Published Posts

A Faster Way to Evaluate Self-Hosted Continuous Deployment from Armory

Sep 30, 2022

Introducing Quick Spin One of the most common challenges that organizations face when implementing a continuous deployment strategy is the time and focus that it takes to set up the tools and processes. But a secure, flexible, resilient and scalable solution is available right now. Want to see if it’s the right tool for your […]

Read more

3 Common Spinnaker Challenges (and Easy Ways to Solve Them)

Sep 27, 2022

Spinnaker is the most powerful continuous delivery tool on the market.  DevOps engineers and developers recognize this power and are looking to use Spinnaker as a foundational tool in their Continuous Integration and Continuous Delivery (CI/CD) process for hybrid and multi-cloud deployments. Such a powerful, expansive open source tool needs expertise within your organization to […]

Read more

Streamline Advanced Kubernetes Deployments from GitHub Actions with New Armory Service

Sep 23, 2022

Today, Armory is excited to announce the availability of the GitHub Action for Armory Continuous Deployment-as-a-Service. GitHub is where developers shape the future of software. After a developer writes and tests their code in GitHub, it must be deployed. Armory’s GitHub Action for Continuous Deployment-as-a-Service extends the best-in-class deployment capabilities to Kubernetes. CD-as-a-Service enables declarative […]

Read more