Editing article

Title

Summary

Content

<h3>Design your Landing Zone — Design Considerations Part 4— IaC, GitOps and CI/CD (Google Cloud Adoption Series)</h3>Welcome to LZ Design Considerations Part 4, where we’ll wrap up the Landing Zone Design Considerations. This is part of the <a href="https://medium.com/google-cloud/google-cloud-adoption-for-the-enterprise-from-strategy-to-operation-part-0-overview-9091f5a1ddfc">Google Cloud Adoption and Migration: From Strategy to Operation</a> series.Previously, we covered <a href="https://medium.com/google-cloud/design-your-landing-zone-design-considerations-part-3-monitoring-logging-billing-and-7b40189a3c81">LZ design decisions relating to monitoring, logging, billing and labelling</a>. In this part, we’ll focus on all things related to automated infra and application deployment, using IaC, GitOps, and CI/CD.<h3>12. IaC, GitOps and CI/CD Strategy</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/677/1*y6Mt0Sgz5pxQ_nwHTvlN1g.png" /><figcaption>Darren’s quote of the day</figcaption></figure><h4>Principles Recap</h4>I recommended a number of principles in a previous article, called <a href="https://medium.com/google-cloud/cloud-adoption-and-cloud-engineering-principles-google-cloud-adoption-part-3-660bdb78cebb">Cloud Adoption and Cloud Consumption Principles</a>. I want to recap a couple here:<ul><li>Automate deployments and installs</li><li>Immutable infrastructure</li></ul>Let me review the rationale for these. In our legacy on-prem world, we had:<ul><li>Fewer, larger machines.</li><li>Machines tended to built once, and rarely rebuilt.</li><li>VMs needed to be looked after. They were treated as pets.</li><li>We had limited scale and limited elasticity.</li><li>We had to care about the underlying physical hardware.</li></ul>But now, in Cloud:<ul><li>We have many, smaller machines.</li><li>VMs, applications and services tend to be rebuilt frequently. It’s more effective to replace poorly services, than to try to fix them. We treat them as cattle.</li><li>Applications are designed to be fault-tolerant.</li><li>We have virtually unlimited scale, and most services are extremely elastic.</li><li>We want services to scale down (or be turned off) when not in use.</li><li>For the most part, we don’t care about the underlying hardware.</li></ul>So, we want our LZ to align to these principles. We need an automated, repeatable way to build immutable, replaceable infrastructure.<h4>Infrastructure-as-Code (IaC) to the Rescue</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*N--Ib58wG_NpNZtm" /><figcaption>IaC, creating infra resources in the Cloud</figcaption></figure>IaC is about automated provisioning of infrastructure resources, using code. It allows us to rapidly provision (and tear down) infrastructure environments in a repeatable, consistent way.Some key tenets of IaC:<ul><li>All the infrastructure provisioning and dependencies are defined in code.</li><li>The code should be stored in source control, such as GitHub. This way it is managed, versioned, and supports collaboration.</li><li>Our code can be imperative — i.e. follow a set of steps to achieve the outcome. Or our code can be declarative — i.e. “here’s the outcome I need, now you work out how to do it.” Declarative is best!</li><li>It is easily deployed as part of an automated CI/CD pipeline.</li><li>It is idempotent — meaning that we can always repeat our deployment, regardless of the current state, and end up with the state that we wanted.</li><li>We can use it to build multiple environments, and we can be sure they look the same. We can always pass in parameters, to apply environment-specific configuration.</li><li>We can use it to deploy DR environments on-demand. (If that is our chosen DR strategy.)</li><li>The code is self-documenting. (And supports comments.) This means that an infrastructure engineer can look at our IaC, and understand what it will do. As such, it provides the implementation of our high level design, and eliminates the need for a significant amount of low level design documentation. Why? Because the IaC is the low level design documentation, for the cloud infrastructure.</li><li>It eliminates configuration drift — not only between the HLD and the deployed environments, but between the environments themselves.</li><li>It eliminates human errors. We don’t have human operators building stuff manually, or tweaking stuff in individual environments.</li></ul><h4>Some Tips and Best Practices for IaC</h4>Here’s the thing… If you’re not using IaC, you’re doing Cloud wrong. So here are a few key takeaways:<ul><li>For initial resource deployment in a Dev environment, you can provision resources using the Google Cloud Console. But when you build any cloud infra resources in any other environments (e.g. UAT, OAT, Staging, Prod, whatever), your should be doing so with IaC. This way, you’re using the same code to deploy to all environments.</li><li>Ensure that your LZ project factory provides service accounts to your tenants, and use those service accounts to actually deploy the resources.</li><li>Don’t allow manual (human) infrastructure tweaking in any environments other than Dev. You can enforce this through policy and IAM. If you allow operators to tweak configuration by hand then you will get configuration drift, and you’ll kill your automation and repeatability benefit.</li><li>If you have any engineers or integrators that say things like, “Let’s just build it manually, and worry about the IaC later” then educate them, or get rid of them. This sort of legacy on-prem thinking kills your cloud agility. I’ve been on projects where system integrators have refused to build the IaC upfront. And the detrimental impact on the project is staggering!</li><li>Create a CI/CD pipeline to automate your IaC deployments.</li></ul><h4>The GitOps Approach</h4>Building our infrastructure using IaC is a good start. But we need to build a CI/CD pipeline, such that IaC changes can be automatically deployed to our target environment. Google advocates for the use of GitOps, which requires that:<ul><li>All resources are deployed using declarative code.</li><li>Our code is hosted in a Git repository.</li><li>All operational changes are made by developers who make a pull request.</li><li>Merging of the pull request results in execution of a build and release pipeline.</li></ul>Here’s a sketch of the overall GitOps pipeline, along with some products and tools you might use at each stage:<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ANWEEAo6sXrznpDVvMEFAg.png" /><figcaption>GitOps Pipeline</figcaption></figure>In the context of deploying cloud infrastructure, our declarative code will be in the form of IaC. Google’s documentation shows this reference example:<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9srXojWGJMB5Hm1oMK4lQQ.png" /><figcaption>Google’s reference architecture for GitOps</figcaption></figure>In this example:<ul><li>Our IaC is written in Terraform. Terraform is an open source cloud-agnostic IaC tool that uses a declarative IaC language.</li><li>We use GitHub to store our Git repo. (But we could use other Git hosting services like GitLab or BitBucket. If we want to stay fully within the Google Cloud ecosystem, we can also use <a href="http://Google Cloud Source Repositories">Google Cloud Source Repositories</a>, CSR. We can use CSR to host our master Git repos; but we could also synchronise our CSR repos from an upstream repo, like GitHub.)</li><li>Infra developers push IaC changes into a feature branch, triggering Google Cloud Build to execute terraform plan. This results in a Terraform manifest, but does not actually apply it. (We could use an alternative tool for executing terraform. For example, if we’re using GitHub for our Git repos, we could use GitHub Actions to trigger terraform.)</li><li>Then the developer raises a pull request for the dev branch. When it is merged, Cloud Build executes terraform apply, thus deploying our Terraform manifest to the dev environment.</li><li>Once the dev build has been validated, the changes are merged into prod, causing the Terraform manifest to be deployed to the prod environment.</li></ul><h4>Pipeline Layers</h4>Google recommends separate pipeline layers, with different teams responsible for each. For example:<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*e6FxEv3FJai3-WkWf5g9Fw.png" /><figcaption>Pipeline layers</figcaption></figure>Here:<ul><li>The foundation pipeline deploys the foundation resources that make up the LZ. This pipeline will typically be the responsibility of a single Platform Team.</li><li>The infrastructure pipeline deploys infrastructure that is used by individual tenants and applications. The pipeline can only be executed by a tenant service account, and this service account can only deploy to resources under this tenant’s folder. Google has example code for creating an application infrastructure pipeline in the GitHub Terraform Example Foundation repo, <a href="https://github.com/terraform-google-modules/terraform-example-foundation/tree/master/4-projects">here</a>.</li><li>The application pipeline deploys application resources, such as images, and GKE application resources.</li></ul><h4>Summary of IaC and GitOps Design Decisions</h4><ul><li>Which IaC tool? I would recommend Terraform, unless you have a compelling reason not to. Terraform is declarative, open source, is cloud agnostic, and works in the enterprise. It is also Google’s recommended tool for IaC with Google Cloud.</li><li>Assuming you’re using Terraform, where will you persist your Terraform state? In the enterprise, you should be storing Terraform state in a remote backend that supports collaborative working, automatic state locking, and granular access control. In the Google Cloud ecosystem, Google GCS is a great choice. But other options include Terraform Cloud (Terraform SaaS), and Terraform Enterprise (self-hosted).</li><li>Which source code platform? E.g. GitHub, GitLab, Google Cloud Source Repos, etc.</li><li>What will be your git branching strategy? Google recommends having a protected main branch, feature and bug-fix branches, plus a separate persistent branch for each environment. This way, changes can be promoted through the environments by merging the changes between the environment branches.</li><li>How will you separate these environments in your IaC repo? Google recommends using a separate folder in the repo for each environment. Each folder will map to a branch, and each branch will deploy to a specific environment.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/789/1*CHnBud_J5muryzzRMLm74w.png" /><figcaption>Separating environments in your repo and in branches</figcaption></figure><ul><li>How many environments will you manage? E.g. dev, uat, staging and prod?</li><li>What Google Cloud resource naming conventions will you adopt? You need to document your naming standards. But before you invent your own, Google has a set of recommended naming conventions <a href="https://cloud.google.com/architecture/security-foundations/summary#naming-conventions">here</a>.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0DBrkSxFyd3cst32asUw3g.png" /><figcaption>Google’s recommended resource naming conventions</figcaption></figure><ul><li>Will you use IaC to deploy your landing zone? If so, will you use an existing IaC LZ blueprint, or create your own? Google provides a couple of open source organisation LZ blueprints and implementations which can be used to rapidly accelerate your LZ deployment. These are <a href="https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/tree/master/fast">Google Cloud Foundation Fabric FAST</a> and <a href="https://github.com/terraform-google-modules/terraform-example-foundation">Cloud Foundation Toolkit (CFT) Terraform Example Foundation</a>. Google Cloud Foundation Fabric FAST is intended to be a pre-composed end-to-end example, which is forked, cloned and modified as required. Whereas CFT is intended to be used a library of opinionated Terraform modules which should be composed as required. Google describes the differences between these two approaches <a href="https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/blob/master/FABRIC-AND-CFT.md">here</a>.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*wlX4xPvM5yR0TFaQ.png" /><figcaption>Google’s Enterprise LZ IaC Accelerator — Google Cloud Foundation Fabric FAST</figcaption></figure><ul><li>How will you organise and manage access to IaC repos? Google recommends using the design principle that configurations with different approval and management requirements are separated into different source control repositories. For example, a central platform team may be responsible for the LZ IaC, shared resources, and the <a href="https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/tree/master/modules/project-factory">tenant factory</a>. Whereas application teams may be responsible for all infrastructure resources deployed within their own folders. This approach — making use of pipeline layers — is recommended in enterprises, as it delegates control to application teams.</li><li>What CI/CD tools will you use in your GitOps pipeline. For example, you might choose Google Cloud Build for seamless integration with Google Cloud, if Google Cloud is the only infrastructure target of your IaC. Alternatively, if you’re already using GitHub and want more cloud agnosticity, then GitHub Actions might be a good choice.</li><li>How will tenants execute their IaC? Best practice is to only allow tenants to execute IaC using provided tenant service accounts.</li><li>IaC standards and best practices? Establish and document your organisation’s IaC and Terraform standards and best practices. And don’t reinvent the wheel. Google has <a href="https://cloud.google.com/docs/terraform/best-practices-for-terraform">great guidance</a> on this already.</li><li>How will you enforce IaC policies and standards? Since all your cloud infrastructure will be deployed using IaC, it is important to ensure that the IaC you execute adheres to your organisation’s policies. For example, you might want to prevent deployment of certain resources, enforce use of labels with a limited set of values, or enforce customer-managed encryption keys on GCS buckets. Consider using an automated policy validation tool, such as Hashicorp Sentinel (but only if you’re using a Terraform Cloud or Terraform Enterprise backend), Terratest (which is open source), or Google’s gcloud terraform vet.</li></ul><h3>Wrap-Up</h3>After four articles on the topic of Google Cloud LZ design considerations, I think you’ll agree that the design phase is not entirely trivial! There are a lot of considerations, and many implications of your choices!In the next part, I’ll show you how to go about making informed decisions. I’ll guide you through the LZ design process, show you how to capture your decisions, and tell you how to get the help you need, so you don’t make any troublesome mistakes!<h3>Before You Go</h3><ul><li>Please share this with anyone that you think will be interested. It might help them, and it really helps me!</li><li>Feel free to leave a comment 💬.</li><li>Follow and subscribe, so you don’t miss my content. Go to my <a href="https://medium.com/@derailed.dash">Profile Page</a>, and click on these icons:</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/163/0*fF62z2-FT03ui0O5.png" /><figcaption>Follow and Subscribe</figcaption></figure><h3>Links</h3><ul><li><a href="https://medium.com/google-cloud/landing-zones-on-google-cloud-b42b08e1abaa">Landing Zones on Google Cloud: What It Is, Why You Need One, and How to Create One</a></li><li><a href="https://cloud.google.com/architecture/landing-zones">Landing zone design in Google Cloud</a></li><li><a href="https://cloud.google.com/docs/terraform/resource-management/managing-infrastructure-as-code">Managing infrastructure-as-code with Terraform, Cloud Build, and GitOps</a></li><li><a href="https://cloud.google.com/docs/terraform">Terraform on Google Cloud</a></li><li><a href="https://cloud.google.com/docs/terraform/best-practices-for-terraform">Best practices for Terraform</a></li><li><a href="https://cloud.google.com/build/docs">Google Cloud Build</a></li><li><a href="https://cloud.google.com/architecture/framework/operational-excellence/automate-your-deployments">Automate your deployments</a></li><li><a href="https://cloud.google.com/architecture/security-foundations/deployment-methodology">Deployment methodology</a></li><li><a href="https://cloud.google.com/source-repositories/docs/features">Google Cloud Source Repositories</a></li><li><a href="https://cloud.google.com/docs/terraform/policy-validation">Google Cloud Terraform policy validation with terraform vet</a></li><li><a href="https://cloud.google.com/docs/terraform/policy-validation/create-policy-library">Google Cloud: creating a Terraform policy library</a></li><li><a href="https://cloud.google.com/architecture/security-foundations/summary#naming-conventions">Google Cloud Resource Naming Standards</a></li><li><a href="https://github.com/terraform-google-modules/terraform-example-foundation">Google CFT Terraform Example Foundation</a></li><li><a href="https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/tree/master/fast">Google Cloud Foundation Fabric FAST</a></li><li><a href="https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/blob/master/FABRIC-AND-CFT.md">Cloud Foundation Fabric FAST vs CFT</a></li><li><a href="https://medium.com/google-cloud/resource-factories-a-descriptive-approach-to-terraform-581b3ebb59c">Resource factories</a></li><li><a href="https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/tree/master/blueprints/factories">Resource factories in Cloud Foundation Fabric FAST</a></li><li><a href="https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/tree/master/modules/project-factory">Cloud Foundation Fabric FAST: project and folder factory</a></li><li><a href="https://registry.terraform.io/modules/terraform-google-modules/project-factory/google/latest">Terraform Google Cloud Project-Factory</a></li><li><a href="https://cloud.google.com/architecture/framework">Google Cloud Architecture Framework</a></li><li><a href="https://cloud.google.com/architecture/security-foundations">Enterprise Foundations Blueprint</a></li></ul><h3>Series Navigation</h3><ul><li><a href="https://medium.com/google-cloud/google-cloud-adoption-for-the-enterprise-from-strategy-to-operation-part-0-overview-9091f5a1ddfc">Series overview and structure</a></li><li>Previous: <a href="https://medium.com/google-cloud/design-your-landing-zone-design-considerations-part-3-monitoring-logging-billing-and-7b40189a3c81">Design your Landing Zone — Design Considerations Part 3: Monitoring, Logging, Billing and Labelling</a></li><li>Next: Design your Landing Zone — How To</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ae3f533c6dbd" width="1" height="1" alt=""><hr><a href="https://medium.com/google-cloud/design-your-landing-zone-design-considerations-part-4-iac-gitops-and-ci-cd-google-cloud-ae3f533c6dbd">Design your Landing Zone — Design Considerations Part 4— IaC, GitOps and CI/CD (Google Cloud…</a> was originally published in <a href="https://medium.com/google-cloud">Google Cloud - Community</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.

Author

Link

Published date

Image url

Feed url

Guid

Hidden blurb

--- !ruby/object:Feedjira::Parser::RSSEntry
title: Design your Landing Zone — Design Considerations Part 4— IaC, GitOps and CI/CD
 (Google Cloud…
url: https://medium.com/google-cloud/design-your-landing-zone-design-considerations-part-4-iac-gitops-and-ci-cd-google-cloud-ae3f533c6dbd?source=rss----e52cf94d98af---4
author: Dazbo (Darren Lester)
categories:
- gitops
- landingzone
- infrastructure-as-code
- google-cloud-platform
- cloud-foundation
published: 2024-04-01 11:51:00.000000000 Z
entry_id: !ruby/object:Feedjira::Parser::GloballyUniqueIdentifier
 is_perma_link: 'false'
 guid: https://medium.com/p/ae3f533c6dbd
carlessian_info:
 news_filer_version: 2
 newspaper: Google Cloud - Medium
 macro_region: Blogs
rss_fields:
- title
- url
- author
- categories
- published
- entry_id
- content
content: "<h3>Design your Landing Zone — Design Considerations Part 4— IaC, GitOps
 and CI/CD (Google Cloud Adoption Series)</h3>Welcome to LZ Design Considerations
 Part 4, where we’ll wrap up the Landing Zone Design Considerations. This is part
 of the <a href=\"https://medium.com/google-cloud/google-cloud-adoption-for-the-enterprise-from-strategy-to-operation-part-0-overview-9091f5a1ddfc\">Google
 Cloud Adoption and Migration: From Strategy to Operation</a> series.Previously,
 we covered <a href=\"https://medium.com/google-cloud/design-your-landing-zone-design-considerations-part-3-monitoring-logging-billing-and-7b40189a3c81\">LZ
 design decisions relating to monitoring, logging, billing and labelling</a>. In
 this part, we’ll focus on all things related to automated infra and application
 deployment, using IaC, GitOps, and CI/CD.<h3>12. IaC, GitOps and CI/CD Strategy</h3><figure><img
 alt=\"\" src=\"https://cdn-images-1.medium.com/max/677/1*y6Mt0Sgz5pxQ_nwHTvlN1g.png\"
 /><figcaption>Darren’s quote of the day</figcaption></figure><h4>Principles Recap</h4>I
 recommended a number of principles in a previous article, called <a href=\"https://medium.com/google-cloud/cloud-adoption-and-cloud-engineering-principles-google-cloud-adoption-part-3-660bdb78cebb\">Cloud
 Adoption and Cloud Consumption Principles</a>. I want to recap a couple here:<ul><li>Automate
 deployments and installs</li><li>Immutable infrastructure</li></ul>Let
 me review the rationale for these. In our legacy on-prem world, we had:<ul><li>Fewer,
 larger machines.</li><li>Machines tended to built once, and rarely rebuilt.</li><li>VMs
 needed to be looked after. They were treated as pets.</li><li>We had limited
 scale and limited elasticity.</li><li>We had to care about the underlying physical
 hardware.</li></ul>But now, in Cloud:<ul><li>We have many, smaller machines.</li><li>VMs,
 applications and services tend to be rebuilt frequently. It’s more effective to
 replace poorly services, than to try to fix them. We treat them as cattle.</li><li>Applications
 are designed to be fault-tolerant.</li><li>We have virtually unlimited scale, and
 most services are extremely elastic.</li><li>We want services to scale down (or
 be turned off) when not in use.</li><li>For the most part, we don’t care about the
 underlying hardware.</li></ul>So, we want our LZ to align to these principles.
 We need an automated, repeatable way to build immutable, replaceable infrastructure.<h4>Infrastructure-as-Code
 (IaC) to the Rescue</h4><figure><img alt=\"\" src=\"https://cdn-images-1.medium.com/max/1024/0*N--Ib58wG_NpNZtm\"
 /><figcaption>IaC, creating infra resources in the Cloud</figcaption></figure>IaC
 is about automated provisioning of infrastructure resources, using code.
 It allows us to rapidly provision (and tear down) infrastructure environments in
 a repeatable, consistent way.Some key tenets of IaC:<ul><li>All
 the infrastructure provisioning and dependencies are defined in code.</li><li>The
 code should be stored in source control, such as GitHub. This way
 it is managed, versioned, and supports collaboration.</li><li>Our code can be imperative — i.e.
 follow a set of steps to achieve the outcome. Or our code can be declarative — i.e.
 “here’s the outcome I need, now you work out how to do it.” Declarative
 is best!</li><li>It is easily deployed as part of an automated CI/CD
 pipeline.</li><li>It is idempotent — meaning that we
 can always repeat our deployment, regardless of the current state, and
 end up with the state that we wanted.</li><li>We can use it to build multiple
 environments, and we can be sure they look the same. We can always pass
 in parameters, to apply environment-specific configuration.</li><li>We can use it
 to deploy DR environments on-demand. (If that is our chosen DR
 strategy.)</li><li>The code is self-documenting. (And supports
 comments.) This means that an infrastructure engineer can look at our IaC, and understand
 what it will do. As such, it provides the implementation of our high level design,
 and eliminates the need for a significant amount of low level design documentation.
 Why? Because the IaC is the low level design documentation, for the cloud
 infrastructure.</li><li>It eliminates configuration drift — not
 only between the HLD and the deployed environments, but between the environments
 themselves.</li><li>It eliminates human errors. We don’t have human
 operators building stuff manually, or tweaking stuff in individual environments.</li></ul><h4>Some
 Tips and Best Practices for IaC</h4>Here’s the thing… If you’re not using IaC,
 you’re doing Cloud wrong. So here are a few key takeaways:<ul><li>For initial
 resource deployment in a Dev environment, you can provision resources using the
 Google Cloud Console. But when you build any cloud infra resources
 in any other environments (e.g. UAT, OAT, Staging, Prod, whatever), your should
 be doing so with IaC. This way, you’re using the same code to deploy to
 all environments.</li><li>Ensure that your LZ project factory provides service
 accounts to your tenants, and use those service accounts to actually deploy
 the resources.</li><li>Don’t allow manual (human) infrastructure tweaking
 in any environments other than Dev. You can enforce this through policy
 and IAM. If you allow operators to tweak configuration by hand then you will get
 configuration drift, and you’ll kill your automation and repeatability benefit.</li><li>If
 you have any engineers or integrators that say things like, “Let’s just build
 it manually, and worry about the IaC later” then educate them, or get rid of
 them. This sort of legacy on-prem thinking kills your cloud agility. I’ve been on
 projects where system integrators have refused to build the IaC upfront. And the
 detrimental impact on the project is staggering!</li><li>Create a CI/CD
 pipeline to automate your IaC deployments.</li></ul><h4>The GitOps Approach</h4>Building
 our infrastructure using IaC is a good start. But we need to build a CI/CD pipeline,
 such that IaC changes can be automatically deployed to our target environment. Google
 advocates for the use of GitOps, which requires that:<ul><li>All resources
 are deployed using declarative code.</li><li>Our code is
 hosted in a Git repository.</li><li>All operational changes are made by
 developers who make a pull request.</li><li>Merging of the pull request results
 in execution of a build and release pipeline.</li></ul>Here’s a sketch of the
 overall GitOps pipeline, along with some products and tools you might use at each stage:<figure><img
 alt=\"\" src=\"https://cdn-images-1.medium.com/max/1024/1*ANWEEAo6sXrznpDVvMEFAg.png\"
 /><figcaption>GitOps Pipeline</figcaption></figure>In the context of deploying
 cloud infrastructure, our declarative code will be in the form of IaC. Google’s
 documentation shows this reference example:<figure><img alt=\"\" src=\"https://cdn-images-1.medium.com/max/1024/1*9srXojWGJMB5Hm1oMK4lQQ.png\"
 /><figcaption>Google’s reference architecture for GitOps</figcaption></figure>In
 this example:<ul><li>Our IaC is written in Terraform. Terraform
 is an open source cloud-agnostic IaC tool that uses a declarative IaC language.</li><li>We
 use GitHub to store our Git repo. (But we could use other Git hosting
 services like GitLab or BitBucket. If we want to stay fully within the Google Cloud
 ecosystem, we can also use <a href=\"http://Google Cloud Source Repositories\">Google
 Cloud Source Repositories</a>, CSR. We can use CSR to host our master Git repos;
 but we could also synchronise our CSR repos from an upstream repo, like GitHub.)</li><li>Infra
 developers push IaC changes into a feature branch, triggering Google Cloud
 Build to execute terraform plan. This results in a Terraform manifest,
 but does not actually apply it. (We could use an alternative tool for executing
 terraform. For example, if we’re using GitHub for our Git repos, we could use GitHub
 Actions to trigger terraform.)</li><li>Then the developer raises a pull
 request for the dev branch. When it is merged, Cloud Build
 executes terraform apply, thus deploying our Terraform manifest to the dev environment.</li><li>Once
 the dev build has been validated, the changes are merged into prod, causing the
 Terraform manifest to be deployed to the prod environment.</li></ul><h4>Pipeline
 Layers</h4>Google recommends separate pipeline layers, with different teams responsible
 for each. For example:<figure><img alt=\"\" src=\"https://cdn-images-1.medium.com/max/1024/1*e6FxEv3FJai3-WkWf5g9Fw.png\"
 /><figcaption>Pipeline layers</figcaption></figure>Here:<ul><li>The foundation
 pipeline deploys the foundation resources that make up the
 LZ. This pipeline will typically be the responsibility of a single Platform Team.</li><li>The
 infrastructure pipeline deploys infrastructure that is
 used by individual tenants and applications. The pipeline can only be executed by
 a tenant service account, and this service account can only deploy to resources
 under this tenant’s folder. Google has example code for creating an application
 infrastructure pipeline in the GitHub Terraform Example Foundation repo, <a href=\"https://github.com/terraform-google-modules/terraform-example-foundation/tree/master/4-projects\">here</a>.</li><li>The
 application pipeline deploys application resources, such
 as images, and GKE application resources.</li></ul><h4>Summary of IaC and GitOps
 Design Decisions</h4><ul><li>Which IaC tool? I would recommend
 Terraform, unless you have a compelling reason not to. Terraform is declarative,
 open source, is cloud agnostic, and works in the enterprise. It is also Google’s
 recommended tool for IaC with Google Cloud.</li><li>Assuming you’re using Terraform,
 where will you persist your Terraform state? In the enterprise,
 you should be storing Terraform state in a remote backend that supports collaborative
 working, automatic state locking, and granular access control. In the Google Cloud
 ecosystem, Google GCS is a great choice. But other options include
 Terraform Cloud (Terraform SaaS), and Terraform Enterprise (self-hosted).</li><li>Which
 source code platform? E.g. GitHub, GitLab, Google Cloud Source Repos, etc.</li><li>What
 will be your git branching strategy? Google recommends having a protected
 main branch, feature and bug-fix branches, plus a separate persistent branch
 for each environment. This way, changes can be promoted through the environments
 by merging the changes between the environment branches.</li><li>How will
 you separate these environments in your IaC repo? Google recommends using
 a separate folder in the repo for each environment. Each folder will map to a branch,
 and each branch will deploy to a specific environment.</li></ul><figure><img alt=\"\"
 src=\"https://cdn-images-1.medium.com/max/789/1*CHnBud_J5muryzzRMLm74w.png\" /><figcaption>Separating
 environments in your repo and in branches</figcaption></figure><ul><li>How
 many environments will you manage? E.g. dev, uat, staging and prod?</li><li>What
 Google Cloud resource naming conventions will you adopt? You need to document
 your naming standards. But before you invent your own, Google has a set of recommended
 naming conventions <a href=\"https://cloud.google.com/architecture/security-foundations/summary#naming-conventions\">here</a>.</li></ul><figure><img
 alt=\"\" src=\"https://cdn-images-1.medium.com/max/1024/1*0DBrkSxFyd3cst32asUw3g.png\"
 /><figcaption>Google’s recommended resource naming conventions</figcaption></figure><ul><li>Will
 you use IaC to deploy your landing zone? If so, will you use an existing
 IaC LZ blueprint, or create your own? Google provides a couple of open source organisation
 LZ blueprints and implementations which can be used to rapidly accelerate
 your LZ deployment. These are <a href=\"https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/tree/master/fast\">Google
 Cloud Foundation Fabric FAST</a> and <a href=\"https://github.com/terraform-google-modules/terraform-example-foundation\">Cloud
 Foundation Toolkit (CFT) Terraform Example Foundation</a>. Google Cloud
 Foundation Fabric FAST is intended to be a pre-composed end-to-end example,
 which is forked, cloned and modified as required. Whereas CFT is
 intended to be used a library of opinionated Terraform modules which should be composed
 as required. Google describes the differences between these two approaches <a href=\"https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/blob/master/FABRIC-AND-CFT.md\">here</a>.</li></ul><figure><img
 alt=\"\" src=\"https://cdn-images-1.medium.com/max/1024/0*wlX4xPvM5yR0TFaQ.png\"
 /><figcaption>Google’s Enterprise LZ IaC Accelerator — Google Cloud Foundation Fabric FAST</figcaption></figure><ul><li>How
 will you organise and manage access to IaC repos? Google recommends using
 the design principle that configurations with different approval and management
 requirements are separated into different source control repositories. For example,
 a central platform team may be responsible for the LZ IaC, shared resources, and
 the <a href=\"https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/tree/master/modules/project-factory\">tenant
 factory</a>. Whereas application teams may be responsible for all infrastructure
 resources deployed within their own folders. This approach — making use of pipeline
 layers — is recommended in enterprises, as it delegates control to application
 teams.</li><li>What CI/CD tools will you use in your GitOps pipeline.
 For example, you might choose Google Cloud Build for seamless integration with Google
 Cloud, if Google Cloud is the only infrastructure target of your IaC. Alternatively,
 if you’re already using GitHub and want more cloud agnosticity, then GitHub Actions
 might be a good choice.</li><li>How will tenants execute their IaC?
 Best practice is to only allow tenants to execute IaC using provided tenant service
 accounts.</li><li>IaC standards and best practices? Establish and
 document your organisation’s IaC and Terraform standards and best practices. And
 don’t reinvent the wheel. Google has <a href=\"https://cloud.google.com/docs/terraform/best-practices-for-terraform\">great
 guidance</a> on this already.</li><li>How will you enforce IaC policies
 and standards? Since all your cloud infrastructure will be deployed using
 IaC, it is important to ensure that the IaC you execute adheres to your organisation’s
 policies. For example, you might want to prevent deployment of certain resources,
 enforce use of labels with a limited set of values, or enforce customer-managed
 encryption keys on GCS buckets. Consider using an automated policy validation tool,
 such as Hashicorp Sentinel (but only if you’re using a Terraform Cloud or Terraform
 Enterprise backend), Terratest (which is open source), or Google’s gcloud terraform
 vet.</li></ul><h3>Wrap-Up</h3>After four articles on the topic of Google Cloud
 LZ design considerations, I think you’ll agree that the design phase is not entirely
 trivial! There are a lot of considerations, and many implications of your choices!In
 the next part, I’ll show you how to go about making informed decisions. I’ll guide
 you through the LZ design process, show you how to capture your decisions, and tell
 you how to get the help you need, so you don’t make any troublesome mistakes!<h3>Before
 You Go</h3><ul><li>Please share this with anyone that you think
 will be interested. It might help them, and it really helps me!</li><li>Feel free
 to leave a comment \U0001F4AC.</li><li>Follow
 and subscribe, so you don’t miss my content. Go to my <a href=\"https://medium.com/@derailed.dash\">Profile
 Page</a>, and click on these icons:</li></ul><figure><img alt=\"\" src=\"https://cdn-images-1.medium.com/max/163/0*fF62z2-FT03ui0O5.png\"
 /><figcaption>Follow and Subscribe</figcaption></figure><h3>Links</h3><ul><li><a
 href=\"https://medium.com/google-cloud/landing-zones-on-google-cloud-b42b08e1abaa\">Landing
 Zones on Google Cloud: What It Is, Why You Need One, and How to Create One</a></li><li><a
 href=\"https://cloud.google.com/architecture/landing-zones\">Landing zone design
 in Google Cloud</a></li><li><a href=\"https://cloud.google.com/docs/terraform/resource-management/managing-infrastructure-as-code\">Managing
 infrastructure-as-code with Terraform, Cloud Build, and GitOps</a></li><li><a href=\"https://cloud.google.com/docs/terraform\">Terraform
 on Google Cloud</a></li><li><a href=\"https://cloud.google.com/docs/terraform/best-practices-for-terraform\">Best
 practices for Terraform</a></li><li><a href=\"https://cloud.google.com/build/docs\">Google
 Cloud Build</a></li><li><a href=\"https://cloud.google.com/architecture/framework/operational-excellence/automate-your-deployments\">Automate
 your deployments</a></li><li><a href=\"https://cloud.google.com/architecture/security-foundations/deployment-methodology\">Deployment
 methodology</a></li><li><a href=\"https://cloud.google.com/source-repositories/docs/features\">Google
 Cloud Source Repositories</a></li><li><a href=\"https://cloud.google.com/docs/terraform/policy-validation\">Google
 Cloud Terraform policy validation with terraform vet</a></li><li><a href=\"https://cloud.google.com/docs/terraform/policy-validation/create-policy-library\">Google
 Cloud: creating a Terraform policy library</a></li><li><a href=\"https://cloud.google.com/architecture/security-foundations/summary#naming-conventions\">Google
 Cloud Resource Naming Standards</a></li><li><a href=\"https://github.com/terraform-google-modules/terraform-example-foundation\">Google
 CFT Terraform Example Foundation</a></li><li><a href=\"https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/tree/master/fast\">Google
 Cloud Foundation Fabric FAST</a></li><li><a href=\"https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/blob/master/FABRIC-AND-CFT.md\">Cloud
 Foundation Fabric FAST vs CFT</a></li><li><a href=\"https://medium.com/google-cloud/resource-factories-a-descriptive-approach-to-terraform-581b3ebb59c\">Resource
 factories</a></li><li><a href=\"https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/tree/master/blueprints/factories\">Resource
 factories in Cloud Foundation Fabric FAST</a></li><li><a href=\"https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/tree/master/modules/project-factory\">Cloud
 Foundation Fabric FAST: project and folder factory</a></li><li><a href=\"https://registry.terraform.io/modules/terraform-google-modules/project-factory/google/latest\">Terraform
 Google Cloud Project-Factory</a></li><li><a href=\"https://cloud.google.com/architecture/framework\">Google
 Cloud Architecture Framework</a></li><li><a href=\"https://cloud.google.com/architecture/security-foundations\">Enterprise
 Foundations Blueprint</a></li></ul><h3>Series Navigation</h3><ul><li><a href=\"https://medium.com/google-cloud/google-cloud-adoption-for-the-enterprise-from-strategy-to-operation-part-0-overview-9091f5a1ddfc\">Series
 overview and structure</a></li><li>Previous: <a href=\"https://medium.com/google-cloud/design-your-landing-zone-design-considerations-part-3-monitoring-logging-billing-and-7b40189a3c81\">Design
 your Landing Zone — Design Considerations Part 3: Monitoring, Logging, Billing and
 Labelling</a></li><li>Next: Design your Landing Zone — How To</li></ul><img src=\"https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ae3f533c6dbd\"
 width=\"1\" height=\"1\" alt=\"\"><hr><a href=\"https://medium.com/google-cloud/design-your-landing-zone-design-considerations-part-4-iac-gitops-and-ci-cd-google-cloud-ae3f533c6dbd\">Design
 your Landing Zone — Design Considerations Part 4— IaC, GitOps and CI/CD (Google
 Cloud…</a> was originally published in <a href=\"https://medium.com/google-cloud\">Google
 Cloud - Community</a> on Medium, where people are continuing the conversation by
 highlighting and responding to this story."

Language

Active

Ricc internal notes

Ricc source

Show this article Back to articles