GCP-GKE-Pulumi-py

GCP GKE PULUMI PLATFORM SETUP

Before you get started with the GCP GKE Pulumi Workflow, you need to check out our setup Workflow to Create your Account on CTO.ai and install CLI.

The GCP-GKE-Pulumi-Py infrastructure as code lets you deploy infrastructure over GCP using the Pulumi Python language framework.

Create Secret from Settings

Back in your CTO.ai dashboard, create your Secrets by selecting Settings and Secrets. The credentials can be generated from your GCP service account. Follow this guide on creating keys on GCP

Secrets are encrypted environment variables that CTO.ai utilizes within your workflow to build and run your application and deployments.

Create Token on Pulumi

Click on the Settings tab and select Access Tokens. Create an Access Token for youro deployment.

GCP GKE Pulumi Py Demo

The GCP GKE Pulumi stack is a complete IaC for deploying infrastructure over GCP: Kubernetes, Container Registry, Database Clusters, Load Balancers, and Project Resource Management all built using Python + Pulumi + CTO.ai.

# for more info visit https://cto.ai/docs
version: "1"
commands:
  - name: setup:0.1.0
    description: Setups Infrastructure over GCP
    run: python3 /ops/src/setup.py
    env:
      static:
        - GOOGLE_ZONE=us-central1-c
        - STACK_TYPE=gcp-gke-pulumi
        - GOOGLE_PROJECT=development-349122 # TODO I think this must be an input as it's a common practice to use one GCP project per environment
        - PULUMI_ORG=cto-ai
      secrets:
        - PULUMI_TOKEN # TODO Could be retrieved using the SDK as well
  - name: destroy:0.1.0
    description: Destroy Infrastructure over GCP
    run: python3 /ops/src/destroy.py
    env:
      static:
        - GOOGLE_ZONE=us-central1-c
        - STACK_TYPE=gcp-gke-pulumi
        - GOOGLE_PROJECT=development-349122 # TODO I think this must be an input as it's a common practice to use one GCP project per environment
        - PULUMI_ORG=cto-ai
        - GOOGLE_SA_EMAIL=pulumi@development-349122.iam.gserviceaccount.com
      secrets:
        - PULUMI_TOKEN # TODO Could be retrieved using the SDK as well
      configs:
        - DEV_GCP_SERVICES
  - name: deploy:0.1.0
    description: Deploy a service over GKE
    run: python3 /ops/src/deploy.py
    env:
      static:
        - GOOGLE_ZONE=us-central1-c
        - STACK_TYPE=gcp-gke-pulumi
        - GOOGLE_PROJECT=development-349122 # TODO I think this must be an input as it's a common practice to use one GCP project per environment
        - PULUMI_ORG=cto-ai
        - ORG=cto-ai
        - GOOGLE_SA_EMAIL=pulumi@development-349122.iam.gserviceaccount.com
      secrets:
        - PULUMI_TOKEN # TODO Could be retrieved using the SDK as well
      configs:
        - DEV_GCP_SERVICES
  - name: vault:0.1.0
    run: python3 /ops/src/vault.py
    description: Manage secrets vault
    env:
      static:
        - GOOGLE_ZONE=us-central1-c
        - GOOGLE_PROJECT=development-349122
        - STACK_TYPE=gcp-gke-pulumi
        - PULUMI_ORG=cto-ai
        - GOOGLE_SA_EMAIL=pulumi@development-349122.iam.gserviceaccount.com
      secrets:
        - PULUMI_TOKEN
    help:
      usage: ops run vault -a/--action <sub-cmd> -k/--key <--key> -v/--value <--value>
      arguments:
        init: Init the environments vault
        create: Set a key in the enviroment vault
        list: List keys in the environment vault
        bulk: Add multiple secrets given a secret key from CTO.ai
        remove: Remove a specific key in the environment vault
        destroy: Destroy the environment vault
pipelines:
  - name: sample-app-gcr-pipeline:0.1.0
    description: Build and push an image to GCR
    env:
      static:
        - ENV=dev
        - GOOGLE_ZONE=us # Valid zones https://cloud.google.com/container-registrypushing-and-pulling#tag
        - GOOGLE_PROJECT=development-349122
        - GOOGLE_SA_EMAIL=pulumi@development-349122.iam.gserviceaccount.com
        - DEBIAN_FRONTEND=noninteractive
        - ORG=cto-ai
        - REPO=sample-app
        - STACK_TYPE=gcp-gke-pulumi
      secrets:
        - GITHUB_TOKEN
    events:
      - "github:github_org/github_repo:pull_request.merged"
      - "github:github_org/github_repo:tag.created"
    jobs:
      - name: sample-app-docker-gcr-job
        description: Job for building and pushing a Docker image to GCR
        packages:
          - git
          - unzip
          - python
          - wget
          - tar
        steps:
          - set +u
          - wget -qO- https://dl.google.com/dl/cloudsdk/channels/rapid/downloads/google-cloud-cli-386.0.0-linux-x86_64.tar.gz | tar xz
          - CLOUDSDK_CORE_DISABLE_PROMPTS=1 ~/google-cloud-sdk/install.sh &> /dev/null && source ~/google-cloud-sdk/path.bash.inc
          - if [ -z $ENV ]; then ENV=$(ux prompt list "dev" "stg" "prd" --message "Select the environment" --default "dev" --name "env"); fi
          - ENV_UPPER=$(echo "$ENV" | tr '[:lower:]' '[:upper:]')
          - GOOGLE_CREDENTIALS=$(sdk secret get -H "$ENV_UPPER"_GOOGLE_CREDENTIALS) && echo $GOOGLE_CREDENTIALS > ~/credentials.json
          - gcloud auth activate-service-account $GOOGLE_SA_EMAIL --key-file=$HOME/credentials.json
          - git clone https://$GITHUB_TOKEN:[email protected]/$ORG/$REPO
          - cd $REPO && ls -asl
          - RAW_BRANCH_LIST=`git branch -r --sort=-committerdate | grep -v HEAD | head -n 20 | sed 's/origin\///' | sed 's/\ \ //' | awk '{print}' ORS="' '"`
          - echo "ux prompt list '${RAW_BRANCH_LIST}custom' --message='Select the repository branch to build' --name='reference'" >> /tmp/ux.sh && chmod 755 /tmp/ux.sh
          - if [ -z "${REF}" ]; then REF=`/tmp/ux.sh`; fi
          - if [ "${REF}" = "custom" ]; then REF=`ux prompt input --message='Enter the repository branch to build' --name='reference'`; fi && echo "Going to checkout $REF"
          - git fetch && git checkout $REF
          - REPO_TAG=$(echo "$REF" | sed 's/\//-/' )
          - gcloud auth configure-docker --quiet --user-output-enabled=false
          - docker build -f Dockerfile -t $GOOGLE_ZONE.gcr.io/$GOOGLE_PROJECT/$ORG/$REPO-$STACK_TYPE:$REPO_TAG .
          - docker push $GOOGLE_ZONE.gcr.io/$GOOGLE_PROJECT/$ORG/$REPO-$STACK_TYPE:$REPO_TAG

Build Pipelines locally with the CTO.ai CLI

In your terminal, enter the ops build . command and select the sample-app-gcr-pipeline. This ops build . command will build your op for sharing, your Docker image from your Docker file, and the set of files located in the specified path you created in your source code.

When the image is built, it’s going to create an image ID, and successfully tag it in your CTO.ai console.

Run and set up your Infrastructure

Next, you need to build and set up the infrastructure that will deploy each resource to GCP using the Pulumi Python framework. Set up your infrastructure using ops run -b . This will provision your stack using Pulumi

select setup Infrastructure over GCP

The process will build your Docker image and start loading up your GCP-GKE Stack

Select the environment to deploy to. In this guide, we are working with the dev environment.

Select the service(s) to deploy, in this guide we will select all

Install the dependencies

Your workflow will start deploying and creating your resources on GCP using Pulumi

You can also see the ouput dirctly in your terminal

View Resources in Pulumi

While your resources are deploying on the terminal you can click on the view/preview link on your terminal, it will redirect you to view the resources directly on Pulumi