Testing and configuring checks is an important part of your software development and delivery cycle. Automated tests and checks can be configured to run as part of the build or release pipeline in CTO.ai deployment pipelines. Check runs can be configured to provide tests and pipeline reports. These logs and reports generated from your environment on CTO.ai build pipelines help you to measure your pipeline quality, review traceability, troubleshoot pipeline environment failures, and notify the team of build failures. With our GitHub integration, you can easily visualize the four DORA metrics (Deployment frequency, Lead time for changes, Time to restore service, and Change failure rate) from your check runs logs gotten from your pipelines.

In this article, we’ll learn:

  • Set up automated pipeline check runs and tests as part of your build pipelines.
  • Configuring an ops.yml file.
  • Generating insights from your Pipeline tests.
  • Get insights into your team's velocity and throughput.

Prerequisites

  • Access to CTO.ai Account and CTO.ai CLI installed on your local machine
  • Docker installed on your machine
  • A GitHub Account  and GitHub Token with admin access
  • OPS_PASSWORD, OPS_USERNAME AND OPS_TEAM values stored in your secretes vault.

Setting up CTO.ai Account and Secrets

  1. Sign up and log into your CTO.ai Account.

2. In this guide, you’ll need to create a Team on CTO.ai, as this will be an ENV VAR you’ll pass in your ops.yml file.

  • In the Setup Status pane, Click on Continue to set up your Workflows and follow the instructions to set up your team.

3. In the Workflow integrations section, integrate your GitHub account with CTO.ai GitHub App to get to configure check runs and run automated tests from your GitHub repository. Click on Let’s Connect GitHub and follow the instructions.

4. Next, configure your team's secret from the Settings page. To build, manage, and test Check Runs, you’ll need to link the OPS_TEAM, OPS_USERNAME, and OPS_PASWORD secrets to your team.

  • On the settings page, click on Secrets and select New Secret.
  • The OPS_TEAM secret is the team you created in step 2.
  • The OPS_USERNAME is your username created when signing up on the CTO.ai platform.
  • The OPS_PASSWORD is the password of your CTO.ai Account.

Configuring the Ops.yml file

Before configuring check runs, make sure your GitHub App and repository have the checks:write permission to create check runs.

In your GitHub code repository, you’ll create the ops.yml file, where you’ll define your CI/CD jobs, pipelines, and steps to be processed and executed. In the ops.yml pipeline section, we will define our pipeline's jobs, workflows, and how we want to configure and build automated tests to run check runs.

Configuring Pipelines using CTO.ai Ops.yml Setup

The pipeline config above is an example template for configuring your pipelines on your applications. You can edit it to suit your needs. In this template, we have the name and description of the pipeline.

  • In the env section, we’re defining the static environment variables and the secrets. Under your secrets, attach your GITHUB_TOKEN and other secrets your specified in your application. These secrets will be stored in the secrets vault in CTO.ai.
  • The events section lets you specify which GitHub org, repo, and activity you want to trigger your checks and tests to run based on GitHub events like; pull request open, pull request merge, pull request closed.
  • A CTO.ai job is a collection of steps you configure on your application to run, test and deploy your application to different platforms. Steps are the building components of your pipelines. All the steps you specified in your job are executed in your CI process within the same workflow, in a fresh container using the Docker executor.  Each workflow in a project can have multiple pipelines, and these jobs run in a new isolated environment created by CTO.ai. Examples include building your code, checking out your code, running automated tests, etc.
pipelines:
  - name: sample-app-pipeline:0.1.0
    description: build a release for deployment on fargate
    env:
      static:
        - DEBIAN_FRONTEND=noninteractive
        - OPS_TEAM=workflows
      secrets:
        - GITHUB_TOKEN
        - AWS_ACCESS_KEY_ID
        - AWS_SECRET_ACCESS_KEY
        - AWS_ACCOUNT_NUMBER
    events:
      - "github:github_org/github_repo:pull_request.merged" 
      - "github:github_org/github_repo:create.tag" 
    jobs:
      - name: sample-app-build-job
        description: example build step
        packages:
          - git
          - unzip
          - python
        steps:
          - curl https://s3.amazonaws.com/aws-cli/awscli-bundle-1.18.200.zip -o awscli-bundle.zip
          - unzip awscli-bundle.zip && ./awscli-bundle/install -b ~/bin/aws
          - export PATH=~/bin:$PATH
          - aws --version
          - git clone https://$GITHUB_TOKEN:[email protected]/$ORG/$REPO
          - cd $REPO && ls -asl
          - aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin $AWS_ACCOUNT_NUMBER.dkr.ecr.us-east-1.amazonaws.com/$REPO
          - docker build -t $AWS_ACCOUNT_NUMBER.dkr.ecr.us-east-1.amazonaws.com/$REPO:$REF .
          - docker push $AWS_ACCOUNT_NUMBER.dkr.ecr.us-east-1.amazonaws.com/$REPO:$REF

Running CTO.ai CI Pipeline with Docker

We looked at how to configure your ops.yml file for your application pipeline configuration above. Now let’s execute and run our pipeline to trigger checks and automated CI tests.

  • Back in your repository, open a pull request with any code changes; you’ll see that the CI tests will start running.
  • When you click on the details, you’ll have a detailed view of the logs in your CTO.ai pipelines dashboard.

  • Once all checks are passed, you can merge your pull request.

Improve Pipeline performance with DORA Metrics

From your pipeline logs; you can automatically generate insights and measure your team's performance to gauge the efficacy of your software development and provide important insights and metrics into areas for improvement.  Using our GitHub action integration. The CTO.ai GitHub integration allows you to easily flow all your metrics from your application pipelines on GitHub into the CTO.ai Insights Dashboard.

  • In your repository, create a new file in the .github/workflows folder and push these configurations.
name: Deployment sample
on: [push, pull_request, release,]
jobs:
 build-and-deploy:
   name: deployment failed
   runs-on: ubuntu-latest

   steps:

   - uses: actions/checkout@v2

   # Add here the Deployment steps

   - name: Report Deployment Sucess

     if: ${{ success() }}
     uses: cto-ai/[email protected]
     id: ctoai-deployment-Succeeded

     with:
       team_id: ${{ secrets.CTOAI_TEAM }}

       token: ${{ secrets.CTOAI_TOKEN }}

       event_name: "deployment"

       event_action: "succeeded"

   - name: Report Deployment Failure

     if: ${{ failure() }}


     uses: cto-ai/[email protected]


     id: ctoai-deployment-failed

     with:

       team_id: ${{ secrets.CTOAI_TEAM }}

       token: ${{ secrets.CTOAI_TOKEN }}

       event_name: "deployment"

       event_action: "failed"

  • When you open a pull_request, push new changes to your application, and deploy a release, you should see the insights on your dashboard in CTO.ai.

Conclusion

In the tutorial above, you can see how check runs can help increase the speed of your software development through faster feedback, modified reruns, and more efficient use of resources. Sign up for free to get started