• Beta
Pipelines: How to Setup
  • 18 Nov 2020
  • 3 Minutes To Read
  • Share
  • Dark
    Light

Pipelines: How to Setup

  • Share
  • Dark
    Light

What is a Pipeline?

Pipelines let you automate multi-stage workflows such as your build, test, or Event Data Pipelines. Pipelines are versioned by default, so that you can easily measure process changes & roll back as needed.

Pipelines are Cloud Native and Serverless, allowing you to run them from Slack or trigger them from manually, from Github, GitLab, or BitBucket. External triggering coming soon.

Get Started with Pipelines

1. Init a Pipeline Op

To start using the Pipeline feature, you will use the Ops init flow on the CLI. Run the following command in your terminal ops init

pickpipeline

  • You will be prompted to choose between different types of Ops.
  • Select pipeline and press enter.
  • Choose a name, description and a version for your Pipeline.

PLinit

You can work on your Pipeline by editing the ops.yml file in the newly created project directory.

Refer below to the "Example Use Case" section for a walk through of a common uses case using Pipelines.

2. Build the Pipeline Op

Before you can run your Pipeline you will need to build it.

  • In your terminal run ops build pipeline_name
    (replace "pipeline_name" by your actual pipeline name)

For more information on Building click the link.

3. Publish the Pipeline Op

Once built, publish your Pipeline to run from the CLI, Slack and/or the CTO.ai Dashboard.

  • In your terminal run ops publish pipeline_name

For more information on Publishing click the link.

Congratulations! Your Pipeline is now built, published and ready for you and your teammates to run it!

4. Start or Run a Pipeline Op

Once the Pipeline has been published it can be started or run from the CLI, Slack and/or the Ops Dashboard. Below walks you through how to run from each.

  1. Using the CLI to start the Pipeline local via Docker

    • Go to your terminal
    • run the following command ops start pipeline_name
    • you will see all your logs displayed in your local terminal
  2. Using Slack

  • Associate your Ops user & team with a Slack Workspace and Channel (if you haven't already)
  • Run directly in the associated Channel with the command /ops run pipeline_name
  1. Through the CTO.ai Dashboard
    • Go to cto.ai
    • Sign in and go to the Dashboard
    • Pick the team you have published your Pipeline to
    • Access the Pipelines tab which will list all of your current Pipelines
    • Select the Pipeline of your choice and click the run button

pipelinewebdetail

Using the CTO.ai Dashboard

The image below shows a list of Pipelines in the Dashboard.
pipelineslist
You can see:

  • The list of Pipelines created
  • Who ran them
  • When it was Run
  • Version
  • Status (Success/Complete, Running, Failed)

You can click into each Pipeline and see a detailed view- Steps, Logs, and other Pipeline specific information

Example Use Case

Let's walk through an example of a common CICD pipeline with 3 steps- Clone, Test, and Deploy.

Below is the sample code for this example.

version: "1"
pipelines:
  - name: example-pipeline:0.1.2
    description: Example Pipeline
    jobs:
      - name: clone
        description: clone the application repo
        sdk: "2"
        packages:
          - git
        steps:
          - git clone https://github.com/shiehn/documentation-example.git /tmp/state/documentation-example
          - echo "successfully cloned application code"
      - name: test
        description: test the code
        sdk: "2"
        packages:
          - golang-go
        steps:
          - cd /tmp/state/documentation-example
          - go test .
          - echo "tests completed succesfully"
      - name: deploy
        description: build and deploy the artifact
        sdk: "2"
        packages:
          - golang-go
        steps:
          - cd /tmp/state/documentation-example
          - go build -o $HOME/final-build
          - echo "the final-build has been created and is ready to deployed!"
# NOTE: it would be very common to deploy this 'final-build' to a server

There are 3 steps:

  1. Step 1: Clone: Clone a public Golang repo into /tmp/state.
    /tmp/state is a special directory which is persisted across all 'steps'- if you put something in /tmp/state in step one it can be accessed in step two

  2. Step 2: Test: Simply run the Golang tests. When the tests succeed it will return a zero exit code and the Pipeline will continue.
    In the event the tests fail it will return a non-zero exit code and the entire Pipeline will fail

  3. Step 3: Deploy: This is placeholder code. This is where you would place code to deploy your artifact, website, etc to a server. For demo reasons this is commented out, but put your own logic here.

🚀 What's Next


Need help getting started?

Contact Support — we're here to help!

Was This Article Helpful?