• Beta
Delivery Events
  • 02 Feb 2021
  • 3 Minutes To Read
  • Share
  • Dark
    Light

Delivery Events

  • Share
  • Dark
    Light

Overview

CTO.ai recommends four, research backed delivery Metrics from the DORA Organization in order to gain observability and insights into your Workflow. These currently are the best indicators of DevOps maturity.

The four Metrics are:

  1. Deployment Frequency

    Higher Deployment Frequency indicates mature tooling that takes human error out of releasing software.

  2. Lead Time for Changes

    Lead Time answers the question: how long does it take for a feature to go from concept to reality?

  3. Change Failure Rate

    Change Failure Rate is a measure of software quality–a high change failure rate suggests weakness in code reviews and testing.

  4. Mean Time to Recovery

    Agile teams release with confidence because their DevOps infrastructure can rollback defects. This shows your resilience to errors.

These Delivery Metrics score your team’s DevOps maturity in relation to the industry, and let you track improvements over time.

Examples are API Specs

These API calls can be made from our GitHub App, CircleCI Orb, or a platform of your choosing using HTTP.

1. Deployment Frequency

How often does your team deploy new changes?

Using the Github App we are able to caculate Deployment Frequency. Depending on the data source you are using, we do this in two different ways.

  1. Events about your deployments can be sent, either manually (as described below) or automatically through your GitHub Org integration. We will use the Deployment Succeeded and Deployment Failure events to calculate your Deployment events.
  2. If you are not getting Deployment Succeeded / Deployment Failure events from your GitHub Integration, we will use the Change Succeeded from Section 2 event from GitHub to calculate your deployments. We will use the Change Succeeded for a successful deployment and Change Failure event from Section 3 for a failed deployment.
Note

If you start using deployment events, we will switch to this method of calcuation. This will hide historical data usings the Change method. Please contact support for more information.

To instrument your pipeline for the deployment frequency metric, send a Deployment Succeeded event each time your pipeline deploys.

Example:

POST /events
{ 
  "event_name": "deployment",
  "event_action": "succeeded",
  "commit": "f22a7e12",
  "repo": "api",
  "branch": "main",
  "team_id": "02f71e12-fa48-45aa-a1f2-956b425e5bee"
}

Event_Name indicates the type of event, in this case it is a Deployment event.

Event_Action Succeeded indicates that the deployment was successful.

Team_Id is provided to you by CTO.ai. Pass this value in this payload so we know which Ops Team the event applies to.

2. Lead Time for Changes

To see Lead Time on your dashboard, instrument your pipelines with API calls for the Change Initiated event and the Change Succeeded event.

You can send these events at different points in your delivery process based on your team’s style.

For example, you could send the Change Initiated event when a pull request is opened, or when a feature ticket is created. If you send more than one Change Initiated event, we will calculate lead time according to the event that came first.

The Change Succeeded event could, for example, be sent when a pull request is merged to master, or when a feature is deployed to production. Where you put it in your pipeline is up to you—track what best fits your team's process.

POST /events
{ 
  "event_name": "change",
  "event_action": "initiated",
  "branch": "main",
  "repo": "api",
  "team_id": "02f71e12-fa48-45aa-a1f2-956b425e5bee"
}

POST /events
{ 
  "event_name": "change",
  "event_action": "succeeded",
  "branch": "main",
  "repo": "api",
  "team_id": "02f71e12-fa48-45aa-a1f2-956b425e5bee"
}

3. Change Failure Rate

To see the change failure rate metric, send Change Failure events when you detect a failure somewhere in your delivery. For example, this could be when a pipeline fails to deploy, or when a defect is detected in production.

POST /events
{ 
  "event_name": "change",
  "event_action": "failed",
  "branch": "main",
  "repo": "api",
  "team_id": "02f71e12-fa48-45aa-a1f2-956b425e5bee"
}

For the change failure rate to be calculated, you will also need to be sending Change Succeeded events.

POST /events
{ 
  "event_name": "change",
  "event_action": "succeeded",
  "branch": "main",
  "repo": "api",
  "team_id": "02f71e12-fa48-45aa-a1f2-956b425e5bee"
}

4. Mean Time to Recovery

Recovery can mean different things to different teams depending on your team’s unique process.

For example, a change failure recovery could be when an incident is marked resolved.

Record a recovery using the Change Recovered event:

POST /events
{ 
  "event_name": "change",
  "event_action": "recovered",
  "commit": "298673a9-a40f-4717-b288-060be948831b",
  "branch": "main",
  "repo": "api",
  "team_id": "02f71e12-fa48-45aa-a1f2-956b425e5bee"
}

🚀 What's next?


📞 Need help? No problem!


Was This Article Helpful?