• Beta
Delivery Events
  • 18 Nov 2020
  • 3 Minutes To Read
  • Share
  • Dark
    Light

Delivery Events

  • Share
  • Dark
    Light

Overview

CTO.ai recommends four, research backed delivery Metrics from the DORA Organization in order to gain observability and insights into your Workflow. These currently are the best indicators of DevOps maturity.

The four Metrics are:

  1. Deployment Frequency

    Higher Deployment Frequency indicates mature tooling that takes human error out of releasing software.

  2. Lead Time for Changes

    Lead Time answers the question: how long does it take for a feature to go from concept to reality?

  3. Change Failure Rate

    Change Failure Rate is a measure of software quality–a high change failure rate suggests weakness in code reviews and testing.

  4. Mean Time to Recovery

    Agile teams release with confidence because their DevOps infrastructure can rollback defects. This shows your resilience to errors.

These Delivery Metrics score your team’s DevOps maturity in relation to the industry, and let you track improvements over time.

Examples are API Specs

These API calls can be made from our GitHub Action, CircleCI Orb, or a platform of your choosing using HTTP.

1. Deployment Frequency

How often does your team deploy new changes?

To instrument your pipeline for the deployment frequency metric, send a Deployment Succeeded event each time your pipeline deploys.

Example:

POST /events
{ 
  "stage": "Deployment",
  "status": "Succeeded",
  "stage_ref": "f22a7e12", // links together multiple deployment events
  "pipeline_id": "api-pipeline-1",
  "change_id": "298673a9-a40f-4717-b288-060be948831b",
  "team_id": "02f71e12-fa48-45aa-a1f2-956b425e5bee"
}

Stage indicates the type of event, in this case it is a Deployment event.

Status Succeeded indicates that the deployment was successful.

Stage Ref is a value you provide for each associated event. For example, if you want to associate a deployment failure with a deployment succeeded, you could give them both the same stage_ref. This will determine how the dashboard calculates metrics and displays certain timelines.

Pipeline_Id is a label you provide for the pipeline you are running. For example, a pipeline that deploys an API could be called “api-pipeline-1”.

Change_Id is an identifier you provide that is used to associate the deployment with change events. For example, if you send the change_id value “change-1”, then every event sent with the value “change-1” will be displayed together on the dashboard. We recommend setting this to an identifier associated with a ticket or a feature branch, depending on how your process works.

Team_Id is provided to you by CTO.ai. Pass this value in this payload so we know which Ops Team the event applies to.

2. Lead Time for Changes

To see Lead Time on your dashboard, instrument your pipelines with API calls for the Change Initiated event and the Change Succeeded event.

You can send these events at different points in your delivery process based on your team’s style.

For example, you could send the Change Initiated event when a pull request is opened, or when a feature ticket is created. If you send more than one Change Initiated event, we will calculate lead time according to the event that came first.

The Change Succeeded event could, for example, be sent when a pull request is merged to master, or when a feature is deployed to production. Where you put it in your pipeline is up to you—track what best fits your team's process.

POST /events
{ 
  "stage": "Change",
  "status": "Initiated",
  "change_id": "298673a9-a40f-4717-b288-060be948831b",
  "pipeline_id": "api-pipeline-1",
  "team_id": "02f71e12-fa48-45aa-a1f2-956b425e5bee"
}

POST /events
{ 
  "stage": "Change",
  "status": "Succeeded",
  "change_id": "298673a9-a40f-4717-b288-060be948831b",
  "pipeline_id": "api-pipeline-1",
  "team_id": "02f71e12-fa48-45aa-a1f2-956b425e5bee"
}

3. Change Failure Rate

To see the change failure rate metric, send Change Failure events when you detect a failure somewhere in your delivery. For example, this could be when a pipeline fails to deploy, or when a defect is detected in production.

POST /events
{ 
  "stage": "Change",
  "status": "Failure",
  "change_id": "298673a9-a40f-4717-b288-060be948831b",
  "pipeline_id": "api-pipeline-1",
  "team_id": "02f71e12-fa48-45aa-a1f2-956b425e5bee"
}

For the change failure rate to be calculated, you will also need to be sending Change Succeeded events.

POST /events
{ 
  "stage": "Change",
  "status": "Succeeded",
  "change_id": "298673a9-a40f-4717-b288-060be948831b",
  "pipeline_id": "api-pipeline-1",
  "team_id": "02f71e12-fa48-45aa-a1f2-956b425e5bee"
}

4. Mean Time to Recovery

Recovery can mean different things to different teams depending on your team’s unique process.

For example, a change failure recovery could be when an incident is marked resolved.

If you don’t send Change Recovery events, we will assume that a Deployment Succeeded event after a Change Failure event marks a Recovery.

Record a recovery using the Change Recovery event:

POST /events
{ 
  "stage": "Change",
  "status": "Recovery",
  "change_id": "298673a9-a40f-4717-b288-060be948831b",
  "pipeline_id": "api-pipeline-1",
  "team_id": "02f71e12-fa48-45aa-a1f2-956b425e5bee"
}
Was This Article Helpful?