Configuring Custom Dashboards with Grafana and CTO.ai Workflows

by Tola Ore-Aruwaji

Grafana is a popular open-source platform for monitoring and observability, offering you the ability to create interactive and visually rich dashboards from various data sources, including but not limited to Prometheus, InfluxDB, and Elasticsearch. In this blog, we will walk through the steps to configure custom dashboards with Grafana and CTO.ai Workflows, allowing you to build a tailored viewing panel that aligns precisely with your monitoring needs.

Prerequisites

Before we dive into this blog, ensure you have the following setups:

  • CTO.ai account
  • Helm installed in your K8s cluster
  • Prometheus and Grafana are installed in your K8s cluster
  • Kubernetes Cluster: A functioning Kubernetes cluster, set up and configured with kubectl.
  • Helm: Helm package manager installed in your system.

Setting up CTO.ai EKS ASG Workflow

Start by connecting your GitHub account and installing the EKS EC2 ASG workflow we support,  and choose the repository where your Kubernetes configurations are stored.

git clone git@github.com:workflows-sh/aws-eks-ec2-asg-cdk.git

cd aws-eks-ec2-asg-cdk

Next, set up and add your secret keys, AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY AWS_ACCOUNT_NUMBER, and GITHUB_TOKEN with write permissions to the project secret settings in CTO.ai.

After cloning the repo from GitHub, run and build your Workflow using ops build -b . and deploy your infrastructure to AWS.

  • Enter the name of your environment. You can use dev as the name of your environment. You can also use Prod or Stage, depending on what you want.
  • Your workflow will start deploying and creating your resources on AWS using CloudFormation

  • After deploying your AWS EC2 and Elastic Kubernetes workflow, you can see your stack directly on AWS CloudFormation. In your CloudFormation Stack, you can see your AWS resources created: Dev-AWS-EKS-ASG-Provider, AWS-EKS-EC2-ASG Resource, Dev-AWS-EKS-EC2-ASG, Sample-App-AWS-EKS, CDKToolkit, Dev-Sample-App-AWS-EKS-EC2-ASG.

The Prometheus stack is open source on GitHub and will deploy all the Kubernetes components like Secrets, Services, ConfigMaps, Deployment, and the Custom Resource Definition.

  • Install the Prometheus helm chart, which contains all the CRD you’ll use using the helm install prometheus  prometheus-community/kube-prometheus-stack command.
  • Get all your components on CLI using kubectl get all

  • After configuring and ensuring all components are running, we will access Grafana using port forwarding. Before you can port forward, you need to get the port in which your Grafana pod is running. kubectl get pod

  • Port forward your Grafana deployment using kubectl port-forward deployment/prometeus-grafana 3000

Kubectl port-forward allows you to access and interact with internal Kubernetes cluster processes from your localhost. You can use this method to investigate issues and adjust your services locally without the need to expose them beforehand.

  • In your browser, go to http://localhost:3000, and you will see the Grafana welcome page.
  • If you click on Dashboards icon, you’ll be able to see all the dashboards for your Kubernetes resources.
  • Next, in the Alert Manager Dashboards, you can see the general overview of your stack.

  • You can add more dashboards and get the total metrics of your Kubernetes resources. Back in your Dashboards, click on Dashboards, and select Kubernetes API Server.

From the dashboard, you can see your:

  • Availability
  • Error budget
  • Write SLI (Requests)
  • Write Availability

  • You can also see your CPU usage, memory quota, and pod usage.

Conclusion

Configuring custom dashboards in Grafana, and CTO.ai not only enhances your monitoring and observability capabilities but also brings in a personalized touch, ensuring that the metrics and data most pertinent to you are displayed prominently. Start building your custom dashboard today and experience a more data-driven and focused approach to data visualization and monitoring.