Optimizing Cloud-Native Applications with CTO.ai, Redis, AWS CDK, and EKS

Introduction

The synergy and functionality between different technologies in the DevOps and Cloud infrastructure often defines the efficiency and scalability of a system. In this blog, we explore how AWS Cloud Development Kit (CDK), Elastic Kubernetes Service (EKS), Redis, and CTO.ai work in concert to create robust, scalable, and efficient cloud-native applications.

We will be exploring the CTO.ai EC2 ASG CDK Workflow, which is open source in our GitHub Organization.

AWS Cloud Development Kit (CDK) - The Foundation

AWS CDK is an open-source software development framework for defining cloud infrastructure in code and provisioning it through AWS CloudFormation. It provides high-level components that preconfigure cloud resources with sensible defaults, making it easier to build cloud applications.

Elastic Kubernetes Service (EKS) - Orchestrating Containers

Amazon EKS is a managed service that makes it easy to run Kubernetes on AWS without needing to install and operate your own Kubernetes control plane. It's highly scalable and secure, making it an ideal choice for running containerized applications.

Redis - High-Performance Data Store

Redis is an in-memory data structure store used as a database, cache, and message broker. It supports various data structures and is known for its high performance.

CTO.ai - Simplifying Developer Experience

CTO.ai is a platform that simplifies the complexity of DevOps processes, making it more accessible to teams without deep operational expertise. It provides tools and workflows that enhance development and deployment operations.

Integrating Redis with AWS CDK

The integration of Redis into a cloud architecture can be efficiently handled using AWS CDK, as demonstrated in the redis.ts configuration. This script automates the setup of an Elasticache Redis cluster within an AWS environment.

import * as cdk from 'aws-cdk-lib';
import * as elasticache from 'aws-cdk-lib/aws-elasticache';
import * as ec2 from 'aws-cdk-lib/aws-ec2';
import { Construct } from 'constructs';

interface StackProps {
  vpc: ec2.Vpc
}

class RedisCluster extends Construct {
  public readonly cluster: elasticache.CfnCacheCluster
  constructor(scope: Construct, id:string, props:StackProps) {
    super(scope, id);

    const targetVpc = props.vpc;

    // Define a group for telling Elasticache which subnets to put cache nodes in.
    const subnetGroup = new elasticache.CfnSubnetGroup(this, `${id}-subnet-group`, {
      description: `List of subnets used for redis cache ${id}`,
      subnetIds: targetVpc.privateSubnets.map(function(subnet) {
        return subnet.subnetId;
      })
    });

    // The security group that defines network level access to the cluster
    const securityGroup = new ec2.SecurityGroup(this, `${id}-security-group`, {
      vpc: targetVpc
    });

    securityGroup.addIngressRule(
      ec2.Peer.anyIpv4(),
      ec2.Port.allTraffic(),
      'Allow all connections to redis from inside the VPC'
    );

    // The cluster resource itself.
    this.cluster = new elasticache.CfnCacheCluster(this, `${id}-cluster`, {
      cacheNodeType: 'cache.t2.micro',
      engine: 'redis',
      numCacheNodes: 1,
      autoMinorVersionUpgrade: true,
      cacheSubnetGroupName: subnetGroup.ref,
      vpcSecurityGroupIds: [
        securityGroup.securityGroupId
      ]
    });
  
  }

}

export { RedisCluster as Cluster }

Key components in redis.ts:

  • VPC Configuration: It sets up a VPC (Virtual Private Cloud) for secure networking of the services.
  • Redis Cluster: It defines an Elasticache Redis cluster, specifying the node type, engine, and other configurations.
  • Security and Network: It includes a security group and subnet group setup for the Redis cluster.

Building an EKS Cluster with AWS CDK

The cluster.ts script exemplifies the power of AWS CDK in orchestrating an EKS cluster. It outlines a comprehensive setup including a VPC, security groups, an EKS cluster, a serverless Aurora MySQL database, and a Redis cluster.

import * as cdk from 'aws-cdk-lib';
import * as iam from 'aws-cdk-lib/aws-iam'
import * as ec2 from 'aws-cdk-lib/aws-ec2'
import * as eks from 'aws-cdk-lib/aws-eks'
import * as rds from 'aws-cdk-lib/aws-rds'
import * as sqs from 'aws-cdk-lib/aws-sqs'
import * as elasticache from './redis'
import * as autoscaling from 'aws-cdk-lib/aws-autoscaling'
import { Construct } from 'constructs';

interface StackProps {
  org: string
  env: string
  repo: string
  tag: string
  key: string
  entropy: string
}

export default class Cluster extends cdk.Stack {

  public readonly id: string
  public readonly org: string
  public readonly env: string
  public readonly repo: string
  public readonly tag: string
  public readonly key: string
  public readonly entropy: string

  public readonly vpc: ec2.Vpc
  public readonly cluster: eks.Cluster
  public readonly db: rds.ServerlessCluster
  public readonly mq: sqs.Queue
  public readonly redis: Construct
  public readonly bastion: ec2.BastionHostLinux

  constructor(scope: Construct, id: string, props?: StackProps) {
    super(scope, id)
    this.id = id
    this.org = props?.org ?? 'cto-ai'
    this.env = props?.env ?? 'dev'
    this.key = props?.key ?? 'aws-eks-ec2-asg'
    this.repo = props?.repo ?? 'sample-expressjs-aws-eks-ec2-asg-cdk'
    this.tag = props?.tag ?? 'main'
    this.entropy = props?.entropy ?? '01012022'

    // todo @kc make AZ a StackProp
    const vpc = new ec2.Vpc(this, `${this.id}-vpc`, { 
      cidr: '10.0.0.0/16',
      natGateways: 1,
      maxAzs: 3,
      subnetConfiguration: [
        {
          name: 'Public',
          subnetType: ec2.SubnetType.PUBLIC,
          cidrMask: 24,
        },
        {
          name: 'Private',
          subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS,
          cidrMask: 24,
        }
      ],
    }); 

    const bastionSecurityGroup = new ec2.SecurityGroup(this, `${this.id}-bastion-sg`, {
      vpc: vpc,
      allowAllOutbound: true,
      description: `bastion security group for ${this.id} cluster`,
      securityGroupName: `${this.id}-bastion-sg`
    });
    bastionSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(22), 'SSH access');

    const bastion = new ec2.BastionHostLinux(this, `${this.id}-bastion`, {
      vpc: vpc,
      instanceName: `${this.id}-bastion`,
      securityGroup: bastionSecurityGroup,
      subnetSelection: {
        subnetType: ec2.SubnetType.PUBLIC
      }
    });

    const cluster = new eks.Cluster(this, `${this.id}-eks`, {
      vpc: vpc,
      defaultCapacity: 0,
      defaultCapacityInstance: ec2.InstanceType.of(ec2.InstanceClass.M5, ec2.InstanceSize.XLARGE),
      version: eks.KubernetesVersion.V1_21,
      mastersRole: new iam.Role(this, 'MastersRole', { assumedBy: new iam.AccountRootPrincipal() })
    });

    const rootVolume: autoscaling.BlockDevice = {
      deviceName: '/dev/xvda', // define the root volume
      volume: autoscaling.BlockDeviceVolume.ebs(100), // override volume size
    };

    // IAM role for our EC2 worker nodes
    const workerRole = new iam.Role(this, `${this.id}-workers` , {
      assumedBy: new iam.ServicePrincipal('ec2.amazonaws.com')
    });

    const onDemandASG = new autoscaling.AutoScalingGroup(this, `${this.id}-asg`, {
      vpc: vpc,
      role: workerRole,
      minCapacity: 1,
      maxCapacity: 10,
      desiredCapacity: 1,
      blockDevices: [rootVolume],
      instanceType: ec2.InstanceType.of(ec2.InstanceClass.M5, ec2.InstanceSize.XLARGE),
      machineImage: new eks.EksOptimizedImage({
        kubernetesVersion: '1.21',
        nodeType: eks.NodeType.STANDARD  // without this, incorrect SSM parameter for AMI is resolved
      }),
      updatePolicy: autoscaling.UpdatePolicy.rollingUpdate()
    });

    cluster.connectAutoScalingGroupCapacity(onDemandASG, {});

    const dbSecurityGroup = new ec2.SecurityGroup(this, `${this.id}-db-sg`, {
      vpc: vpc,
      allowAllOutbound: true,
      description: `db security group for ${this.id} db`,
      securityGroupName: `${this.id}-db-sg`
    });
    dbSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(3306), 'MySQL access');

    const db = new rds.ServerlessCluster(this, `${this.id}-db`, {
      vpc: vpc,
      defaultDatabaseName: `${this.env}`,
      engine: rds.DatabaseClusterEngine.AURORA_MYSQL,
      scaling: { autoPause: cdk.Duration.seconds(0) },
      vpcSubnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS },
      securityGroups: [dbSecurityGroup],
      credentials: rds.Credentials.fromGeneratedSecret('root')
    });

    const redis = new elasticache.Cluster(this, `${this.id}-redis`, { vpc: vpc });
    const mq = new sqs.Queue(this, `${this.id}-sqs`);

    this.vpc = vpc;
    this.cluster = cluster;
    this.bastion = bastion;
    this.redis = redis;
    this.db = db;
    this.mq = mq;

    new cdk.CfnOutput(this, `${this.id}VpcId`, { value: this.vpc.vpcId})
    new cdk.CfnOutput(this, `${this.id}ClusterArn`, { value: this.cluster.clusterArn})
    new cdk.CfnOutput(this, `${this.id}DbArn`, { value: this.db?.clusterArn})

  }
}

Key points in cluster.ts:

  • EKS Cluster: Establishes an EKS cluster, specifying the version and instance types.
  • Worker Nodes: Sets up Auto Scaling Groups for worker nodes, ensuring scalability.
  • Aurora MySQL: Integrates a serverless Aurora MySQL cluster, providing a relational database.
  • Redis Integration: Incorporates the Redis cluster setup from redis.ts.
  • Additional Resources: Sets up other resources like SQS queues and bastion hosts for secure access.

CTO.ai - Orchestrating the Workflow

CTO.ai comes into play by providing the DevOps teams with tools to manage and automate workflows around the infrastructure defined by CDK. It can be used to enhance the deployment process, manage Kubernetes clusters, and monitor the performance of applications.

Conclusion

The combination of AWS CDK, EKS, Redis, and CTO.ai offers a powerful stack for building and managing cloud-native applications. CDK provides the infrastructure-as-code capability, EKS offers a managed Kubernetes service, Redis brings in high-performance data storage, and CTO.ai simplifies the entire DevOps cycle. This integrated approach not only enhances efficiency but also ensures the scalability and reliability of cloud-based applications, making it an ideal choice for modern application development and deployment.

Ready to unlock the power of CTO.ai for your team? Schedule your consultation now with one of our experts today!