Optimizing Cloud-Native Applications with CTO.ai, Redis, AWS CDK, and EKS
Introduction
The synergy and functionality between different technologies in the DevOps and Cloud infrastructure often defines the efficiency and scalability of a system. In this blog, we explore how AWS Cloud Development Kit (CDK), Elastic Kubernetes Service (EKS), Redis, and CTO.ai work in concert to create robust, scalable, and efficient cloud-native applications.
We will be exploring the CTO.ai EC2 ASG CDK Workflow, which is open source in our GitHub Organization.
AWS Cloud Development Kit (CDK) - The Foundation
AWS CDK is an open-source software development framework for defining cloud infrastructure in code and provisioning it through AWS CloudFormation. It provides high-level components that preconfigure cloud resources with sensible defaults, making it easier to build cloud applications.
Elastic Kubernetes Service (EKS) - Orchestrating Containers
Amazon EKS is a managed service that makes it easy to run Kubernetes on AWS without needing to install and operate your own Kubernetes control plane. It's highly scalable and secure, making it an ideal choice for running containerized applications.
Redis - High-Performance Data Store
Redis is an in-memory data structure store used as a database, cache, and message broker. It supports various data structures and is known for its high performance.
CTO.ai - Simplifying Developer Experience
CTO.ai is a platform that simplifies the complexity of DevOps processes, making it more accessible to teams without deep operational expertise. It provides tools and workflows that enhance development and deployment operations.
Integrating Redis with AWS CDK
The integration of Redis into a cloud architecture can be efficiently handled using AWS CDK, as demonstrated in the redis.ts configuration. This script automates the setup of an Elasticache Redis cluster within an AWS environment.
import * as cdk from 'aws-cdk-lib';
import * as elasticache from 'aws-cdk-lib/aws-elasticache';
import * as ec2 from 'aws-cdk-lib/aws-ec2';
import { Construct } from 'constructs';
interface StackProps {
vpc: ec2.Vpc
}
class RedisCluster extends Construct {
public readonly cluster: elasticache.CfnCacheCluster
constructor(scope: Construct, id:string, props:StackProps) {
super(scope, id);
const targetVpc = props.vpc;
// Define a group for telling Elasticache which subnets to put cache nodes in.
const subnetGroup = new elasticache.CfnSubnetGroup(this, `${id}-subnet-group`, {
description: `List of subnets used for redis cache ${id}`,
subnetIds: targetVpc.privateSubnets.map(function(subnet) {
return subnet.subnetId;
})
});
// The security group that defines network level access to the cluster
const securityGroup = new ec2.SecurityGroup(this, `${id}-security-group`, {
vpc: targetVpc
});
securityGroup.addIngressRule(
ec2.Peer.anyIpv4(),
ec2.Port.allTraffic(),
'Allow all connections to redis from inside the VPC'
);
// The cluster resource itself.
this.cluster = new elasticache.CfnCacheCluster(this, `${id}-cluster`, {
cacheNodeType: 'cache.t2.micro',
engine: 'redis',
numCacheNodes: 1,
autoMinorVersionUpgrade: true,
cacheSubnetGroupName: subnetGroup.ref,
vpcSecurityGroupIds: [
securityGroup.securityGroupId
]
});
}
}
export { RedisCluster as Cluster }
Key components in redis.ts:
- VPC Configuration: It sets up a VPC (Virtual Private Cloud) for secure networking of the services.
- Redis Cluster: It defines an Elasticache Redis cluster, specifying the node type, engine, and other configurations.
- Security and Network: It includes a security group and subnet group setup for the Redis cluster.
Building an EKS Cluster with AWS CDK
The cluster.ts script exemplifies the power of AWS CDK in orchestrating an EKS cluster. It outlines a comprehensive setup including a VPC, security groups, an EKS cluster, a serverless Aurora MySQL database, and a Redis cluster.
import * as cdk from 'aws-cdk-lib';
import * as iam from 'aws-cdk-lib/aws-iam'
import * as ec2 from 'aws-cdk-lib/aws-ec2'
import * as eks from 'aws-cdk-lib/aws-eks'
import * as rds from 'aws-cdk-lib/aws-rds'
import * as sqs from 'aws-cdk-lib/aws-sqs'
import * as elasticache from './redis'
import * as autoscaling from 'aws-cdk-lib/aws-autoscaling'
import { Construct } from 'constructs';
interface StackProps {
org: string
env: string
repo: string
tag: string
key: string
entropy: string
}
export default class Cluster extends cdk.Stack {
public readonly id: string
public readonly org: string
public readonly env: string
public readonly repo: string
public readonly tag: string
public readonly key: string
public readonly entropy: string
public readonly vpc: ec2.Vpc
public readonly cluster: eks.Cluster
public readonly db: rds.ServerlessCluster
public readonly mq: sqs.Queue
public readonly redis: Construct
public readonly bastion: ec2.BastionHostLinux
constructor(scope: Construct, id: string, props?: StackProps) {
super(scope, id)
this.id = id
this.org = props?.org ?? 'cto-ai'
this.env = props?.env ?? 'dev'
this.key = props?.key ?? 'aws-eks-ec2-asg'
this.repo = props?.repo ?? 'sample-expressjs-aws-eks-ec2-asg-cdk'
this.tag = props?.tag ?? 'main'
this.entropy = props?.entropy ?? '01012022'
// todo @kc make AZ a StackProp
const vpc = new ec2.Vpc(this, `${this.id}-vpc`, {
cidr: '10.0.0.0/16',
natGateways: 1,
maxAzs: 3,
subnetConfiguration: [
{
name: 'Public',
subnetType: ec2.SubnetType.PUBLIC,
cidrMask: 24,
},
{
name: 'Private',
subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS,
cidrMask: 24,
}
],
});
const bastionSecurityGroup = new ec2.SecurityGroup(this, `${this.id}-bastion-sg`, {
vpc: vpc,
allowAllOutbound: true,
description: `bastion security group for ${this.id} cluster`,
securityGroupName: `${this.id}-bastion-sg`
});
bastionSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(22), 'SSH access');
const bastion = new ec2.BastionHostLinux(this, `${this.id}-bastion`, {
vpc: vpc,
instanceName: `${this.id}-bastion`,
securityGroup: bastionSecurityGroup,
subnetSelection: {
subnetType: ec2.SubnetType.PUBLIC
}
});
const cluster = new eks.Cluster(this, `${this.id}-eks`, {
vpc: vpc,
defaultCapacity: 0,
defaultCapacityInstance: ec2.InstanceType.of(ec2.InstanceClass.M5, ec2.InstanceSize.XLARGE),
version: eks.KubernetesVersion.V1_21,
mastersRole: new iam.Role(this, 'MastersRole', { assumedBy: new iam.AccountRootPrincipal() })
});
const rootVolume: autoscaling.BlockDevice = {
deviceName: '/dev/xvda', // define the root volume
volume: autoscaling.BlockDeviceVolume.ebs(100), // override volume size
};
// IAM role for our EC2 worker nodes
const workerRole = new iam.Role(this, `${this.id}-workers` , {
assumedBy: new iam.ServicePrincipal('ec2.amazonaws.com')
});
const onDemandASG = new autoscaling.AutoScalingGroup(this, `${this.id}-asg`, {
vpc: vpc,
role: workerRole,
minCapacity: 1,
maxCapacity: 10,
desiredCapacity: 1,
blockDevices: [rootVolume],
instanceType: ec2.InstanceType.of(ec2.InstanceClass.M5, ec2.InstanceSize.XLARGE),
machineImage: new eks.EksOptimizedImage({
kubernetesVersion: '1.21',
nodeType: eks.NodeType.STANDARD // without this, incorrect SSM parameter for AMI is resolved
}),
updatePolicy: autoscaling.UpdatePolicy.rollingUpdate()
});
cluster.connectAutoScalingGroupCapacity(onDemandASG, {});
const dbSecurityGroup = new ec2.SecurityGroup(this, `${this.id}-db-sg`, {
vpc: vpc,
allowAllOutbound: true,
description: `db security group for ${this.id} db`,
securityGroupName: `${this.id}-db-sg`
});
dbSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(3306), 'MySQL access');
const db = new rds.ServerlessCluster(this, `${this.id}-db`, {
vpc: vpc,
defaultDatabaseName: `${this.env}`,
engine: rds.DatabaseClusterEngine.AURORA_MYSQL,
scaling: { autoPause: cdk.Duration.seconds(0) },
vpcSubnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS },
securityGroups: [dbSecurityGroup],
credentials: rds.Credentials.fromGeneratedSecret('root')
});
const redis = new elasticache.Cluster(this, `${this.id}-redis`, { vpc: vpc });
const mq = new sqs.Queue(this, `${this.id}-sqs`);
this.vpc = vpc;
this.cluster = cluster;
this.bastion = bastion;
this.redis = redis;
this.db = db;
this.mq = mq;
new cdk.CfnOutput(this, `${this.id}VpcId`, { value: this.vpc.vpcId})
new cdk.CfnOutput(this, `${this.id}ClusterArn`, { value: this.cluster.clusterArn})
new cdk.CfnOutput(this, `${this.id}DbArn`, { value: this.db?.clusterArn})
}
}
Key points in cluster.ts:
- EKS Cluster: Establishes an EKS cluster, specifying the version and instance types.
- Worker Nodes: Sets up Auto Scaling Groups for worker nodes, ensuring scalability.
- Aurora MySQL: Integrates a serverless Aurora MySQL cluster, providing a relational database.
- Redis Integration: Incorporates the Redis cluster setup from
redis.ts. - Additional Resources: Sets up other resources like SQS queues and bastion hosts for secure access.
CTO.ai - Orchestrating the Workflow
CTO.ai comes into play by providing the DevOps teams with tools to manage and automate workflows around the infrastructure defined by CDK. It can be used to enhance the deployment process, manage Kubernetes clusters, and monitor the performance of applications.
Conclusion
The combination of AWS CDK, EKS, Redis, and CTO.ai offers a powerful stack for building and managing cloud-native applications. CDK provides the infrastructure-as-code capability, EKS offers a managed Kubernetes service, Redis brings in high-performance data storage, and CTO.ai simplifies the entire DevOps cycle. This integrated approach not only enhances efficiency but also ensures the scalability and reliability of cloud-based applications, making it an ideal choice for modern application development and deployment.
Ready to unlock the power of CTO.ai for your team? Schedule your consultation now with one of our experts today!