Easily Run Data Science Workflows in Slack using Python, Jupyter & AWS EKS

by Kyle Campbell

Last month, I had the opportunity to present at the second virtual SF Python Meetup about how CTO.ai is making it easier for Python developers and Data Scientists to streamline their critical workflows in Slack.

Going into the meetup, I admittedly didn't know that much about how this community managed their workflows, but one of our prior team members Ethan had built a really awesome automation with Jupyter on top of AWS EKS.

When the meetup was announced, I found that Aly Sivji was doing a 45 minute keynote about almost the exact same topic! Initially I thought, oh no, this is awkward, but then I realized it was a good opportunity to better demonstrate how developers can find efficiency in their workflows via The Ops Platform.

Here is Aly's talk from the meetup, where he does an amazing job to explain the intricacies of how to use Docker in your workflow, which is a great overview for anyone who wants to understand this process from first principals:

Aly's talk is awesome. He goes deep into all of the important aspects of how to use Docker in your workflows and why you can benefit from containerized technology to easily customize a portable environment with your team.

Building on this, the talk I did was intended to demonstrate how CTO.ai is making these sorts of workflows dead simple, while also enhancing this natural paradigm with increased portability / accessibility / transparency - via Slack. We bring 12 factor application principals via Secrets, Configs, Logs, Events and (very soon) Metrics that "just work" for anyone who wants to streamline their team workflows.

Here is the talk I did which shows a similar process of creating a Jupyter playbook on AWS EKS / Kubernetes from Slack, in about 5 minutes total.

Both of these presentations show the power of Cloud Native workflows when it comes to streamlining the developer experience for teams, especially for complex tasks, that often need to leverage the exponential compute power of the cloud.

At the same time, you can see how complex these workflows can be when adopted from first principals, relative to how easy they can be with an approach like CTO.ai, which is purpose built to empower the developer experience, is used in your team.

Our goal is to save developers times and help them be more focused on the most meaningful work that they can deliver, in turn driving more success for their team and their own careers. This is what we think of as 10x development.

10x development is helping 5 people be 2x more successful.

If you have any questions about this or have some ideas for interesting Data Science workflows, please feel free to send me a message via our Slack Community.

We're really excited about how we can do more workflows like this for Data Science use cases, so we'd love to collaborate with you if you have some ideas!