SDG Hackathon: Up and Running with Kubernetes, Crossplane, and ArgoCD
Written by Cooper Miller
The Solution Design Group (SDG) DevOps community recently hosted a Hackathon night where team members from various business units got together to eat pizza and hack through a project with unfamiliar technology! During the event, I had the opportunity to learn and use Crossplane to help create resources in Amazon Web Services (AWS) and enforce resource drift by automatically resolving state modifications. Along with having AWS resources defined in YAML and managed through a Kubernetes Cluster.
The project was simple. Each participant would create a project with Kubernetes, Crossplane, and ArgoCD to automate deploying and managing resources of our choosing in an AWS account. For my AWS resources, I wanted to see if I could create an AWS Lambda function that would send an inspirational quote to a channel in my personal Slack account. Okay… given the tools and goals, maybe it wasn’t so simple. To get a better picture of the project, let’s look at the tools I’ll be using.
The Tools
A widely adopted open-source container orchestration platform that can run and manage containerized applications for microservices, web applications, batch processing, and more
A not-so-widely used open-source tool that connects a Kubernetes cluster to external non-Kubernetes-based resources like a Cloud Provider (AWS, Google Cloud, Microsoft Azure) and more. It can also monitor and enforce the state of its external resources (like an S3 bucket) to address a problem with current “Infrastructure as Code” (IaC) tools: Drift! Which is when Cloud resources become out of sync with the IaC that’s supposed to represent them.
A GitOps tool that enables continuous deployments for applications running in Kubernetes. It connects to a Git repository and automatically checks for changes. If changes are found, it can automatically deploy them. For example, to automatically update a website, or in the case of this project, update AWS resources.
So, why would I want to participate in a Hackathon that’s using these tools? What problems are they solving? Why are they worth testing out? Well, Kubernetes is one of the most popular tools at the moment, making it a fantastic pick to use in a hackathon. Although Crossplane is less well known, it provided the inspiration for the Hackathon for its appeal of IaC with built-in state management. Now, we’re DevOps folks, so of course we needed a solution to automate deploying any changes we make. We can’t do that manually for goodness’ sake. So ArgoCD was the perfect tool, members had familiarity with what it was but hadn’t really had hands-on experience with it yet.
Creating the Project
To start, I followed the Crossplane “AWS Quickstart” guide’s recommendation to use Kind for creating a local Kubernetes cluster. I’d used Minikube in the past and was pleasantly surprised by how a single command created a fully functioning local Kubernetes cluster. Kind… kind of just works.
Next, I installed the Crossplane resources in the local cluster and was able to successfully connect Crossplane with my AWS account. At this point, I was able to create a YAML file that defined an AWS S3 bucket. By running a single Kubernetes CLI command and referencing the file, Crossplane took the data from the file and used it to easily create an AWS S3 bucket in my account!
I was now able to start installing the final piece of the hackathon puzzle, ArgoCD. I referenced their “Getting Started” guide and followed along to install the needed resources in my local cluster. After doing this, I was then able to access the ArgoCD User Interface that was created and was able to log in and link my project’s Git repository to create an “Application” that ArgoCD could manage.
At this point, I’d moved past the “Getting Started” guides and began digging into the Crossplane and Upbound Marketplace documentation. Crossplane leverages the “Upbound Marketplace” which holds the provider (AWS) specific configurations that are used to create resources.
While looking at the Upbound Marketplace, I knew I’d need to find the configurations to create an IAM role, policy, and a Lambda function. Looking through the Upbound AWS provider resource configuration descriptions and examples, I found they were generally confusing and not always clear on how they should be implemented. I think part of this was my unfamiliarity with the tool. Where I didn’t fully understand the attributes and structure of the configurations fully. If I’d been more familiar with them, it would have been easier to follow and modify resources. But since this was a hackathon and my time was running out, I wasn’t able to fully understand the format.
Ultimately, I was able to hack my way through creating the IAM role, policy, and Lambda function and successfully deploy them to AWS through ArgoCD. This was exciting to see! For my final MVP test, I was able to log into the AWS console, view my Lambda function, and run a “test invocation” that triggered the function to run. When the function ran, I received an inspirational quote sent to my personal Slack channel! Mission accomplished!
Overall, the SDG Hackathon was a fantastic opportunity to work alongside team members and hack together a mini project with new tools and technologies that we hadn’t used before. Coming from a background of primarily using Terraform for IaC, it was fun to work with Crossplane and see how the product differed in managing AWS infrastructure. I can see the appeal of using it to enforce resource drift by automatically resolving state modifications along with having AWS resources defined in YAML (“YAML everything!” as a coworker said in the event). I also saw some potential drawbacks with the tool. Since it isn’t widely used, there aren’t many active projects that can be referenced, and the documentation and the provider CRD examples were repetitive and had opportunities for improvement.
At the end of the day, Crossplane is in its early stages and I’m curious to see how it evolves. The documentation seemed to be okay in most cases and provided me with enough information to complete my MVP. If anyone’s interested in learning more and or following along with my hackathon project to recreate this for themselves, please reference my GitHub project with step-by-step instructions to get up and running with Kubernetes, Crossplane, and ArgoCD!