Edge computing is becoming increasingly important. In this blog, we would like to explain how Edge deployments can be simplified using EMCO and GitOps.
What is EMCO and its significance in the Edge cloud sector?
The Edge Multi-Cluster Orchestrator (EMCO) is an open-source project, part of the Linux Foundation Networking umbrella backed by companies including Intel, Equinix, Nokia, Aarna Networks, and others. It is a software framework for the intent-based deployment of cloud-native applications to a set of Kubernetes clusters, spanning enterprise data centers, multiple cloud service providers, and numerous edge locations. It can be leveraged for Private 5G, O-RAN (Open Radio Access Networks), and multi-access edge computing (MEC) applications. It has many unique features like extensibility via an intelligent selection of clusters to place the workloads, tenant Isolation using logical clouds, automation of service mesh and other connectivity & security infrastructure, updates, and rollbacks.
In brief, we can say it’s a one-stop solution for edge deployments using Kubernetes, and what makes it better is being open-sourced, it is completely free.
What is GitOps and what are its advantages?
In simple words, GitOps is a set of practices to manage infrastructure and application configurations using the Git version control system. It seeks to bring the advantages of git together with what are now becoming more standard tools and concepts to the DevOps and Automation world like Infrastructure as Code, Merge Requests/Pull Requests, and Continuous Integration and Delivery/Deployment.
To elaborate, in a basic GitOps flow the user pushes all the resources to the git repo considered to contain the entire state of the system. And usually, an agent like FluxCD or ArgoCD running inside the environment continuously polls this Git repo and/or container registry for changes. As soon as it detects a mismatch between the defined state and the running state, the agent pulls the defined configuration into the environment. A few major advantages of using this mode of deployment are
- Reduced security and compliance risk: Because the agent is running inside the cluster, there is no need to store credentials externally.
- Consistency: Due to the poll and compare mechanism followed by the agents it can detect and remediate configuration drift if changes are made to the cluster manually.
GitOps flow
The following figure shows a typical GitOps flow.
Let us try to understand this GitOps flow, which has three actors: a developer (dev), Operations Engineer (Ops), and an Infrastructure Engineer (Infra). Let us cycle through steps 0-5 and understand what actions are being taken by these three actors.
- In step 0, the infrastructure engineer creates the Kubernetes cluster with a GitOps controller like FluxCD/ArgoCD. This GitOps controller is watching the Git Deploy Repository for changes to synchronize.
- In step 1a, the developer pushes the application changes to the Git App Repository, which triggers step 2 docker build. Once the build gets completed, step 3 kicks in, and the images get pushed to a package repository.
- After this, step 4 gets triggered and updates the Git Deploy repository with these new images.
- As mentioned earlier, the GitOps controller deployed in the Kubernetes continuously watches the Git Deploy repository for changes. As soon as the new images get updated, the GitOps controller picks up the changes as shown in step 5 and updates the workload deployed in the Kubernetes cluster.
Now each time the developer pushes in changes these steps are repeated, thus ensuring that the deployments in the cluster always remain in sync with the latest updates published in the Git repository.
How well does EMCO complement GitOps?
In the context of a multi-cluster, multi-app edge/cloud deployment, such deployments, which involve deploying multiple applications on different cluster types located in different environments, add complexity to management, security, and consistency. GitOps simplifies the process but lacks the intelligence to solve these complexities. So, EMCO can become the intelligent agent that writes resources to the git location to solve these complexities and accomplish all the below-mentioned points.
- On-demand instantiation of applications on K8s clusters
- Workload placement using intelligent cluster selection.
- On-demand scale-out (bursting) of the applications
- Customization of resources to the applications
- Automation of service mesh and other connectivity & security infrastructure
- Dependency and order of priority of application deployments between clusters
What GitOps Plugins are available in EMCO?
GitOps pattern is being adopted in many public and private clouds. FluxCD and ArgoCD are two Kubernetes-native applications that facilitate and help enforce the GitOps pattern. Azure supports GitOps on an Azure Arc-enabled Kubernetes cluster, and GCP (Google Cloud Platform) supports GitOps with Anthos.
EMCO provides GitOps plugins for
- Azure Arc-enabled Kubernetes clusters
Azure Arc-enabled Kubernetes allows you to attach and configure Kubernetes clusters running anywhere.
- FluxCD
FluxCD is a set of continuous and progressive delivery solutions for Kubernetes that are open and extensible.
- Google Anthos, and plans to support various other vendors and technologies with its extensible design.
Please feel free to read more about Azure-Arc enabled Kubernetes Clusters and FluxCD
Demo examples
We have a few demo examples to demonstrate how to use GitOps with Azure Arc-enabled clusters, clusters with standalone FluxCD controllers, and Google Anthos using EMCO as the intelligent agent. Please feel free to try it.
The EMCO orchestrator supports placement and action controllers to control the deployment of applications. Placement controllers allow the orchestrator to choose the exact locations to place the application in the composite application. Action controllers can modify the state of a resource (create additional resources to be deployed, modify, or delete the existing resources).
GAC is an action controller registered with the central orchestrator. GAC allows you to create a new Kubernetes object and deploy it with a specific application. The app should be part of the composite application. GAC allows you to modify an existing Kubernetes object which may have deployed using the helm chart for an app or GAC itself. Modification may correspond to specific fields in the YAML definition of the Kubernetes object.
Components of GAC
- GenericK8sIntent - Specifies the parent intent supports the resource and their customizations.
- Resource - Specifies the Kubernetes object that needs to be created. You can create a resource template file with all the required configurations for the resource and upload it. GAC reads the configurations and applies them to the resource. A resource template file is mandatory except for ConfigMap/Secret.
- Customizations - Specifies the configurations for the Kubernetes object. Customization allows you to customize an existing Kubernetes object or a newly created one. Customization supports uploading multiple data files with configuration data for a ConfigMap/Secret. Customization also helps modify existing Kubernetes objects using a JSON patch or Kubernetes strategic merge patch.
- JSON patch - GAC supports the standard JSON patch. You can modify the specific fields in the Kubernetes resource definition using the JSON patch. You need to specify the path, operation, and value in the GAC customization. You can also create a ConfigMap/Secret using the JSON patch. You can pass the JSON patch data in the customization. GAC will create the ConfigMap/Secret using the values provided in the resourceGVK (APIVersion, Kind, Name) and apply the configuration data provided in the JSON patch.
- Strategic Merge patch – GAC is integrated with the Kubernetes Strategic Merge patch. This is specific to Kubernetes and is an extension of the standard JSON merge patch. The main advantage of using a Strategic Merge patch over a JSON patch or JSON Merge patch is to merge lists strategically. You can only modify specific fields using a standard JSON patch. A simple JSON Merge patch will always replace the list. For example, the container in a Deployment is a list. A simple JSON Merge patch will replace the original document's containers with the patch's list. Strategic Merge internally uses the go struct tag to determine how the patch should be applied (merge/delete/replace). Since the containers field in the go struct has a declarative tag patchStrategy=merge, and patchMergeKey=name when we apply the Strategic Merge patch, the list will be merged, not replaced. This is feasible only for specific fields. Please see notes on the strategic merge patch for more details.
Please refer to GAC design document for more details
Please see the
Generic Controller Intentsection in the OpenAPI for the detailed definition of APIs exposed by GAC.Please refer to Creating Kubernetes Objects Using GAC for the step by step guide to create a Kubernetes resource using GAC
Please refer to EMCO Resource Lifecycle and Status for more details on the GAC resources lifecycle.
Please see the examples in test-gac for a complete walkthrough of GAC customizations
Please refer to the Kubernetes official Strategic Merge Patch documentation for more details about Strategic Merge Patch.