Overview
Xenit Kubernetes Service (XKS) is an opinionated Kubernetes deployment on a public cloud provider's managed Kubernetes service. It combines a cloud provider's managed Kubernetes offering and ancillary services with additional configuration and services that run on top of Kubernetes.
Architecture​
XKS is set up from a set of Terraform modules that when combined creates the full XKS service. There are multiple individual states that all fulfill their own purpose and build upon each other in a hierarchical manner. The first setup requires applying the Terraform in the correct order, but after that ordering should not matter. Separate states are used as it allows for a more flexible architecture that could be changed in parallel.
Network diagram​
Looking at a cluster, the simple network diagram looks like this:
Terraform modules​
The following Terraform modules are used in XKS.
Governance​
Governance is split into global and regional, it handles the creation and delegation of Azure Resource Groups, Azure KeyVaults, Azure AD groups, Service Principals and resources like that.
Core​
Core sets up the main network for an environment.
Hub​
Hub is setup in the production subscription and is used for things like Azure Pipelines agents.
AKS​
The AKS Terraform contains three modules that are used to setup a Kubernetes cluster. To allow for blue/green deployments of AKS clusters resources have to be split up into global resources that can be shared between the clusters, and cluster specific resources.
The aks-global module contains the global resources like ACR, DNS and Azure AD configuration.
The aks and aks-core modules create a AKS cluster and configures it. This cluster will have a suffix, normally a number to allow for temporarily creating multiple clusters when performing a blue/green deployment of the clusters. Namespaces will be created in the cluster for each of the configured tenants. Each namespaces is linked to a resource group in Azure where namespace resources are expected to be created.
