🕑 Estimated reading time: 8mn
Kubernetes is very complicated and that is okay. Even though it is present mostly everywhere today, most of its capacity remains unused. In order for Kubernetes to reach further, Rancher Labs built a lightweight version of it called K3s. With requirements of 200MB of storage and 512MB of RAM for the master and 75MB of RAM for each node, K3s is the perfect solution for resource-constrained environments. Not only that but it requires slightly less CPU resources than its fully featured counterpart.
From vanilla Kubernetes, K3s removes legacy and alpha features, most of the plugins (storage, cloud providers) and replaces etcd with SQLite, Docker with containerd and uses CoreDNS for service discovery, Flannel as the network fabric, Traefik as Ingress Controller and Helm.
With Air Gap, you can also run K3s with no internet access! With this feature, you could store images as tar.gz files and embed them with your portable K3s solution.
The next step of the way for K3s is its purpose-built OS, K3OS, which will bridge K3s and the underlying Linux distribution together. It is currently based on a combination of Alpine and Ubuntu. It is still heavy and could gain from optimization work and current progress on Unikernels.
If you wish to easily run K3s within Docker to try it out, you can do so with K3d.
Big cloud vendors seem to be choosing containerd instead of Docker for their workloads today. The exact reason is unknown but we can suppose that it may be related to the fact that containerd being Docker without overhead, it initializes and attaches volumes faster.
Helm comes preinstalled without Tiller. You can still preload charts and execute them when the cluster initializes.
A good release should be:
Inside Kubernetes, the classic workload organization places Pods inside ReplicaSets, which in turn are inside Deployments. The IngressController integrates the Services which direct the requests towards the pods they match.
Deploying is as simple as running the following command:
kubectl set image deployment/your-workload containername=newimage:newtag
This will start a rolling update:
It goes without saying: Liveness and Readiness probes must be implemented to help Kubernetes if, from the domain perspective, the pod is alive and running. Kubernetes supports HTTP, TCP and shell scripts to check the probes.
⚠️ If a pod cannot start fast enough, it will be automatically restarted by Kubernetes. To avoid that, configure the
initialDelaySeconds variable. The official documentation hints at other solutions, depending on your workload.
This mechanism is very simple, keeps the latest ReplicaSets alive, is Zero-Downtime compliant and transparent for Ops. Nevertheless, nothing is tested during the deployment. Indeed, something that works on your staging environment may not survive when running at production scale.
All of this can be done through labels and label selectors. The new deployment has a new label and your service will point to that new label once it is ready to be released.
By doing this, you are ensured the v1 is still present in case you need to roll back. Releasing the v2 only required a Service update, which also makes ZDD impossible because IPTables updates causes a period of service unavailability while reloading. That and the tests are not executed under production conditions.
v1 and v2 run in parallel and we actively choose which users go to v1 and which go to v2. There are multiple methods to implement Canary releases with Kubernees.
The Service knows v1 and v2 and evenly shares traffic. This method is not precise and wastes resources; to reach certain amounts like 27%, you will need 100 total pods running your application, 27 of which will be in the target version. Kubernetes will also work against your goal as the auto-scaling feature may slightly impact your release.
Ingress are increasingly clever and can absorb more logic. They could themselves handle the allocation of the traffic to the right version. Taefik and NginX can do that natively and offer support through annotations right from the Ingress resource.
Working with annotations is way safer than changing a resource specification as it does not trigger a full reload of that resource. Hence, this method conserves Zero Downtime Deployment, is officially supported by major Ingress controllers, gives you detailed control over different versions of your service and is unaffected by the autoscalability feature.
But IngressControllers cannot solve everything: while this approach works with services exposed to the outside through HTTP, internal services are not served by an IngressController, thus will not benefit from this approach.
Services will not communicate between them but will rather use a monitored and traced proxy layer. Istio can handle this with its Custom Resource Definitions, or CRDs.
VirtualService and DestinationRule are the CRDs to use. The first one defines the general traffic matching and routing while the second defines policies to apply after successful routing.
You could create a VirtualService with a route for the two versions of your application with specific weights and a DestinationRule which points to both versions of your application. Istio would then route requests taking your weights into account to create the right shares of traffic.
Unfortunately, Service Meshes add a high amount of complexity to your architecture, makes debugging more complex and is often considered overkill.
By using a proxy like Nginx, HAProxy or Envoy, you could manually perform canary releases. Even though this solution is very straightforward, you would have to manage and eventually automate it yourself. To do so, you could rely on Operators to extend Kubernetes. For instance, whenever a certain object is detected, automatically generate the proxy configuration and update.
If your testing process is not at least partially automated, do not deploy using Blue/Green as you would not be able to certify that the new version is properly working.
Similarly, if you do not collect and process metrics, do not move to canary deployments, as you need to analyze metrics generated at production scale to carry on or cancel your deployment.