Add-ons

Argo CD is a continuous delivery (CD) tool for Kubernetes that uses the GitOps approach. It automatically deploys and updates applications in the cluster based on their configuration in the Git repository. Argo CD helps to:

• Accelerate and simplify GitOps processes: it reduces the time needed to launch, scale, and deploy changes, as well as the complexity of implementing them.
• Improve the reliability and manageability of clusters: it minimizes manual steps and errors when changing cluster configurations. Argo CD automatically detects the differences and returns the cluster to the state described in its Git repository.
• Ensure actions in clusters are more transparent and improve cluster security by controlling and auditing all changes through Git.

For Argo CD to work correctly, it needs stable access to Git repositories that contain cluster configurations. The add-on must also have the rights to create applications in clusters that you specify through the Kubernetes Role-Based Access Control (RBAC) model. Otherwise, it might result in errors when attempting to synchronize the state of the cluster with its repository.

Kubernetes clusters allow you to organize the logical division of Kubernetes resources at the level of individual namespaces. However, this may not be enough to achieve resource separation and isolation in complex scenarios. For example, let's say you want to provide isolated sets of resources to multiple development teams so that they are not accessible to each other. A typical solution to this problem is to create several separate clusters for each team. With this approach, as the number of teams increases, the number of clusters also increases, which complicates the administration of these clusters.

Capsule allows you to organize isolated sets of resources within one cluster using tentants. An individual tenant represents namespaces assigned to a group of users combined with restrictions on the creation and consumption of Kubernetes resources. The Capsule policy engine not only monitors compliance with resource usage policies within a tenant, but also ensures the isolation of one tenant from another. Thus, it becomes possible to organize the work of several teams within one multi-tenant cluster without the need to administer additional clusters.

cert-manager helps to manage certificates in Kubernetes clusters:

• Issue certificates, including self-signed certificates. To do this, cert-manager sends requests to sources acting as certificate authority (CA).

  Examples of the sources:

  • Cybersecurity solutions providers such as Venafi.
  • Certificate providers, such as Let’s Encrypt.
  • Storage for secrets, such as HashiCorp Vault.
  • Local containers containing the public part of a certificate and private key.

• Automatically reissue expiring certificates.

A certificate issued with cert-manager will be available to other Kubernetes resources. For example, it can be used by Ingress.

Container Storage Interface for S3 (S3-CSI) is a Container Storage Interface driver for Kubernetes that allows to mount S3-compatible object storages (VK Object Storage, AWS S3, MinIO, and others) as PVs (persistent volumes).

S3-CSI is based on a high-performance file system GeeseFS powered by FUSE. The add-on is almost completely POSIX-compatible and supports the following actions with objects:

• read and write
• block
• rename and delete

S3-CSI operates at the bucket level of the object storage and can function in two modes:

• Dynamic provisioning of buckets. When creating a PVC (persistent volume claim) with a specified storage class, the add-on automatically creates a new bucket in the target S3-compatible storage. After creating a PVC, this bucket is immediately ready for use as a PV.
• Static provisioning of existing buckets. You can use a pre-created bucket in any S3-compatible storage. To do this, the add-on creates a PV with the name of the existing bucket, and then the PVC is associated with this PV.

S3-CSI can create buckets in any S3-compatible storage that you specify when installing the add-on.

Docker Registry is designed to host and store Docker images. It works in a high availability (HA) configuration. Registry images can be used when deploying services in a cluster.

See Connecting to the Docker registry for details.

Fluent Bit allows you to flexibly configure log collection in Cloud Containers clusters and analyze them in the Cloud Logging service, for example, using Elasticsearch or Loki plugins.

The sources of the logs are kubelet services and pods located on cluster nodes.

Fluent Bit in combination with special filters, written in Lua, allows you to organize the delivery of logs from the Cloud Containers cluster to the Cloud Logging service for further analysis of these logs.

The sources of the logs are kubelet services and pods located on cluster nodes. For more information about how the add-on works, see the sections on installing the add-on for first-generation and second-generation clusters.

GPU Operator allows you to manage GPUs on cluster nodes to perform machine learning or big data processing.

The following options for using GPUs in a cluster are available:

• One pod uses one or more GPUs.
• The add-on distributes one GPU between several pods using the MIG strategy.
• The add-on distributes one GPU between several pods using the MPS strategy.

add-on components:

• NVIDIA GPU Operator for GPU management.

• Service validators for CUDA (Compute Unified Device Architecture) validation after configuration changes.

• Operator self-diagnosis tools.

• NVIDIA device plugin for automating GPU resource binding and allocation.

• Node Feature Discovery for identifying and registering features available on cluster nodes. The component contains the following services:

  • NFD-Master is responsible for connecting to the Kubernetes API server and updating node objects.
  • NFD-Worker connects to the NFD-Master service to advertise hardware features.
  • NFD Garbage-Collector ensures that all Node Feature Discovery objects have corresponding nodes and removes stale objects for non-existent nodes.

Read more about the add-on and its components: NVIDIA GPU Operator, NVIDIA device plugin, Node Feature Discovery.

Ingress controller based on NGINX works as a reverse proxy and allows to organize single entry point for services in cluster which work via HTTP or HTTPS.

If you have a controller, it is sufficient to create Ingress resource to make such services available from outside the Cloud Containers cluster.

The pre-installed Ingress controller integrates tightly with the VK Cloud platform. For more information, see Network in cluster.

Istio allows you to monitor and manage the network interaction between your services without having to change the application code. Istio uses the service mesh approach, which means that all traffic between services goes through a managed proxy layer. The addon supports this as both a classic model with sidecar proxies at the pod level and the Ambient Mesh mode, which doesn't use sidecar containers (node-level proxies). Use Istio to:

• set security and traffic management policies between services
• use TLS for communication between services
• expand monitoring of inter-service traffic (metrics collection, logging, distributed tracing)
• configure complex routing and balancing of traffic between services
• configure query replays, timeouts, and other means of fault tolerance control
• use Canary Deployment and Blue-Green Deployment for gradual deployment

Installation of Istio is different for the first-generation and second-generation clusters.

In distributed systems based on microservices, requests are constantly exchanged. The Jaeger platform created for distributed query tracing. Jaeger tracks the flow of requests through microservices and allows you to:

• collect information about the interrelationships of the system components in terms of the flow of requests;
• detect query problems or bottlenecks in the system architecture related to processing the request stream.

Such a tool is necessary because query-related factors can significantly affect the behavior and performance of these systems as a whole. It is not enough to provide monitoring only for individual microservices.

Jaeger performs query tracing based on the data it receives from microservices. Therefore, it is necessary to integrate into microservices tool stack OpenTelemetry to send data about requests. You can get learn about the integration of OpenTelemetry into a microservice application using the example of Hot ROD in the Using Jaeger section.

Kgateway is a tool for routing and traffic management in Kubernetes via the Gateway API. The add-on:

• Provides publication of Kubernetes services and proxying requests to external services.
• Automatically scales to the load.
• Reduces the startup time of applications and services.
• Allows to centrally set entry points, TLS, routes, and access policies.

For more details on working with the add-on, refer to the Using Kgateway section.

Kiali is a web interface for working with Istio. It allows to manage, monitor and visualize a service mesh.

The system for monitoring the status of the cluster and the services deployed in it is implemented on the basis of Prometheus and visualization tool Grafana.

See Cluster Monitoring for details.