VK Cloud logo

Installation and configuration of the Monitoring service for Arenadata DB

Monitoring for Arenadata DB is a ready-made solution for monitoring Arenadata databases in the VK Cloud infrastructure. It includes a set of pre-installed tools for collecting metrics, visualization, and timely alerting on issues.

By completing all the steps of this guide, you will:

  1. Enable the Monitoring for Arenadata DB service in your VK Cloud management console.

    Upon activation, a virtual machine will be automatically created with the following installed on it:

    • Grafana — a service for visualizing and analyzing monitoring data;
    • Prometheus — a monitoring and alerting service;
    • ADB Exporter — a service for collecting metrics from Arenadata DB;
    • Alertmanager — an alert handler for alerts coming from Prometheus.
  2. Prepare the Arenadata database to work with the Monitoring for Arenadata DB service.

  3. Configure the ADB Exporter service.

  4. Install and configure Node Exporter — an agent for collecting operating system state metrics.

  5. Install and configure cAdvisor — an agent for collecting Docker container metrics.

  6. Log in to the Grafana service and view dashboards with data coming from Prometheus.

Preparation steps

  1. Create a Telegram chat to which alerts from the Monitoring for Arenadata DB service will be sent, and save the chat ID.

  2. Create a Telegram bot that will be used for sending alerts, and save the bot token.

  3. Register in VK Cloud if you have not done so earlier.

  4. Create an Arenadata DB instance. Save the data specified when creating the instance:

    • The Arenadata DB cluster name. It is also the instance name.
    • The Arenadata database name.
    • The name and password of the Arenadata database user.

    This data will be needed when connecting to and configuring the Monitoring for Arenadata DB service.

  5. Go to the Analytical DBs → ADB Instances section.

  6. Click the name of the desired instance.

  7. On the Information tab, in the Connecting settings row, find and save the following data:

    • The IP address of the Arenadata DB host — the value of the -h argument in the psql command.
    • The port for connecting to Arenadata DB — the value of the -p argument in the psql command.

    This data will be needed when connecting to and configuring the Monitoring for Arenadata DB service.

1. Enable the Monitoring for Arenadata DB service

  1. Go to the VK Cloud management console.

  2. Create a network with Internet access if it has not been created earlier.

  3. In the subnet settings, where the VM of the Monitoring for Arenadata DB service will be located, disable the Private DNS option.

  4. Go to the Application Store section and click the All solutions button on the section page.

  5. On the Monitoring for Arenadata DB service card, click the Learn more button.

  6. On the service description page, click the Activate button.

  7. Enter the data on the Server parameters page.

    1. Specify the parameters of the virtual machine where the service instance will be deployed:

      • Availability zone: select the data center where the VM will run.
      • Virtual machine type: select a pre-installed VM configuration.
      • Network: select a network with Internet access and a subnet for which the Private DNS option is disabled.
    2. Specify the parameters of the system disk and the data disk:

      • Disk size: specify the desired VM disk size in gigabytes.
      • Disk type: select one of the disk types — HDD, SSD, or High-IOPS SSD.
  8. Click the Next step button.

  9. Enter the data on the Monitoring parameters page.

    1. Specify the previously saved data for connecting to the Arenadata database:

      • the name of the Arenadata DB cluster (instance) to be monitored;
      • the IP address of the Arenadata DB cluster host to which the Monitoring for Arenadata DB service will connect;
      • the port for connecting to Arenadata DB;
      • the login and password of the Arenadata database user;
      • the name of the database in the Arenadata DB cluster.
    2. Set the login and password of the user for logging in to Grafana. This user will have Grafana administrator rights.

    3. Specify the previously saved Telegram bot parameters for alerts.

  10. Click the Next step button.

  11. Review the tariff terms and infrastructure costs for deploying the service and click the Activate plan button.

    After successful deployment, messages will be sent to the email registered in VK Cloud:

    • a notification with a link to the service instance in the VK Cloud management console;
    • an email with a one-time link.
  12. Follow the one-time link from the email and save the data for accessing the service:

    • grafanaUser and grafanaPassword — the login and password of the Grafana administrator;
    • productURL — the link to the Grafana web interface;
    • sshUser — the name of the administrator of the Monitoring for Arenadata DB service VM;
    • sshKey — the private key for connecting to the VM via SSH.

2. Connect to the VM of the Monitoring for Arenadata DB service

  1. Go to the VK Cloud management console.

  2. Go to the Cloud Computing → Virtual Machines section.

  3. Click the name of the VM starting with adb-monitoring-.

  4. On the VM page, copy and save the external IP address. It will be needed to connect to the Monitoring for Arenadata DB service VM.

  5. Create a vkc_monitoring.pem file on your computer and copy the SSH key received in the email into it.

  6. Set access to the vkc_monitoring.pem file so that only the owner has read and write permissions. To do this, run the command:

    chmod 600 vkc_monitoring.pem
  7. Connect to the Monitoring for Arenadata DB service VM by running the command:

    ssh -i <KEY_PATH>/vkc_monitoring.pem <LOGIN>@<VM_IP_ADDRESS>

    Where:

    • <KEY_PATH> — the path to the directory with the vkc_monitoring.pem file.
    • <LOGIN> — the VM administrator name received in the email (sshUser).
    • <VM_IP_ADDRESS> — the external IP address of the Monitoring for Arenadata DB service VM saved earlier.

3. Prepare Arenadata DB to work with the Monitoring for Arenadata DB service

  1. On the Monitoring for Arenadata DB service VM, connect to the Arenadata database. To do this, run the command, substituting the previously saved data:

    psql -h <DB_IP_ADDRESS> -p <PORT> -U <LOGIN> <DB_NAME>

    Where:

    • <DB_IP_ADDRESS> — the IP address of the Arenadata DB cluster host.
    • <PORT> — the port for connecting to Arenadata DB.
    • <LOGIN> — the login of the Arenadata DB user.
    • <DB_NAME> — the name of the database in the Arenadata DB cluster.
  2. Open the database initialization script /home/almalinux/init.sql for editing.

  3. Generate or come up with a password for the vkc_monitoring user of the Arenadata database, on whose behalf the Monitoring for Arenadata DB service will run.

  4. Insert this password into the /home/almalinux/init.sql script on the line:

    CREATE USER vkc_monitoring WITH login password `<ПАРОЛЬ>` RESOURCE GROUP monitoring_group CONNECTION LIMIT 5;
  5. Save the changes.

  6. Inside the psql terminal, run the database initialization script:

    \i /home/almalinux/init.sql
  7. Exit the psql terminal using the \q command.

4. Configure the ADB Exporter service

The ADB Exporter service is designed to export metrics from Arenadata databases to Prometheus. The service generates metrics based on descriptions in the configuration file and automatically updates their values according to a specified schedule by executing SQL queries against the database.

ADB Exporter is automatically installed and configured on the instance VM when the Monitoring for Arenadata DB service is deployed. To make ADB Exporter receive metrics from your Arenadata DB, specify the database access parameters in its configuration file:

  1. On the Monitoring for Arenadata DB service VM, open the ADB Exporter service configuration file:

    sudo nano /etc/adb_exporter/config.yml
  2. Add the Arenadata database connection parameters using the previously created vkc_monitoring user to the configuration file:

    datasource:   dsn: "postgresql://vkc_monitoring:<ПАРОЛЬ>@<IP-АДРЕС_БД>:<ПОРТ>/<ИМЯ_БД>?sslmode=disable"

    Where:

    • <ПАРОЛЬ> — the previously set password of the vkc_monitoring user.
    • <IP-АДРЕС_БД> — the IP address of the Arenadata DB cluster host.
    • <ПОРТ> — the port for connecting to Arenadata DB.
    • <ИМЯ_БД> — the name of the database in the Arenadata DB cluster.
  3. Restart ADB Exporter:

    sudo systemctl restart adb_exporter
  4. Check the ADB Exporter status:

    sudo systemctl status adb_exporter

    If the service has started successfully and is running, the command output will contain the line:

    Active: active (running)

5. Install and configure the Node Exporter agent

The Node Exporter agent collects system metrics from servers, including CPU, memory, disk, file system, and network interface usage. Without installing this agent, some Grafana dashboards and Prometheus alerts will not work.

  1. Install the Node Exporter agent on all nodes of the Arenadata DB cluster as described on the product page on GitHub.

  2. On the Monitoring for Arenadata DB service VM, open the Prometheus service configuration file:

    sudo nano /opt/monitoring_data/prometheus/prometheus.yml
  3. Add information about the metric sources to the configuration file: the cluster node IP addresses (targets) and metric labels (labels).

    Example of a Prometheus configuration:

    - job_name: 'node_exporter'  static_configs:    - targets: ['localhost:9100']      labels:        instance: segment-1        adb_cluster: staging
  4. Restart Prometheus.

    sudo systemctl restart prometheus
  5. Check the Prometheus status:

    sudo systemctl status prometheus

    If the service has started successfully and is running, the command output will contain the line:

    Active: active (running)

6. Install and configure the cAdvisor agent

The cAdvisor agent collects metrics on Docker container operation, including CPU, memory, disk I/O, and network traffic usage. Without installing this agent, some Grafana dashboards and Prometheus alerts will not work.

  1. Install the cAdvisor agent on all nodes of the Arenadata DB cluster where containerized applications are running. The installation procedure is described on the product page on GitHub.

  2. On the Monitoring for Arenadata DB service VM, open the Prometheus service configuration file:

    sudo nano /opt/monitoring_data/prometheus/prometheus.yml
  3. Add information about the metric sources to the configuration file: the cluster node IP addresses (targets) and metric labels (labels).

    Example of a Prometheus configuration:

    - job_name: 'docker_exporter'  metrics_path: /metrics  static_configs:    - targets: ['localhost:8080']      labels:        instance: adcc-node-1
  4. Restart Prometheus.

    sudo systemctl restart prometheus
  5. Check the Prometheus status:

    sudo systemctl status prometheus

    If the service has started successfully and is running, the command output will contain the line:

    Active: active (running)

7. Log in to the Grafana web interface

  1. Follow the productURL link saved earlier from the email. The Grafana web interface will open.

  2. Log in using the Grafana administrator login and password received in the email (grafanaUser and grafanaPassword).

  3. Review the contents of the dashboards displaying metrics collected by Prometheus.

Was this article helpful?