VK Cloud logo

H2O LLM Studio Deployment

H2O LLM Studio is an open-source platform for training and fine-tuning large language models (LLM) through a convenient graphical interface. The platform allows you to run your own training pipelines, evaluate experiments, and manage model parameters without the need for programming. It is ideal for teams working with enterprise or scientific text data.

H2O LLM Studio features:

  • LLM training and fine-tuning: support for HuggingFace models (LLaMA, Falcon, Mistral, etc.) with the ability to run training on your data.
  • Visual interface: manage pipelines, hyperparameters, and experiments through a web interface.
  • Integration with H2O Wave: display of training progress, logs, and charts with results.
  • CPU and GPU support: select the desired VM instance configuration, from basic CPU to accelerated GPU training.
  • Configuration management: configure the environment via variables, configuration files, or the interface.
  • Compatibility with VK Cloud Monitoring: ability to connect instance system monitoring.

By using H2O LLM Studio, you agree to the license agreements:

To deploy the H2O LLM Studio service on a VM in your VK Cloud project:

  1. Sign up and go to the VK Cloud management console.

  2. To have access to the VM with the deployed service and to the H2O LLM Studio web interface from the Internet:

    1. Create a network with Internet access if it has not been created earlier.
    2. In the subnet settings, where the VM with the deployed service will be placed, disable the Private DNS option.
  3. Go to the App Store section.

  4. If there is no H2O LLM Studio service card on the page, click the All solutions button and find the card in the catalog.

  5. Click on the H2O LLM Studio service card.

  6. On the page with the service description, click the Connect button.

  7. Enter the data on the Server parameters page.

    1. Specify the H2O LLM Studio parameters:

      • How H2O LLM Studio will be deployed: select the service deployment type.

        • external — with an external IP address and a domain name provided by VK Cloud. Select this deployment type to access the VM with the deployed H2O LLM Studio service from the Internet.
        • internal — with access only within VK Cloud.
      • Enable backup to S3: enable the option to create daily backups of H2O LLM Studio data in the VK Cloud object storage. The storage is created automatically. The last 7 copies are retained.

      • IP for service access: specify the external IP address from which you plan to connect to the service, in IPv4 format without a mask, for example 255.255.255.255. The /32 mask will be added to the address automatically. Do not specify other mask values.

      • Enable monitoring: enable the option to automatically send metrics of the virtual machine where H2O LLM Studio is deployed to the Cloud Monitoring service. These metrics will be available in the Monitoring section of the management console.

    2. Specify the parameters of the virtual machine on which the service instance will be deployed:

      • Network: select the network where the VM with the deployed service will be placed. If you selected the external deployment type, choose a network with Internet access and a subnet for which the Private DNS option is disabled.
      • Availability zone: select the availability zone where the VM will be launched.
      • Virtual machine type: for the full-fledged operation of the H2O LLM Studio service, it is recommended to select a VM configuration template with a GPU. If there is no such template in the project, connect it.
    3. Specify the parameters of the system disk and the data disk.

      • Disk size: specify the desired VM disk size in gigabytes.
      • Disk type: select one of the disk types — HDD, SSD, or High-IOPS SSD.
  8. Click the Next step button.

  9. Review the cost of the infrastructure for deploying the service and click the Connect plan button.

    The cloud infrastructure deployment will begin, and the service properties page will open. The status of the process will also be displayed in the App Store → My services section.

    After successful deployment, messages will be sent to the email linked to your VK Cloud account:

    • a notification with a link to the service instance in the VK Cloud management console;
    • an email with a one-time link to the service access data.
  10. Follow the one-time link from the email and save the service access data:

    • h2o_url — a link to the H2O LLM Studio web interface.
    • ingress_ip — the IP address from which access to the H2O LLM Studio web interface and to the VM with the deployed service is allowed. This is the address you specified when connecting the service in the management console.
    • os_user — the name of the system user of the service instance VM for SSH login;
    • keypair — a private SSH key for connecting to the instance VM.
  11. Go to the H2O LLM Studio service web interface using the link from h2o_url.

    If the page does not open, check whether the IP address from which you are trying to access the web interface matches the IP address from ingress_ip.

    • If the IP addresses match, contact technical support.

    • If the IP addresses do not match, specify the correct IP address in the security group settings for the service:

      1. Open the security group named ХХХХ-h2o-llm-studio, where XXXX is the first part of your service instance identifier.
      2. Delete the incoming traffic rules for unneeded IP addresses and add new rules for the needed ones.
  12. (Optional) Use the H2O LLM Studio quickstart guide to get started with the service. If necessary, review the instructions on configuring models and pipelines.

Was this article helpful?