VK Cloud logo

Deploying ML Models with MLflow Client

Manage MLflow Deploy instances using the MLflow Deployment Client — a Python library preinstalled in JupyterHub from VK Cloud.

Before you start

  1. Create a JupyterHub instance.
  2. Create an MLflow instance.
  3. Reboot the virtual machine with JupyterHub:
    1. Go to the Cloud Computing → Virtual Machines section.
    2. In the ML Platform Instances subsection, click  for the desired JupyterHub instance and select Reboot.

Preparing an ML model

  1. Connect to the JupyterHub instance.

  2. Go to the tutorials directory.

    JupyterHub from VK Cloud includes training Jupyter notebooks: mlflow_demo.ipynb and mlflow_deploy_demo.ipynb.

  3. Prepare a test ML model:

    1. Double-click on mlflow_demo.ipynb.
    2. In the window that opens, click on the cell with the example code and select Run → Run Selected Cells from the menu.
    3. Repeat the operation for all cells with code.

Managing an instance

To complete the entire scenario of the training Jupyter notebook:

  1. Double-click on mlflow_deploy_demo.ipynb.
  2. In the window that opens, click on the cell with the example code and select Run → Run Selected Cells from the menu.
  3. Repeat the operation for all cells with code.

Let us take a closer look at the operations performed with the deployment of the server and the Docker container.

Creating MlflowClient

Create an MlflowClient from the MLflow Tracking module to obtain the ML model URI. The URI will be needed later for deploying the ML model. The example shows accessing an ML model by URI.

from mlflow.tracking import MlflowClientcli = MlflowClient()

For the example, use the very first ML model:

model_source_uri = cli.search_registered_models()[0].latest_versions[0].sourceprint("ML model name: ", cli.search_registered_models()[0].latest_versions[0].name)print("ML model URI: ", model_source_uri)

Create a Client from the MLflow Deployments module to work in Cloud ML Platform:

from mlflow.deployments import get_deploy_clientclient = get_deploy_client('vk-cloud-mlplatform')

Creating an endpoint

endpoint in the terminology of VK Cloud MLflow Deploy is a VM configured as a deployment server.

deploy_server_name = "deploy_server_one"client.create_endpoint(name=deploy_server_name)

client.create_endpoint(name, performance="low", disk_size=50, disk_type="ceph-ssd", av_zone=None)

The full list of parameters is listed above. A deployment server can also be created specifying only the server name. In this case, the value of the av_zone parameter will be set to the zone where the associated MLflow service is located.

The perfomance parameter in the create_enpoint method is responsible for the virtual machine configuration. The following values are available:

  • low — standard 4-4 (4 cores, 4 GB of RAM);
  • mid — advanced 8-8 (8 cores, 8 GB of RAM);
  • high — advanced 16-16 (16 cores, 16 GB of RAM).

Getting the list and status of servers

The deployment server is ready for operation after its status changes from CREATING to RUNNING. Typically, preparing a deployment server takes about five to ten minutes.

client.list_endpoints()

Get information about the deployment server status by its name:

client.get_endpoint(deploy_server_name)

Creating a deployment

deployment in the terminology of VK Cloud MLflow Deploy is a running Docker container with an ML model on the deployment server.

deployment_name="test_deployment"client.create_deployment(server_name=deploy_server_name, deployment_name=deployment_name, model_uri=model_source_uri, port_out = None)

port_out can be omitted; the first free port in the range from 62000 to 65000 will be selected. Starting an ML model usually takes less than a minute.

Getting the list and status of deployments

  1. List the running ML models on the deployment server:

    client.list_deployments(deploy_server_name)
  2. Get information about the deployed ML model by the deployment server name and the ML model name:

    client.get_deployment(deploy_server_name, deployment_name)
  3. Use the predict method of the ML model in the Docker container:

    data = {"inputs":[[0.045341,  0.050680,  0.060618,  0.031065,  0.028702, -0.047347, -0.054446, 0.071210,  0.133597, 0.135612],[0.075341,  0.010680,  0.030618,  0.011065,  0.098702, -0.007347, -0.014446, 0.071210,  0.093597, 0.115612]]}client.predict(deploy_server_name, deployment_name, data)

Creating a deployment accessible via public DNS

  1. Set the username and password.

    auth_value = "user:PasswordDA@dvv//!123$"auth_deployment_name = "test_deploy_auth"client.create_deployment(deploy_server_name, auth_deployment_name, model_source_uri, auth=auth_value)
  2. Get information about the deployed ML model:

    deployment_info = client.get_deployment(deploy_server_name, auth_deployment_name)print(deployment_info)
  3. Get the DNS name for accessing the ML model:

    print(deployment_info['model_ref'])
  4. Create a request with authorization data (an example is given below):

    import requestsdata = {"inputs":[[0.045341,  0.050680,  0.060618,  0.031065,  0.028702, -0.047347, -0.054446, 0.071210,  0.133597, 0.135612],[0.075341,  0.010680,  0.030618,  0.011065,  0.098702, -0.007347, -0.014446, 0.071210,  0.093597, 0.115612]]}response = requests.post('https://ml-platform-3bb5b04ebb82c0.ml.msk.vkcs.cloud/deploy/0e84f86c-b9f0-4102-861d-222c41a81452/test_deploy_auth/invocations', json=data, auth=("user", "PasswordDA@dvv//!123$"))print(response.text)
  5. Use the predict method of the ML model in the Docker container:

    client.predict(deploy_server_name, auth_deployment_name, data)

Deleting a deployment

Delete the deployment from the deployment server by referencing the server name and the deployment:

client.delete_deployment(deploy_server_name, deployment_name)

Deleting a deployment

client.delete_endpoint(deploy_server_name)

Brief list of methods

from mlflow.deployments import get_deploy_clientclient = get_deploy_client('vk-cloud-mlpatform')# endpoint is a VM configured as a deployment serverclient.create_endpoint(name, performance="low", disk_size=50, disk_type="ceph-ssd", av_zone=None)client.list_endpoints()client.get_endpoint(server_name)# deployment is a running Docker container with an ML model on the deployment serverclient.create_deployment(server_name, deployment_name, model_uri, port_out = None)# port_out can be omitted; the first free port in the range from 62000 to 65000 will be selectedclient.list_deployments(server_name)client.get_deployment(server_name, deployment_name)# calling the predict method of the ML model in the Docker containerclient.predict(server_name, deployment_name, df_payload)client.delete_deployment(server_name, deployment_name)client.delete_endpoint(server_name)
Was this article helpful?