Hardware GPU partitioning with MIG
MIG (Multi-Instance GPU) allows you to partition the memory and compute resources of a single GPU into MIG devices — logical GPUs with a dedicated share of compute resources and memory, isolated at the hardware level according to the MIG profile used.
VK Cloud supports working with MIG in the Managed Containers service. MIG devices are created through the GPU Operator addon, after which they can be assigned to individual pods. For more information about configuring the addon, see the ссылка guide.
- Supported only on certain GPU models. For each supported model, there is a fixed set of MIG profiles from which you can create a configuration with its own set of MIG devices.
- You can change the MIG configuration only when none of the MIG devices are in use. To do this, delete the pods that use them in the Managed Containers cluster. After the configuration is changed, recreate the pods according to the new configuration.
The resources available to a MIG profile are specified in its name in the Xg.Ygb format:
Xgdefines the available share of compute resources — slices. Here,Xis their number.Ygbindicates the available video memory (VRAM). Here,Yis the memory size in gigabytes.
Some MIG profiles have the +me (media extensions) suffix with extended quotas for using media processing technologies: NVDEC, hardware acceleration for JPEG processing, OFA (Optical Flow Accelerator).
1g.18gb— Jupyter environments for developers, inference of small neural networks (up to 10–14 billion parameters in INT8), computer vision tasks (object recognition), audio processing.1g.18gb+me— H200 profile with an additional quota for media processing using OFA. Similar quotas are available in the7g.141gbconfiguration.1g.35gb— more complex models of up to 30 billion parameters in INT8 and datasets, or a larger video memory reserve than1g.18gb.2g.35gb— inference of medium-sized LLMs (14–30 billion parameters), decoding streaming video using NVDEC.3g.71gb/4g.71gb— inference of large models (from 70 billion parameters in INT8), full fine-tuning, batch processing of large data arrays.7g.141gb— maximum resources for a single task: the heaviest models, fine-tuning of models with 100+ billion parameters in INT4.
INT8 and INT4 are model weight quantization formats that save memory:
- INT8 — 1 byte per parameter: a model with 7 billion parameters takes about 7 GB.
- INT4 — 0.5 bytes per parameter: a model with 7 billion parameters takes about 3.5 GB.
Detailed specifications are available in the vendor's official documentation.