VK Cloud logo

Cloud Trino in the Data Lakehouse architecture

Cloud Trino is a massively parallel analytical SQL engine for processing large volumes of data from various sources. It allows you to:

  • run SQL read queries in parallel from multiple sources and storage systems;
  • organize ETL and ELT pipelines without using specialized services in the DLH architecture.

Cloud Trino makes it possible to move analytical workloads from expensive-to-maintain and hard-to-scale enterprise data warehouses (for example, based on Greenplum) to cheaper data lakes and DLH.

In the Data Lakehouse architecture, the component performs the following functions:

  • Fetching data from sources using the PULL model (for simple scenarios).

  • Performing read and write operations on data in the S3 object storage via the Apache Iceberg API.

  • Load balancing of the incoming flow of SQL queries from sources and consumers in the DLH.

  • Publishing information about DLH data sources and structure in an external metadata catalog for data management and data lineage tracking.

  • Providing data to external service requests directly from S3 storage for report generation, BI analytics, and machine learning.

  • Interacting with specialized analytical DBMSs (for example, ClickHouse, PostgreSQL) to perform real-time analytics.

  • Multi-phase query plan optimization — building a fixed sequence of actions when executing an SQL query:

    • applying filters;
    • choosing the optimal order using JOIN;
    • replacing tables in the query with a view (VIEW);
    • performing data transformation operations.
Was this article helpful?