Skip to content

Adding additionalProperties in ArrowColumnHandle that can be used in remote datasources #26012

@Ajas-Mangal

Description

@Ajas-Mangal

Issue Details: In Presto, we have Arrow connector which uses Arrow Flight protocol to connect with remote datasources. Some IBM connectors like IBM GFS Connector extent Arrow connector to communicate with the Queryplex server—a backend service capable of connecting to multiple data sources and is responsible for federated query pushdown. This Queryplex server needs the remote database column definitions too along with the Presto generated sql to properly execute and pushdown queries.

Currently, ArrowColumnHandle contains the column name and the column type, where the type is converted from Queryplex metadata into a Presto-compatible type. These are used for holding query execution result in Presto. However, for Queryplex execution, it also need to propagate the remote column definition to the connector. To enable this, we need to introduce a new member variable in ArrowColumnHandle, to store additional informations like original metadata column definition.

Presto Component, Service, or Connector

Changes required in Arrow Connector service and the component name is presto-base-arrow-flight

Possible Implementation

Could add additionalProperties in ArrowColumnHandle, which holds all the additional properties in a map structure.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions