Add the CLP connector RFC #37
Conversation
tdcmeehan
left a comment
tdcmeehan
left a comment
There was a problem hiding this comment.
This all seems pretty reasonable to me. There doesn't seem to be anything that needs to change in the SPI, runtime or execution engines besides things that are at the connector level.
One suggestion is to keep the CLP C++ connector in Presto for now. We can always move it to Velox later, and it will make the landing of this code more predictable.
|
|
||
| ## Table & Column Resolution | ||
|
|
||
| Implementing `ConnectorMetadata` requires determining what tables and columns exist in a CLP dataset, which in turn requires querying CLP's metadata database. Since different users may have their own implementation of a CLP metadata database, the connector should expose the required methods through a `ClpMetadataProvider` interface. For table and column resolution, the interface should contain the following methods: |
There was a problem hiding this comment.
To be clear, ClpMetadataProvider lives in the CLP connector, not the runtime?
Thanks for the suggestion! We'll spend some time working on it. |
4 participants
This PR adds an RFC for the CLP connector, which enables SQL queries on the compressed CLP archives.