Add Apache Flink engine support

[ikyranas]

Motivation

Apache Flink is a popular framework offering querying capabilities and READ&WRITE operations to Apache Iceberg, Apache Hudi and Delta Lake. Unlike Spark, which can be used only for batch processing, Flink can be also used for streaming analytics applications.

Purpose

Is it of any interest to the team to integrate Apache Flink to the supported engines in the lst-bench?

Suggestion for PR

A PR to integrate Flink and be in accordance to the current repo's structure would need changes in the following:

• Include a Flink dependency and profile.
• Create a .github/flink folder with the setup scripts.
• Create a workflows for the jobs.
• Create the flink folder in run to include the configuration with accordance to trino or spark.
• Create FlinkConnection and FlinkConnectionManager classes in accordance to the Spark ones.
• Create the resources config for flink experiments.
• Create a new launcher script if necessary (???).

If this aligns with the team's plans I could create the PR for this.

[jcamachor]

@ikyranas , thanks for opening this issue. This would be fantastic; we would love to have support for other popular OSS engines in LST-Bench. Also, it would be very valuable to add new workloads that are representative for Flink and not currently covered by our implementation. Please let us know if you hit any blockers. Looking forward to your contribution!