Support CoW incremental query

[xushiyan]

No description provided.

[gohalo]

From the official docs, there are two ways to implement incremental queries.

1. Configuration passed by options, details Spark Incremental Query For Hudi-0.13.0
2. Through the hudi_table_changes TVF, details Spark Incremental Query For Hudi-0.14.1

Which method do you suggest using?

[jonathanc-n]

@xushiyan Hullo, I would like to work on this. The high level implementation would be to:

• Use timeline to retrieve latest commit or specific commit to use as a checkpoint
• Get metadata for commits
• Query on the changed data from that last check point.
  If there is any more specifics let me know!

[xushiyan]

thank you both for the interest! we will do table api support first for incremental query, and then move on to sql support using datafusion. i'll lay out some groundwork first before splitting more follow up tasks.

[jonathanc-n]

@xushiyan Is there anything I can help with for oroviding table api support?