Supporting Merge

[jonathanc-n]

I would like to propose implementing MERGE according to snowflake specification (here). I am not sure if the delete + update functionality is done in this repository, so for now only appends would be supported while the others would be left as unimplemented!(). This will be supported by the datafusion dataframe apis.

[jonathanc-n]

What are your thoughts @JanKaul ?

[JanKaul]

The incremental updates for materialized views are done with something like a merge operation. The pos_files are the parquet files containing new and updated entries and the neg_files are parquet files containing the equality deletes for the old values.

If MERGE will be supported by the datafusion dataframe apis, will there be a LogicalPlan Node for it? Currently I can't find it in DmlStatements.

[jonathanc-n]

I currently have a pull request out for a logical plan node here: apache/datafusion#15862. PTAL if possible.