[Feature][Core] Enable the SPI factory classes to improve the entire process.

[ashulin]

Search before asking

• I had searched in the feature and found no similar feature requirement.

Description

Currently, the way we parse RowType is inconsistent.

These factories should be enabled to allow users to configure more easily and support more features;

Below I briefly explain the factory:

1. CatalogFactory: a feature with the same target as the user-defined schema, used to obtain the structured information of the table;
2. DeserializationFormatFactory & SerializationFormatFactory: used to support kafka, file and other unstructured connectors to process data in various formats;
3. TableSourceFactory: It is used to support multi-table reading and metadata column features. It is an upper-level extension of source;
4. TableSinkFactory: The same level as TableSourceFactory, but there are currently no supported features;

Usage Scenario

1. Provides a unified schema.
2. Support for metadata columns.
3. Support for CDC connector.
4. Support for format SPI.
5. Provide a list of options to the Web page.

Related requirement

unified schema：
#2469

create table automaticly:
#3271

metadata columns：
#2915
#2960

visualization of options:
#3273

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[Carl-Zhou-CN]

@ashulin hi, is it possible to identify the source of meta data in the schema definition and treat it as a normal field,Load when it needs to be processed

[ashulin]

@ashulin hi, is it possible to identify the source of meta data in the schema definition and treat it as a normal field,Load when it needs to be processed

@Carl-Zhou-CN That's how it's designed right now.

[hailin0]

To support cdc, we also need to support RowKind(delete/update) based on primary key define in sink

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.

[github-actions]

This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future.

[github-actions]

This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future.