diff --git a/docs/en/concept/schema-feature.md b/docs/en/concept/schema-feature.md new file mode 100644 index 00000000000..88120075f02 --- /dev/null +++ b/docs/en/concept/schema-feature.md @@ -0,0 +1,62 @@ +# Intro to schema feature + +## Why we need schema + +Some NoSQL databases or message queue are not strongly limited schema, so the schema cannot be obtained through the api. At this time, a schema needs to be defined to convert to SeaTunnelRowType and obtain data. + +## What type supported at now + +| Data type | Description | +| :-------- | :----------------------------------------------------------- | +| string | string | +| boolean | boolean | +| tinyint | -128 to 127 regular. 0 to 255 unsigned*. Specify the maximum number of digits in parentheses. | +| smallint | -32768 to 32767 General. 0 to 65535 unsigned*. Specify the maximum number of digits in parentheses. | +| int | All numbers from -2,147,483,648 to 2,147,483,647 are allowed. | +| bigint | All numbers between -9,223,372,036,854,775,808 and 9,223,372,036,854,775,807 are allowed. | +| float | Float-precision numeric data from -1.79E+308 to 1.79E+308. | +| double | Double precision floating point. Handle most decimals. | +| decimal | DOUBLE type stored as a string, allowing a fixed decimal point. | +| null | null | +| bytes | bytes. | +| date | Only the date is stored. From January 1, 0001 to December 31, 9999. | +| time | Only store time. Accuracy is 100 nanoseconds. | +| timestamp | Stores a unique number that is updated whenever a row is created or modified. timestamp is based on the internal clock and does not correspond to real time. There can only be one timestamp variable per table. | +| row | Row type,can be nested. | + +## How to use schema + +`schema` defines the format of the data,it contains`fields` properties. `fields` define the field properties,it's a K-V key-value pair, the Key is the field name and the value is field type. Here is an example. + +``` +source { + FakeSource { + parallelism = 2 + result_table_name = "fake" + row.num = 16 + schema = { + fields { + id = bigint + c_map = "map" + c_array = "array" + c_string = string + c_boolean = boolean + c_tinyint = tinyint + c_smallint = smallint + c_int = int + c_bigint = bigint + c_float = float + c_double = double + c_decimal = "decimal(2, 1)" + c_bytes = bytes + c_date = date + c_timestamp = timestamp + } + } + } +} +``` + +## When we should use it or not + +If there is a `schema` configuration project in Options,the connector can then customize the schema. Like `Fake` `Pulsar` `Http` source connector etc. diff --git a/docs/sidebars.js b/docs/sidebars.js index c7da60b3ffa..bc10eca4b82 100644 --- a/docs/sidebars.js +++ b/docs/sidebars.js @@ -93,7 +93,8 @@ const sidebars = { "label": "Concept", "items": [ "concept/config", - "concept/connector-v2-features" + "concept/connector-v2-features", + 'concept/schema-feature' ] }, {