Skip to content

Commit

Permalink
#170 Add renaming transformer usage description to README.md.
Browse files Browse the repository at this point in the history
  • Loading branch information
yruslan committed Oct 30, 2020
1 parent 5860169 commit 239ee57
Showing 1 changed file with 13 additions and 0 deletions.
13 changes: 13 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -159,6 +159,19 @@ Caution: This transformer requires a writer which defines `writer.parquet.destin
| :--- | :---: | :--- |
| `transformer.{transformer-id}.report.date` | No | User-defined date for `hyperdrive_date` in format `yyyy-MM-dd`. Default date is the date of the ingestion |

##### ColumnRenamingStreamTransformer
`ColumnRenamingStreamTransformer` allows renaming of columns specified in the configuration.

To add the transformer to the pipeline use this class name:
```
component.transformer.class.{transformer-id} = za.co.absa.hyperdrive.ingestor.implementation.transformer.column.renaming.ColumnRenamingStreamTransformer
```

| Property Name | Required | Description |
| :--- | :---: | :--- |
| `transformer.{transformer-id}.columns.rename.from` | Yes | A comma-separated list of columns to rename. For example, `column1, column2`. |
| `transformer.{transformer-id}.columns.rename.to` | Yes | A comma-separated list of new column names. For example, `column1_new, column2_new`. |

See [Pipeline settings](#pipeline-settings) for details about `{transformer-id}`.
##### ParquetStreamWriter
| Property Name | Required | Description |
Expand Down

0 comments on commit 239ee57

Please sign in to comment.