Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Default Ingestion Configuration Logging and Update Documentation to Reflect Runtime Naming Convention #49

Merged
merged 8 commits into from
Mar 27, 2024
Merged

Add Default Ingestion Configuration Logging and Update Documentation to Reflect Runtime Naming Convention #49

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
b657d95
Change all references for pre-compiled binaries in the documentation …
forestmvey Mar 27, 2024
9fce2f3
Change configuration default logs to debug level, and added log for l…
forestmvey Mar 27, 2024
d160998
Updating logs to level info for ingestion configurations.
forestmvey Mar 27, 2024
3ade8a1
Fix typo and grammar.
forestmvey Mar 27, 2024
5f937ab
Updated log for query client in lambda handler.
forestmvey Mar 27, 2024
e9c5079
Added check when running binary that default database and table value…
forestmvey Mar 27, 2024
674d49b
Revising initialization log format.
forestmvey Mar 27, 2024
c0b7832
Merge pull request #18 from forestmvey/dev-improve-logging-and-docume…
forestmvey Mar 27, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 13 additions & 17 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -133,20 +133,16 @@ This is the easiest and recommended method for running the connector.

### Run with Precompiled Binaries

Run the precompiled binaries with required arguments `default-database` and `default-table`.
The pre-compiled binaries independent of platform will have the name `bootstrap` to align with the `provided.al2023` lambda runtime naming convention. Run the precompiled binaries with required arguments `default-database` and `default-table`.

| Platform | Command |
| -------- | ------------------------------------------------------------ |
| Linux | `./timestream-prometheus-connector-linux-amd64-1.0.0 --default-database=prometheusDatabase --default-table=prometheusMetricsTable` |
| macOS | `./timestream-prometheus-connector-darwin-amd64-1.0.0 --default-database=prometheusDatabase --default-table=prometheusMetricsTable` |
| Windows | `timestream-prometheus-connector-windows-amd64-1.0.0 --default-database=prometheusDatabase --default-table=prometheusMetricsTable` |
`./bootstrap --default-database=prometheusDatabase --default-table=prometheusMetricsTable`

It is recommended to secure the Prometheus requests with TLS encryption. To enable TLS encryption:

1. Specify the server certificate and the server private key through the `tls-certificate` and `tls-key` configuration options. An example for macOS is as follows:

```shell
./timestream-prometheus-connector-darwin-amd64-1.0.0 \
./bootstrap \
--default-database=prometheusDatabase \
--default-table=prometheusMetricsTable \
--tls-certificate=serverCertificate.crt \
Expand All @@ -163,7 +159,7 @@ For more examples on configuring the Prometheus Connector see [Configuration Opt

The following error message may show up when running the precompiled binary on macOS:

`"timestream-prometheus-connector-darwin-amd64" cannot be opened because the developer cannot be verified.`
`"bootstrap" cannot be opened because the developer cannot be verified.`

Follow these steps to resolve:

Expand Down Expand Up @@ -391,7 +387,7 @@ To provide access to this newly created role, add a permission to the current us
Go to [Configuration Options](#configuration-options) to see more information.
9. Scroll down to basic settings.
10. Click `Edit`.
11. In the `Handler` section, enter the name of the Amazon Timestream Prometheus Connector ZIP file, which will be `timestream-prometheus-connector-1.0.0`.
11. In the `Handler` section, enter the name of the Amazon Timestream Prometheus Connector ZIP file, which will be `bootstrap`.
12. Click `Save`.

#### Create the API on Amazon API Gateway
Expand Down Expand Up @@ -544,28 +540,28 @@ The default-database name and default-table name are required for data ingestion

| Runtime | Command |
| -------------------- |-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Precompiled Binaries | `./timestream-prometheus-connector-linux-amd64-1.0.0 --default-database=PrometheusDatabase --default-table=PrometheusMetricsTable --region=us-west-2` |
| Precompiled Binaries | `./bootstrap --default-database=PrometheusDatabase --default-table=PrometheusMetricsTable --region=us-west-2` |
| AWS Lambda Function | `aws lambda update-function-configuration --function-name PrometheusConnector --environment "Variables={default_database=prometheusDatabase,default_table=prometheusMetricsTable,region=us-west-2}"` |

2. Configure the Prometheus Connector listen for requests on an HTTPS server `https://localhost:9201` with TLS encryption.

| Runtime | Command |
| -------------------- |------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Precompiled Binaries | `./timestream-prometheus-connector-linux-amd64-1.0.0 --default-database=PrometheusDatabase --default-table=PrometheusMetricsTable --tls-certificate=serverCertificate.crt --tls-key=serverPrivateKey.key` |
| Precompiled Binaries | `./bootstrap --default-database=PrometheusDatabase --default-table=PrometheusMetricsTable --tls-certificate=serverCertificate.crt --tls-key=serverPrivateKey.key` |
| AWS Lambda Function | `N/A` |

3. Configure the Prometheus Connector to listen for Prometheus requests on `http://localhost:3080`.

| Runtime | Command |
| -------------------- |----------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Precompiled Binaries | `./timestream-prometheus-connector-linux-amd64-1.0.0 --default-database=PrometheusDatabase --default-table=PrometheusMetricsTable --web.listen-address=:3080` |
| Precompiled Binaries | `./bootstrap --default-database=PrometheusDatabase --default-table=PrometheusMetricsTable --web.listen-address=:3080` |
| AWS Lambda Function | `N/A` |

4. Configure the Prometheus Connector to listen for Prometheus requests on `http://localhost:3080` and serve collected metrics to `http://localhost:3080/timestream-metrics`.

| Runtime | Command |
| -------------------- |---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Precompiled Binaries | `./timestream-prometheus-connector-linux-amd64-1.0.0 --default-database=PrometheusDatabase --default-table=PrometheusMetricsTable --web.listen-address=:3080 --web.telemetry-path=/timestream-metrics` |
| Precompiled Binaries | `./bootstrap --default-database=PrometheusDatabase --default-table=PrometheusMetricsTable --web.listen-address=:3080 --web.telemetry-path=/timestream-metrics` |
| AWS Lambda Function | `N/A` |

### Retry Configuration Options
Expand All @@ -582,7 +578,7 @@ Configure the Prometheus Connector to retry up to 10 times upon recoverable erro

| Runtime | Command |
| -------------------- |---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Precompiled Binaries | `./timestream-prometheus-connector-linux-amd64-1.0.0 --default-database=PrometheusDatabase --default-table=PrometheusMetricsTable --max-retries=10` |
| Precompiled Binaries | `./bootstrap --default-database=PrometheusDatabase --default-table=PrometheusMetricsTable --max-retries=10` |
| AWS Lambda Function | `aws lambda update-function-configuration --function-name PrometheusConnector --environment "Variables={default_database=prometheusDatabase,default_table=prometheusMetricsTable,max_retries=10}"` |

### Logger Configuration Options
Expand Down Expand Up @@ -610,21 +606,21 @@ To quickly spot and resolve issues that may be caused by ignored Prometheus time

| Runtime | Command |
| -------------------- |--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Precompiled Binaries | `./timestream-prometheus-connector-linux-amd64-1.0.0 --default-database=PrometheusDatabase --default-table=PrometheusMetricsTable --enable-logging=false` |
| Precompiled Binaries | `./bootstrap --default-database=PrometheusDatabase --default-table=PrometheusMetricsTable --enable-logging=false` |
| AWS Lambda Function | `aws lambda update-function-configuration --function-name PrometheusPrometheus Connector --environment "Variables={default_database=prometheusDatabase,default_table=prometheusMetricsTable,enable_logging=false}"` |

2. Toggle the Prometheus Connector to halt on: <br />- label names exceeding the maximum length supported by Amazon Timestream;<br />- Prometheus time series with non-finite values.

| Runtime | Command |
| -------------------- |-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Precompiled Binaries | `./timestream-prometheus-connector-linux-amd64-1.0.0 --default-database=PrometheusDatabase --default-table=PrometheusMetricsTable --fail-on-long-label=true --fail-on-invalid-sample=true` |
| Precompiled Binaries | `./bootstrap --default-database=PrometheusDatabase --default-table=PrometheusMetricsTable --fail-on-long-label=true --fail-on-invalid-sample=true` |
| AWS Lambda Function | `aws lambda update-function-configuration --function-name PrometheusConnector --environment "Variables={default_database=prometheusDatabase,default_table=prometheusMetricsTable,fail_on_long_label=true, fail_on_invalid_sample_value=true}"` |

3. Configure the Prometheus Connector to output the logs at debug level and in JSON format.

| Runtime | Command |
| -------------------- |---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Precompiled Binaries | `./timestream-prometheus-connector-linux-amd64-1.0.0 --default-database=PrometheusDatabase --default-table=PrometheusMetricsTable --log.level=debug --log.format=json` |
| Precompiled Binaries | `./bootstrap --default-database=PrometheusDatabase --default-table=PrometheusMetricsTable --log.level=debug --log.format=json` |
| AWS Lambda Function | `aws lambda update-function-configuration --function-name PrometheusConnector --environment "Variables={default_database=prometheusDatabase,default_table=prometheusMetricsTable,log_level=debug, log_format=json}"` |

## Relabel Long Labels
Expand Down
19 changes: 14 additions & 5 deletions main.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -117,14 +117,14 @@ func main() {
awsWriteConfigs.MaxRetries = aws.Int(writeClientMaxRetries)
timestreamClient.NewWriteClient(logger, awsWriteConfigs, cfg.failOnLongMetricLabelName, cfg.failOnInvalidSample)

timestream.LogInfo(logger, "Successfully created Timestream clients to handle read and write requests from Prometheus.")

timestream.LogInfo(logger, fmt.Sprintf("Timestream connection is initialized (Database: %s, Table: %s, Region: %s)", cfg.defaultDatabase, cfg.defaultTable, cfg.clientConfig.region))
// Register TimestreamClient to Prometheus for it to scrape metrics
prometheus.MustRegister(timestreamClient)

writers = append(writers, timestreamClient.WriteClient())
readers = append(readers, timestreamClient.QueryClient())

timestream.LogInfo(logger, "The Prometheus Connector is now ready to begin serving ingestion and query requests.")
if err := serve(logger, cfg.listenAddr, writers, readers, cfg.certificate, cfg.key); err != nil {
timestream.LogError(logger, "Error occurred while listening for requests.", err)
os.Exit(1)
Expand Down Expand Up @@ -184,8 +184,7 @@ func handleWriteRequest(reqBuf []byte, timestreamClient *timestream.Client, awsC

createWriteClient(timestreamClient, logger, awsConfigs, cfg.failOnLongMetricLabelName, cfg.failOnInvalidSample)

timestream.LogInfo(logger, "Successfully created a Timestream write client to handle write requests from Prometheus.")

timestream.LogInfo(logger, fmt.Sprintf("Timestream write connection is initialized (Database: %s, Table: %s, Region: %s)", cfg.defaultDatabase, cfg.defaultTable, cfg.clientConfig.region))
if err := getWriteClient(timestreamClient).Write(&writeRequest, credentials); err != nil {
errorCode := http.StatusBadRequest

Expand Down Expand Up @@ -214,7 +213,7 @@ func handleReadRequest(reqBuf []byte, timestreamClient *timestream.Client, awsCo

createQueryClient(timestreamClient, logger, awsConfigs, cfg.maxRetries)

timestream.LogInfo(logger, "Successfully created a Timestream query client to handle write requests from Prometheus.")
timestream.LogInfo(logger, fmt.Sprintf("Timestream query connection is initialized (Database: %s, Table: %s, Region: %s)", cfg.defaultDatabase, cfg.defaultTable, cfg.clientConfig.region))

response, err := getQueryClient(timestreamClient).Read(&readRequest, credentials)
if err != nil {
Expand Down Expand Up @@ -386,6 +385,16 @@ func parseFlags() *connectionConfig {
os.Exit(1)
}

if cfg.defaultDatabase == "" {
kingpin.Errorf("The default database value must be set through the flag --default-database")
os.Exit(1)
}

if cfg.defaultTable == "" {
kingpin.Errorf("The default table value must be set through the flag --default-table")
os.Exit(1)
}

return cfg
}

Expand Down
Loading