Skip to content

Commit

Permalink
Merge branch 'master' into async-nats
Browse files Browse the repository at this point in the history
  • Loading branch information
paolobarbolini committed Aug 6, 2023
2 parents 83ab34c + be551c8 commit 5d38f14
Show file tree
Hide file tree
Showing 2,442 changed files with 80,115 additions and 131,168 deletions.
9 changes: 8 additions & 1 deletion .cargo/config.toml
Original file line number Diff line number Diff line change
@@ -1,11 +1,14 @@
[alias]
vdev = "run --quiet --package vdev --"

[target.'cfg(all())']
rustflags = [
"-Dclippy::print_stdout",
"-Dclippy::print_stderr",
"-Dclippy::dbg_macro",
]

# We need to bring in `libstdc++` for things that build against C++ (LevelDB, librdkafka, etc) which comes along in the
# We need to bring in `libstdc++` for things that build against C++ (librdkafka, etc) which comes along in the
# `cross` base image but _isn't_ in a path searched by the linker normally. Additionally, our custom Docker image that
# we base on the `cross` image moves `libstdc++` into this custom-looking directory to avoid some _other_ libraries
# included in the `cross` base image from having a higher precedence than some of the "self-contained" libraries that
Expand All @@ -24,3 +27,7 @@ rustflags = ["-C", "link-args=-rdynamic"]

[target.aarch64-unknown-linux-gnu]
rustflags = ["-C", "link-args=-rdynamic"]

[target.x86_64-pc-windows-msvc]
# https://github.com/dtolnay/inventory/issues/58
rustflags = ["-C", "codegen-units=1"]
1 change: 1 addition & 0 deletions .config/nextest.toml
Original file line number Diff line number Diff line change
Expand Up @@ -16,3 +16,4 @@ fail-fast = false
[profile.default.junit]
# output test results at target/nextest/default/junit.xml
path = "junit.xml"
store-failure-output = true
1 change: 1 addition & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
Expand Up @@ -9,3 +9,4 @@
!Cargo.toml
!rust-toolchain.toml
!scripts
!vdev
99 changes: 99 additions & 0 deletions .github/CODEOWNERS
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
docs/ @vectordotdev/ux-team
lib/file-source/ @spencergilbert @vectordotdev/integrations-team
lib/k8s-e2e-tests/ @spencergilbert @vectordotdev/integrations-team
lib/k8s-test-framework/ @spencergilbert @vectordotdev/integrations-team
lib/opentelemetry-proto/ @spencergilbert @vectordotdev/integrations-team
lib/vector-common/ @vectordotdev/core-team
lib/vector-config/ @vectordotdev/core-team
lib/vector-config-common/ @vectordotdev/core-team
lib/vector-config-macros/ @vectordotdev/core-team
lib/vector-core/ @vectordotdev/core-team
lib/vector-vrl-functions/ @vectordotdev/processing-team
lib/vrl/ @vectordotdev/processing-team
src/config/ @vectordotdev/core-team
src/internal_telemetry/ @vectordotdev/core-team
src/sinks/ @vectordotdev/integrations-team
src/sinks/amqp/ @StephenWakely @vectordotdev/integrations-team
src/sinks/appsignal/ @neuronull @vectordotdev/integrations-team
src/sinks/aws_cloudwatch_logs/ @spencergilbert @vectordotdev/integrations-team
src/sinks/aws_cloudwatch_metrics/ @spencergilbert @vectordotdev/integrations-team
src/sinks/aws_kinesis/ @spencergilbert @vectordotdev/integrations-team # sink_aws_kinesis_firehose,sink_aws_kinesis_stream
src/sinks/aws_s3/ @spencergilbert @vectordotdev/integrations-team
src/sinks/aws_sqs/ @spencergilbert @vectordotdev/integrations-team
src/sinks/axiom.rs @spencergilbert @vectordotdev/integrations-team
src/sinks/azure_blob/ @dsmith3197 @vectordotdev/integrations-team
src/sinks/azure_monitor_logs.rs @dsmith3197 @vectordotdev/integrations-team
src/sinks/blackhole/ @dsmith3197 @vectordotdev/integrations-team
src/sinks/clickhouse/ @dsmith3197 @vectordotdev/integrations-team
src/sinks/console/ @dsmith3197 @vectordotdev/integrations-team
src/sinks/databend/ @spencergilbert @vectordotdev/integrations-team
src/sinks/datadog_events/ @neuronull @vectordotdev/integrations-team
src/sinks/datadog_logs/ @neuronull @vectordotdev/integrations-team
src/sinks/datadog_metrics/ @neuronull @vectordotdev/integrations-team
src/sinks/datadog_traces/ @neuronull @vectordotdev/integrations-team
src/sinks/elasticsearch/ @spencergilbert @vectordotdev/integrations-team
src/sinks/file/ @spencergilbert @vectordotdev/integrations-team
src/sinks/gcp/ @StephenWakely @vectordotdev/integrations-team # sink_gcp_chronicle_unstructured,sink_gcp_cloud_storage,sink_gcp_pubsub,sink_gcp_stackdriver_logs,sink_gcp_stackdriver_metrics
src/sinks/honeycomb.rs @spencergilbert @vectordotdev/integrations-team
src/sinks/http.rs @neuronull @vectordotdev/integrations-team
src/sinks/humio/ @StephenWakely @vectordotdev/integrations-team # sink_humio_logs,sink_humio_metrics
src/sinks/influxdb/ @dsmith3197 @vectordotdev/integrations-team # sink_influxdb_logs,sink_influxdb_metrics
src/sinks/kafka/ @dsmith3197 @vectordotdev/integrations-team
src/sinks/logdna.rs @neuronull @vectordotdev/integrations-team
src/sinks/loki/ @spencergilbert @vectordotdev/integrations-team
src/sinks/nats.rs @StephenWakely @vectordotdev/integrations-team
src/sinks/new_relic/ @dsmith3197 @vectordotdev/integrations-team # sink_newrelix,sink_newrelic_logs
src/sinks/papertrail.rs @StephenWakely @vectordotdev/integrations-team
src/sinks/prometheus/ @StephenWakely @vectordotdev/integrations-team # sink_prometheus_exporter,sink_prometheus_remote_write
src/sinks/pulsar.rs @dsmith3197 @vectordotdev/integrations-team
src/sinks/redis.rs @StephenWakely @vectordotdev/integrations-team
src/sinks/sematext/ @spencergilbert @vectordotdev/integrations-team # sink_sematext_logs,sink_sematext_metrics
src/sinks/socket.rs @neuronull @vectordotdev/integrations-team
src/sinks/splunk_hec/ @StephenWakely @vectordotdev/integrations-team # sink_splunk_hec_logs,sink_splunk_hec_metrics
src/sinks/statsd.rs @neuronull @vectordotdev/integrations-team
src/sinks/vector/ @neuronull @vectordotdev/integrations-team
src/sinks/websocket/ @neuronull @vectordotdev/integrations-team
src/source_sender/ @vectordotdev/core-team
src/sources/ @vectordotdev/integrations-team
src/sources/amqp.rs @StephenWakely @vectordotdev/integrations-team
src/sources/apache_metrics/ @dsmith3197 @vectordotdev/integrations-team
src/sources/aws_ecs_metrics/ @spencergilbert @vectordotdev/integrations-team
src/sources/aws_kinesis_firehose/ @spencergilbert @vectordotdev/integrations-team
src/sources/aws_s3/ @spencergilbert @vectordotdev/integrations-team
src/sources/aws_sqs/ @spencergilbert @vectordotdev/integrations-team
src/sources/datadog_agent/ @neuronull @vectordotdev/integrations-team
src/sources/demo_logs.rs @StephenWakely @vectordotdev/integrations-team
src/sources/dnstap/ @StephenWakely @vectordotdev/integrations-team
src/sources/docker_logs/ @spencergilbert @vectordotdev/integrations-team
src/sources/eventstoredb_metrics/ @dsmith3197 @vectordotdev/integrations-team
src/sources/exec/ @dsmith3197 @vectordotdev/integrations-team
src/sources/file.rs @spencergilbert @vectordotdev/integrations-team
src/sources/file_descriptors/ @dsmith3197 @vectordotdev/integrations-team # source_file_descriptor,source_stdin
src/sources/fluent/ @neuronull @vectordotdev/integrations-team
src/sources/gcp_pubsub.rs @StephenWakely @vectordotdev/integrations-team
src/sources/heroku_logs.rs @spencergilbert @vectordotdev/integrations-team
src/sources/host_metrics/ @dsmith3197 @vectordotdev/integrations-team
src/sources/http_client/ @neuronull @vectordotdev/integrations-team
src/sources/http_server.rs @neuronull @vectordotdev/integrations-team
src/sources/internal_logs.rs @neuronull @vectordotdev/integrations-team
src/sources/internal_metrics.rs @neuronull @vectordotdev/integrations-team
src/sources/journald.rs @spencergilbert @vectordotdev/integrations-team
src/sources/kafka.rs @dsmith3197 @vectordotdev/integrations-team
src/sources/kubernetes_logs/ @spencergilbert @vectordotdev/integrations-team
src/sources/logstash.rs @neuronull @vectordotdev/integrations-team
src/sources/mongodb_metrics/ @dsmith3197 @vectordotdev/integrations-team
src/sources/nats.rs @StephenWakely @vectordotdev/integrations-team
src/sources/nginx_metrics/ @dsmith3197 @vectordotdev/integrations-team
src/sources/opentelemetry/ @spencergilbert @vectordotdev/integrations-team
src/sources/postgresql_metrics.rs @dsmith3197 @vectordotdev/integrations-team
src/sources/prometheus/ @StephenWakely @vectordotdev/integrations-team # source_prometheus_remote_write,source_prometheus_scrape
src/sources/redis/ @StephenWakely @vectordotdev/integrations-team
src/sources/socket/ @neuronull @vectordotdev/integrations-team
src/sources/splunk_hec/ @StephenWakely @vectordotdev/integrations-team
src/sources/statsd/ @neuronull @vectordotdev/integrations-team
src/sources/syslog.rs @StephenWakely @vectordotdev/integrations-team
src/sources/vector/ @neuronull @vectordotdev/integrations-team
src/test_util/ @vectordotdev/core-team
src/topology/ @vectordotdev/core-team
src/transforms/ @vectordotdev/processing-team
website/cue @vectordotdev/ux-team
58 changes: 58 additions & 0 deletions .github/ISSUE_TEMPLATE/minor-release.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
---
name: Vector minor release
about: Use this template for a new minor release.
title: "Vector [version] release"
labels: "domain: releasing"
---

The week before the release:

- [ ] Create a new release branch from master to freeze commits
- `git fetch && git checkout origin/master && git checkout -b v0.<new version number> && git push -u`
- [ ] Create a new release preparation branch from `master`
- `git checkout -b prepare-v0.<new version number> && git push -u`
- [ ] Check if there is a newer version of Alpine or Debian available to update the release images
in `distribution/docker/`. Update if so.
- [ ] Run `cargo vdev build release-cue` to generate a new cue file for the release
- [ ] Add `changelog` key to generated cue file
- [ ] `git log --no-merges --cherry-pick --right-only <last release tag>...`
- [ ] Should be hand-written list of changes
([example](https://github.com/vectordotdev/vector/blob/9fecdc8b5c45c613de2d01d4d2aee22be3a2e570/website/cue/reference/releases/0.19.0.cue#L44))
- [ ] Ensure any breaking changes are highlighted in the release upgrade guide
- [ ] Ensure any deprecations are highlighted in the release upgrade guide
- [ ] Ensure all notable features have a highlight written
- [ ] Add description key to the generated cue file with a description of the release (see
previous releases for examples).
- [ ] Update version number in `website/cue/reference/administration/interfaces/kubectl.cue`
- [ ] Update version number in `distribution/install.sh`
- [ ] Add new version to `website/cue/reference/versions.cue`
- [ ] Create new release md file by copying an existing one in `./website/content/en/releases/` and
updating version number
- [ ] Commit these changes
- [ ] Open PR against the release branch (`v0.<new version number>`) for review
- [ ] PR approval

On the day of release:

- [ ] Rebase the release preparation branch on the release branch
- [ ] Squash the release preparation commits (but not the cherry-picked commits!) to a single
commit. This makes it easier to cherry-pick to master after the release. 
- [ ] Ensure release date in cue matches current date.
- [ ] Merge release preparation branch into the release branch
- `git co v0.<new version number> && git merge --ff-only prepare-v0.<new version number>`
- [ ] Tag new release
- [ ] `git tag v0.<minor>.0 -a -m v0.<minor>.0`
- [ ] `git push origin v0.<minor>.0`
- [ ] Wait for release workflow to complete
- Discoverable via [https://github.com/timberio/vector/actions/workflows/release.yml](https://github.com/timberio/vector/actions/workflows/release.yml)
- [ ] Release updated Helm chart. See [releasing Helm chart](https://github.com/vectordotdev/helm-charts#releasing).
- [ ] Once Helm chart is released, updated Vector manifests
- Run `cargo vdev build manifests` and open a PR with changes
- [ ] Add docker images to [https://github.com/DataDog/images](https://github.com/DataDog/images/tree/master/vector) to have them available internally.
- [ ] Cherry-pick any release commits from the release branch that are not on `master`, to `master`
- [ ] Bump the release number in the `Cargo.toml` on master to the next major release
- [ ] [Update Netlify deploy settings](https://app.netlify.com/sites/vector-project/settings/deploys#deploy-contexts)
- [ ] Update production branch to the new major version
- [ ] Update branch deploys to include the previous major version
- [ ] Add [branch subdomain](https://app.netlify.com/sites/vector-project/settings/domain) for previous version
- [ ] Kick-off post-mortems for any regressions resolved by the release
53 changes: 53 additions & 0 deletions .github/ISSUE_TEMPLATE/patch-release.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
---
name: Vector patch release
about: Use this template for a new patch release.
title: "Vector [version] release"
labels: "domain: releasing"
---

Before the release:

- [ ] Create a new release preparation branch from the current release branch
- `git fetch && git checkout v0.<current minor version> && git checkout -b prepare-v0.<new version number>`
- [ ] Cherry-pick in all commits to be released from the associated release milestone
- If any merge conflicts occur, attempt to solve them and if needed enlist the aid of those familiar with the conflicting commits.
- [ ] Run `cargo vdev build release-cue` to generate a new cue file for the release
- [ ] Add `changelog` key to generated cue file
- [ ] `git log --no-merges --cherry-pick --right-only <last release tag>...`
- [ ] Should be hand-written list of changes
([example](https://github.com/vectordotdev/vector/blob/9fecdc8b5c45c613de2d01d4d2aee22be3a2e570/website/cue/reference/releases/0.19.0.cue#L44))
- [ ] Add description key to the generated cue file with a description of the release (see
previous releases for examples).
- [ ] Update version number in `distribution/install.sh`
- [ ] Add new version to `website/cue/reference/versions.cue`
- [ ] Create new release md file by copying an existing one in `./website/content/en/releases/` and
updating version number
- [ ] Bump the release number in the `Cargo.toml` to the current version number
- [ ] Run `cargo check` to regenerate `Cargo.lock` file
- [ ] Commit these changes
- [ ] Open PR against the release branch (`v0.<new version number>`) for review
- [ ] PR approval

On the day of release:

- [ ] Ensure release date in cue matches current date.
- [ ] Rebase the release preparation branch on the release branch
- Squash the release preparation commits (but not the cherry-picked commits!) to a single
commit. This makes it easier to cherry-pick to master after the release.
- `git checkout prepare-v0.<new version number> && git rebase -i v0.<current minor version>`
- [ ] Merge release preparation branch into the release branch
- `git co v0.<current minor version> && git merge --ff-only prepare-v0.<current minor version>.<patch>`
- [ ] Tag new release
- [ ] `git tag v0.<minor>.<patch> -a -m v0.<minor>.<patch>`
- [ ] `git push origin v0.<minor>.<patch>`
- [ ] Wait for release workflow to complete
- Discoverable via [https://github.com/timberio/vector/actions/workflows/release.yml](https://github.com/timberio/vector/actions/workflows/release.yml)
- [ ] Push the release branch to update the remote (This should close the preparation branch PR).
- `git checkout v0.<current minor version> && git push`
- [ ] Release updated Helm chart. See [releasing Helm chart](https://github.com/vectordotdev/helm-charts#releasing).
- [ ] Once Helm chart is released, updated Vector manifests
- Run `cargo vdev build manifests` and open a PR with changes
- [ ] Add docker images to [https://github.com/DataDog/images](https://github.com/DataDog/images/tree/master/vector) to have them available internally.
- Follow the [instructions at the top of the mirror.yaml file](https://github.com/DataDog/images/blob/fbf12868e90d52e513ebca0389610dea8a3c7e1a/mirror.yaml#L33-L49).
- [ ] Cherry-pick any release commits from the release branch that are not on `master`, to `master`
- [ ] Kick-off post-mortems for any regressions resolved by the release
106 changes: 104 additions & 2 deletions .github/SUPPORT.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,107 @@
# Support

Vector is a growing open source project with a variety of people willing to help you through various mediums.
Vector is a growing open source project with a variety of people willing to help
you through various mediums.

Take a look at those mediums listed at https://vector.dev/community
Take a look at those mediums listed at <https://vector.dev/community>

## How to ask a question about Vector

Whether in our community [Discord][discord] server or [GitHub Discussions][discussions], framing
your question well and providing the right level of details will improve your chances of getting
your question answered. Here are some tips:

### Before asking

Check the [Vector documentation](https://vector.dev/docs/) first to see if it answers your question.
If your question is about [VRL](https://vector.dev/docs/reference/vrl/#learn), you can also try out
the [VRL playground](vrl_playground).

If the docs do not answer your question, try using the search feature on [Discord][discord] or
[GitHub][vector], to search for keywords related to your issue. It is quite possible someone has
already asked your question before. This is especially useful if you have a specific error message
you are observing.

### Provide details

Essential details to include:

- Errors- for each error, please provide the full error message snippet, and
details such as where the error is observed, at what stage in the process
(e.g. at boot time, after some specific condition etc.).
- What is the version of Vector (and the Helm chart if deploying via Helm) and
the versions of any other systems in use (like Elasticsearch, NATS, etc.).
- What is your Vector configuration. See the below section on [how to format
your config](#formatting).
- How are you [deploying](https://vector.dev/docs/setup/deployment/) Vector?
- What is your complete deployment architecture? For example: I have Logstash
agents sending to Vector over syslog that is being forwarded to Loki.

Situation specific (not exhaustive):

- Did it occur after upgrading to a new version of Vector?
- Are you trying out Vector for the first time, or did you have a previous
working configuration?

These are just some examples of questions that may or may not apply to your
situation.

### Formatting

When providing snippets of configuration files, or log messages, format these with backticks to
improve the legibility for readers.

#### Blocks

Blocks should be formatted with three backticks (\`\`\`)

This should be used for any configuration snippets of Vector, Helm values, etc.
and for Vector console log messages.

See the [markdown documentation](https://www.markdownguide.org/basic-syntax/#fenced-code-blocks)
for an explanation.

For example:

[sinks.sink0]

inputs = ["source0"]

target = "stdout"

type = "console"

should be written as

```toml
[sinks.sink0]
inputs = ["source0"]
target = "stdout"
type = "console"
```

#### Single word or phrase

Formatting things like component names, versions etc. is done with single
backticks (\`) around the word or phrase.

This is less critical but also helps readability a lot and is greatly
appreciated.

See the [markdown documentation](https://www.markdownguide.org/basic-syntax/#code)
for an explanation.

For example:

> We upgraded from v24 to 25.1 and are seeing the following error output from
kafka.

should be written as

> We upgraded from `v0.24.0` to `v0.25.1` and are seeing the following error
output from the `kafka` sink.

[discord]: https://chat.vector.dev
[discussions]: https://github.com/vectordotdev/vector/discussions
[vector]: https://github.com/vectordotdev/vector
[vrl_playground]: https://playground.vrl.dev
Loading

0 comments on commit 5d38f14

Please sign in to comment.