From 973b96d0945c94d41f4487aeec4c97c350c4638a Mon Sep 17 00:00:00 2001 From: Takayuki Watanabe Date: Sat, 8 Feb 2020 00:10:13 +0900 Subject: [PATCH] Unite docs for fluentd plugin (#1634) * Use original README.md for fluentd plugin explanations `doc/client/fluentd.m` includes obsolete explanations and it confuses new users. For example, label_keys option was removed at https://github.com/grafana/loki/pull/1186 but `fluentd.md` has the explanation. It's simple to maitain `fluentd/fluent-plugin-grafana-loki/README.md` for the explantion of the output plugin and use the link when we share how to use it. * Remove `label_keys` option and simplify usage for `drop_single_key` * Fix environment variable: FLUENT_CONFIG -> FLUENTD_CONFIG `docker-compose up` fails due to the wrong environment variable specifying the fluentd config. As a result this causes the following error ``` docker-compose up --build WARNING: The LOKI_USERNAME variable is not set. Defaulting to a blank string. WARNING: The LOKI_PASSWORD variable is not set. Defaulting to a blank string. Building fluentd Step 1/17 : FROM fluent/fluentd:v1.3.2-debian ---> 4790aaf4d1e5 Step 2/17 : USER root ---> Using cache ---> 7b54da6cec9e Step 3/17 : WORKDIR /home/fluent ---> Using cache ---> a161d2fcf64b Step 4/17 : ENV PATH /fluentd/vendor/bundle/ruby/2.3.0/bin:$PATH ---> Using cache ---> b67529aad0b5 Step 5/17 : ENV GEM_PATH /fluentd/vendor/bundle/ruby/2.3.0 ---> Using cache ---> 3a8b05beb185 Step 6/17 : ENV GEM_HOME /fluentd/vendor/bundle/ruby/2.3.0 ---> Using cache ---> d68a94345cb4 Step 7/17 : ENV FLUENTD_DISABLE_BUNDLER_INJECTION 1 ---> Using cache ---> 70f9bbf646e8 Step 8/17 : COPY docker/Gemfile* /fluentd/ ---> Using cache ---> 5f8b2f6ffe1a Step 9/17 : RUN buildDeps="sudo make gcc g++ libc-dev ruby-dev" && apt-get update && apt-get install -y --no-install-recommends $buildDeps libsystemd0 net-tools libjemalloc1 && gem install bundler --version 1.16.2 && bundle config silence_root_warning true && bundle install --gemfile=/fluentd/Gemfile --path=/fluentd/vendor/bundle && sudo gem sources --clear-all && SUDO_FORCE_REMOVE=yes apt-get purge -y --auto-remove -o APT::AutoRemove::RecommendsImportant=false $buildDeps && rm -rf /var/lib/apt/lists/* /home/fluent/.gem/ruby/2.3.0/cache/*.gem /tmp/* /var/tmp/* /usr/lib/ruby/gems/*/cache/*.gem ---> Using cache ---> 36c960a53c2c Step 10/17 : COPY docker/entrypoint.sh /fluentd/entrypoint.sh ---> Using cache ---> 7f8d72ae63ca Step 11/17 : COPY lib/fluent/plugin/out_loki.rb /fluentd/plugins/out_loki.rb ---> Using cache ---> d09473ee4f25 Step 12/17 : COPY docker/conf/ /fluentd/etc/loki/ ---> Using cache ---> 00d0439ef3c8 Step 13/17 : ENV FLUENTD_CONF="/fluentd/etc/loki/fluentd.conf" ---> Using cache ---> 2d33a002baaf Step 14/17 : ENV FLUENTD_OPT="" ---> Using cache ---> c314df064756 Step 15/17 : ENV LOKI_URL "https://logs-us-west1.grafana.net" ---> Using cache ---> df78c829d416 Step 16/17 : ENV LD_PRELOAD="/usr/lib/x86_64-linux-gnu/libjemalloc.so.1" ---> Using cache ---> 3d7d14cf37f5 Step 17/17 : ENTRYPOINT ["/fluentd/entrypoint.sh"] ---> Using cache ---> 82c70c2354ac Successfully built 82c70c2354ac Successfully tagged fluentd:loki Starting docker_fluentd_1 ... done Attaching to docker_fluentd_1 fluentd_1 | /fluentd/vendor/bundle/ruby/2.3.0/gems/fluentd-1.6.2/lib/fluent/supervisor.rb:769:in `initialize': No such file or directory @ rb_sysopen - /fluentd/etc/loki/fluentd.conf (Errno::ENOENT) fluentd_1 | from /fluentd/vendor/bundle/ruby/2.3.0/gems/fluentd-1.6.2/lib/fluent/supervisor.rb:769:in `open' fluentd_1 | from /fluentd/vendor/bundle/ruby/2.3.0/gems/fluentd-1.6.2/lib/fluent/supervisor.rb:769:in `read_config' fluentd_1 | from /fluentd/vendor/bundle/ruby/2.3.0/gems/fluentd-1.6.2/lib/fluent/supervisor.rb:479:in `run_supervisor' fluentd_1 | from /fluentd/vendor/bundle/ruby/2.3.0/gems/fluentd-1.6.2/lib/fluent/command/fluentd.rb:314:in `' fluentd_1 | from /usr/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:in `require' fluentd_1 | from /usr/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:in `require' fluentd_1 | from /fluentd/vendor/bundle/ruby/2.3.0/gems/fluentd-1.6.2/bin/fluentd:8:in `' fluentd_1 | from /fluentd/vendor/bundle/ruby/2.3.0/bin/fluentd:22:in `load' fluentd_1 | from /fluentd/vendor/bundle/ruby/2.3.0/bin/fluentd:22:in `
' docker_fluentd_1 exited with code 1 ``` * Use docs/clients/fluentd/README.md for fluent-plugin-grafana-loki explanation * Arrange list order based on number * Revert "Fix environment variable: FLUENT_CONFIG -> FLUENTD_CONFIG" This reverts commit 8ed58ac97a82d5536321fb9a76120fed766ae506. * Elaborate the way to use docker image * Fix grammar: remove unnecessary with * Add disclaimer for multi-worker usage --- docs/clients/README.md | 2 +- docs/clients/fluentd.md | 179 ---------------- docs/clients/fluentd/README.md | 211 ++++++++++++++++++ fluentd/fluent-plugin-grafana-loki/README.md | 212 +------------------ 4 files changed, 213 insertions(+), 391 deletions(-) delete mode 100644 docs/clients/fluentd.md create mode 100644 docs/clients/fluentd/README.md diff --git a/docs/clients/README.md b/docs/clients/README.md index b58dbc6e58cf3..bb54fd657b4e4 100644 --- a/docs/clients/README.md +++ b/docs/clients/README.md @@ -4,8 +4,8 @@ Loki supports the following official clients for sending logs: 1. [Promtail](./promtail/README.md) 2. [Docker Driver](./docker-driver/README.md) +3. [Fluentd](./fluentd/README.md) 4. [Fluent Bit](../../cmd/fluent-bit/README.md) -3. [Fluentd](./fluentd.md) ## Picking a Client diff --git a/docs/clients/fluentd.md b/docs/clients/fluentd.md deleted file mode 100644 index d2d2380c6c87e..0000000000000 --- a/docs/clients/fluentd.md +++ /dev/null @@ -1,179 +0,0 @@ -# Fluentd - -Loki has a [Fluentd](https://www.fluentd.org/) output plugin called -`fluent-plugin-grafana-loki` that enables shipping logs to a private Loki -instance or [Grafana Cloud](https://grafana.com/oss/loki). - -The plugin offers two line formats and uses protobuf to send compressed data to -Loki. - -Key features: - -* `extra_labels`: Labels to be added to every line of a log file, useful for - designating environments -* `label_keys`: Customizable list of keys for stream labels -* `line_format`: Format to use when flattening the record to a log line (`json` - or `key_value`). - -## Installation - -```bash -$ gem install fluent-plugin-grafana-loki -``` - -## Usage - -In your Fluentd configuration, use `@type loki`. Additional configuration is -optional. Default values look like this: - -``` - - @type loki - url "https://logs-us-west1.grafana.net" - username "#{ENV['LOKI_USERNAME']}" - password "#{ENV['LOKI_PASSWORD']}" - extra_labels {"env":"dev"} - flush_interval 10s - flush_at_shutdown true - buffer_chunk_limit 1m - -``` - -### Multi-worker usage - -Loki doesn't currently support out-of-order inserts - if you try to insert a log -entry with an earlier timestamp after a log entry with with identical labels but -a later timestamp, the insert will fail with the message -`HTTP status code: 500, message: rpc error: code = Unknown desc = Entry out of -order`. Therefore, in order to use this plugin in a multi-worker Fluentd setup, -you'll need to include the worker ID in the labels sent to Loki. - -For example, using -[fluent-plugin-record-modifier](https://github.com/repeatedly/fluent-plugin-record-modifier): - -``` - - @type record_modifier - - fluentd_worker "#{worker_id}" - - - - - @type loki - # ... - label_keys "fluentd_worker" - # ... - -``` - -## Docker Image - -There is a Docker image `grafana/fluent-plugin-grafana-loki:master` which -contains default configuration files to git log information -a host's `/var/log` directory, and from the host's journald. To use it, you can set -the `LOKI_URL`, `LOKI_USERNAME`, and `LOKI_PASSWORD` environment variables -(`LOKI_USERNAME` and `LOKI_PASSWORD` can be left blank if Loki is not protected -behind an authenticating proxy). - -An example Docker Swarm Compose configuration looks like: - -```yaml -services: - fluentd: - image: grafana/fluent-plugin-grafana-loki:master - command: - - "fluentd" - - "-v" - - "-p" - - "/fluentd/plugins" - environment: - LOKI_URL: http://loki:3100 - LOKI_USERNAME: - LOKI_PASSWORD: - deploy: - mode: global - configs: - - source: loki_config - target: /fluentd/etc/loki/loki.conf - networks: - - loki - volumes: - - host_logs:/var/log - # Needed for journald log ingestion: - - /etc/machine-id:/etc/machine-id - - /dev/log:/dev/log - - /var/run/systemd/journal/:/var/run/systemd/journal/ - logging: - options: - tag: infra.monitoring -``` - -## Configuration - -### Proxy Support - -Starting with version 0.8.0, this gem uses `excon`, which supports proxy with -environment variables - https://github.com/excon/excon#proxy-support - -### `url` - -The URL of the Loki server to send logs to. When sending data the publish path -(`/loki/api/v1/push`) will automatically be appended. By default the URL is set to -`https://logs-us-west1.grafana.net`, the URL of the Grafana Labs [hosted -Loki](https://grafana.com/loki) service. - -### `username` / `password` - -Specify a username and password if the Loki server requires authentication. -If using the Grafana Labs' hosted Loki, the username needs to be set to your -instanceId and the password should be a grafana.com API Key. - -### `tenant` - -Loki is a multi-tenant log storage platform and all requests sent must include a -tenant. For some installations (like Hosted Loki) the tenant will be set -automatically by an authenticating proxy. Otherwise you can define a tenant to -be passed through. The tenant can be any string value. - -### output format - -Loki is intended to index and group log streams using only a small set of -labels and is not intended for full-text indexing. When sending logs to Loki, -the majority of log message will be sent as a single log "line". - -There are few configurations settings to control the output format: - -- `extra_labels`: (default: nil) set of labels to include with every Loki - stream. (e.g., `{"env":"dev", "datacenter": "dc1"}`) - -- `remove_keys`: (default: nil) comma separated list of record keys to - remove. All other keys will be placed into the log line. - -- `label_keys`: (default: "job,instance") comma separated list of keys to use as - stream labels. - -- `line_format`: format to use when flattening the record to a log line. Valid - values are `json` or `key_value`. If set to `json` the log line sent to Loki - will be the fluentd record (excluding any keys extracted out as labels) dumped - as json. If set to `key_value`, the log line will be each item in the record - concatenated together (separated by a single space) in the format - `=`. - -- `drop_single_key`: if set to true, when the set of extracted `label_keys` - after dropping with `remove_keys`, the log line sent to Loki will just be - the value of the single remaining record key. - -### Buffer options - -`fluentd-plugin-loki` extends [Fluentd's builtin Output -plugin](https://docs.fluentd.org/v1.0/articles/output-plugin-overview) and uses -the `compat_parameters` plugin helper. It adds the following options: - -``` -buffer_type memory -flush_interval 10s -retry_limit 17 -retry_wait 1.0 -num_threads 1 -``` diff --git a/docs/clients/fluentd/README.md b/docs/clients/fluentd/README.md new file mode 100644 index 0000000000000..faf011ab4edca --- /dev/null +++ b/docs/clients/fluentd/README.md @@ -0,0 +1,211 @@ +# Fluentd + +Loki has a [Fluentd](https://www.fluentd.org/) output plugin called +`fluent-plugin-grafana-loki` that enables shipping logs to a private Loki +instance or [Grafana Cloud](https://grafana.com/oss/loki). + +## Installation + +``` +$ gem install fluent-plugin-grafana-loki +``` + +## Usage +In your Fluentd configuration, use `@type loki`. Additional configuration is optional, default values would look like this: +``` + + @type loki + url "https://logs-us-west1.grafana.net" + username "#{ENV['LOKI_USERNAME']}" + password "#{ENV['LOKI_PASSWORD']}" + extra_labels {"env":"dev"} + flush_interval 10s + flush_at_shutdown true + buffer_chunk_limit 1m + +``` + +### Using labels + +Simple label from top level attribute +``` + + @type loki + # ... + + # ... + +``` + +You can rewrite the label keys as well as the following + +``` + + @type loki + # ... + + # ... + +``` + +You can use record accessor syntax for nested field. https://docs.fluentd.org/plugin-helper-overview/api-plugin-helper-record_accessor#syntax + +``` + + @type loki + # ... + + # ... + +``` + +### Extracting Kubernetes labels + +As Kubernetes labels are a list of nested key-value pairs there is a separate option to extract them. +Note that special characters like "`. - /`" will be overwritten with `_`. +Use with the `remove_keys kubernetes` option to eliminate metadata from the log. +``` + + @type loki + # ... + extract_kubernetes_labels true + remove_keys kubernetes + + # ... + +``` + +### Multi-worker usage + +Loki doesn't currently support out-of-order inserts - if you try to insert a log entry an earlier timestamp after a log entry with identical labels but a later timestamp, the insert will fail with `HTTP status code: 500, message: rpc error: code = Unknown desc = Entry out of order`. Therefore, in order to use this plugin in a multi worker Fluentd setup, you'll need to include the worker ID in the labels or otherwise [ensure log streams are always sent to the same worker](https://docs.fluentd.org/deployment/multi-process-workers#less-than-worker-n-greater-than-directive). + +For example, using [fluent-plugin-record-modifier](https://github.com/repeatedly/fluent-plugin-record-modifier): +``` + + @type record_modifier + + fluentd_worker "#{worker_id}" + + + + + @type loki + # ... + + # ... + +``` + +### Using multiple buffer flush threads + +Similarly, when using `flush_thread_count` > 1 in the [`buffer`](https://docs.fluentd.org/configuration/buffer-section#flushing-parameters) +section, a thread identifier must be added as a label to ensure that log chunks flushed in parallel to loki by fluentd always have increasing +times for their unique label sets. + +This plugin automatically adds a `fluentd_thread` label with the name of the buffer flush thread when `flush_thread_count` > 1. + +## Docker Image + +There is a Docker image `grafana/fluent-plugin-grafana-loki:master` which contains [default configuration files](https://github.com/grafana/loki/tree/master/fluentd/fluent-plugin-grafana-loki/docker/conf). By default, fluentd containers use the configurations but you can also specify your `fluentd.conf` with `FLUENTD_CONF` environment variable. + +This image also uses `LOKI_URL`, `LOKI_USERNAME`, and `LOKI_PASSWORD` environment variables to specify the Loki's endpoint, user, and password (you can leave the USERNAME and PASSWORD blank if they're not used). + +### Example + +A Docker Compose configuration that will work looks like: + +``` +services: + fluentd: + image: grafana/fluent-plugin-grafana-loki:master + command: + - "fluentd" + - "-v" + - "-p" + - "/fluentd/plugins" + environment: + LOKI_URL: http://loki:3100 + LOKI_USERNAME: + LOKI_PASSWORD: + deploy: + mode: global + configs: + - source: loki_config + target: /fluentd/etc/loki/loki.conf + networks: + - loki + volumes: + - host_logs:/var/log + # Needed for journald log ingestion: + - /etc/machine-id:/etc/machine-id + - /dev/log:/dev/log + - /var/run/systemd/journal/:/var/run/systemd/journal/ + logging: + options: + tag: infra.monitoring +``` + +## Configuration + +### url +The url of the Loki server to send logs to. When sending data the publish path (`/api/prom/push`) will automatically be appended. +By default the url is set to `https://logs-us-west1.grafana.net`, the url of the Grafana Labs preview (hosted Loki)[https://grafana.com/loki] service. + +#### Proxy Support + +Starting with version 0.8.0, this gem uses excon, which supports proxy with environment variables - https://github.com/excon/excon#proxy-support + +### username / password +Specify a username and password if the Loki server requires authentication. +If using the GrafanaLab's hosted Loki, the username needs to be set to your instanceId and the password should be a Grafana.com api key. + +### tenant +Loki is a multi-tenant log storage platform and all requests sent must include a tenant. For some installations the tenant will be set automatically by an authenticating proxy. Otherwise you can define a tenant to be passed through. The tenant can be any string value. + +### client certificate verification +Specify a pair of client certificate and private key with `cert` and `key` if a reverse proxy with client certificate verification is configured in front of Loki. `ca_cert` can also be specified if the server uses custom certificate authority. + +``` + + @type loki + + url "https://loki" + + cert /path/to/certificate.pem + key /path/to/key.key + ca_cert /path/to/ca.pem + + ... + +``` + +### output format +Loki is intended to index and group log streams using only a small set of labels. It is not intended for full-text indexing. When sending logs to Loki the majority of log message will be sent as a single log "line". + +There are few configurations settings to control the output format. + - extra_labels: (default: nil) set of labels to include with every Loki stream. eg `{"env":"dev", "datacenter": "dc1"}` + - remove_keys: (default: nil) comma separated list of needless record keys to remove. All other keys will be placed into the log line. You can use [record_accessor syntax](https://docs.fluentd.org/plugin-helper-overview/api-plugin-helper-record_accessor#syntax). + - line_format: format to use when flattening the record to a log line. Valid values are "json" or "key_value". If set to "json" the log line sent to Loki will be the fluentd record (excluding any keys extracted out as labels) dumped as json. If set to "key_value", the log line will be each item in the record concatenated together (separated by a single space) in the format `=`. + - drop_single_key: if set to true and a record only has 1 key after extracting `` blocks, set the log line to the value and discard the key. + +### Buffer options + +`fluentd-plugin-loki` extends [Fluentd's builtin Output plugin](https://docs.fluentd.org/v1.0/articles/output-plugin-overview) and use `compat_parameters` plugin helper. It adds the following options: + +``` +buffer_type memory +flush_interval 10s +retry_limit 17 +retry_wait 1.0 +num_threads 1 +``` diff --git a/fluentd/fluent-plugin-grafana-loki/README.md b/fluentd/fluent-plugin-grafana-loki/README.md index 8e2f66d8fd41a..21554b72254fc 100644 --- a/fluentd/fluent-plugin-grafana-loki/README.md +++ b/fluentd/fluent-plugin-grafana-loki/README.md @@ -1,216 +1,6 @@ # fluent-plugin-grafana-loki -[Fluentd](https://fluentd.org/) output plugin to ship logs to a Loki server. - -This plugin offers two line formats and uses protobuf to send compressed data to Loki. - -Key features: - * extra_labels - labels to be added to every line of a logfile, useful for designating environments - * label - This section allows you to specify labels from your log fields - -## Installation - -``` -$ gem install fluent-plugin-grafana-loki -``` - -## Usage -In your Fluentd configuration, use `@type loki`. Additional configuration is optional, default values would look like this: -``` - - @type loki - url "https://logs-us-west1.grafana.net" - username "#{ENV['LOKI_USERNAME']}" - password "#{ENV['LOKI_PASSWORD']}" - extra_labels {"env":"dev"} - flush_interval 10s - flush_at_shutdown true - buffer_chunk_limit 1m - -``` - -### Using labels - -Simple label from top level attribute -``` - - @type loki - # ... - - # ... - -``` - -You can rewrite the label keys as well as the following - -``` - - @type loki - # ... - - # ... - -``` - -You can use record accessor syntax for nested field. https://docs.fluentd.org/plugin-helper-overview/api-plugin-helper-record_accessor#syntax - -``` - - @type loki - # ... - - # ... - -``` - -### Extracting Kubernetes labels - -As Kubernetes labels are a list of nested key-value pairs there is a separate option to extract them. -Note that special characters like "`. - /`" will be overwritten with `_`. -Use with the `remove_keys kubernetes` option to eliminate metadata from the log. -``` - - @type loki - # ... - extract_kubernetes_labels true - remove_keys kubernetes - - # ... - -``` - -### Multi-worker usage - -Loki doesn't currently support out-of-order inserts - if you try to insert a log entry an earlier timestamp after a log entry with with identical labels but a later timestamp, the insert will fail with `HTTP status code: 500, message: rpc error: code = Unknown desc = Entry out of order`. Therefore, in order to use this plugin in a multi worker Fluentd setup, you'll need to include the worker ID in the labels. - -For example, using [fluent-plugin-record-modifier](https://github.com/repeatedly/fluent-plugin-record-modifier): -``` - - @type record_modifier - - fluentd_worker "#{worker_id}" - - - - - @type loki - # ... - - # ... - -``` - -### Using multiple buffer flush threads - -Similarly, when using `flush_thread_count` > 1 in the [`buffer`](https://docs.fluentd.org/configuration/buffer-section#flushing-parameters) -section, a thread identifier must be added as a label to ensure that log chunks flushed in parallel to loki by fluentd always have increasing -times for their unique label sets. - -This plugin automatically adds a `fluentd_thread` label with the name of the buffer flush thread when `flush_thread_count` > 1. - -## Docker Image - -There is a Docker image `grafana/fluent-plugin-grafana-loki:master` which contains default configuration files to git log information -a host's `/var/log` dir, and from the host's Journald. To use it, you can set the `LOKI_URL`, `LOKI_USERNAME`, and `LOKI_PASSWORD` environment variables (you can leave the USERNAME and PASSWORD blank if they're not used.) - - -A Docker Swarm Compose configuration that will work looks like: - -``` -services: - fluentd: - image: grafana/fluent-plugin-grafana-loki:master - command: - - "fluentd" - - "-v" - - "-p" - - "/fluentd/plugins" - environment: - LOKI_URL: http://loki:3100 - LOKI_USERNAME: - LOKI_PASSWORD: - deploy: - mode: global - configs: - - source: loki_config - target: /fluentd/etc/loki/loki.conf - networks: - - loki - volumes: - - host_logs:/var/log - # Needed for journald log ingestion: - - /etc/machine-id:/etc/machine-id - - /dev/log:/dev/log - - /var/run/systemd/journal/:/var/run/systemd/journal/ - logging: - options: - tag: infra.monitoring -``` - -## Configuration - -### url -The url of the Loki server to send logs to. When sending data the publish path (`/api/prom/push`) will automatically be appended. -By default the url is set to `https://logs-us-west1.grafana.net`, the url of the Grafana Labs preview (hosted Loki)[https://grafana.com/loki] service. - -#### Proxy Support - -Starting with version 0.8.0, this gem uses excon, which supports proxy with environment variables - https://github.com/excon/excon#proxy-support - -### username / password -Specify a username and password if the Loki server requires authentication. -If using the GrafanaLab's hosted Loki, the username needs to be set to your instanceId and the password should be a Grafana.com api key. - -### tenant -Loki is a multi-tenant log storage platform and all requests sent must include a tenant. For some installations the tenant will be set automatically by an authenticating proxy. Otherwise you can define a tenant to be passed through. The tenant can be any string value. - -### client certificate verification -Specify a pair of client certificate and private key with `cert` and `key` if a reverse proxy with client certificate verification is configured in front of Loki. `ca_cert` can also be specified if the server uses custom certificate authority. - -``` - - @type loki - - url "https://loki" - - cert /path/to/certificate.pem - key /path/to/key.key - ca_cert /path/to/ca.pem - - ... - -``` - -### output format -Loki is intended to index and group log streams using only a small set of labels. It is not intended for full-text indexing. When sending logs to Loki the majority of log message will be sent as a single log "line". - -There are few configurations settings to control the output format. - - extra_labels: (default: nil) set of labels to include with every Loki stream. eg `{"env":"dev", "datacenter": "dc1"}` - - remove_keys: (default: nil) comma separated list of needless record keys to remove. All other keys will be placed into the log line. You can use [record_accessor syntax](https://docs.fluentd.org/plugin-helper-overview/api-plugin-helper-record_accessor#syntax). - - line_format: format to use when flattening the record to a log line. Valid values are "json" or "key_value". If set to "json" the log line sent to Loki will be the fluentd record (excluding any keys extracted out as labels) dumped as json. If set to "key_value", the log line will be each item in the record concatenated together (separated by a single space) in the format `=`. - - drop_single_key: if set to true and after extracting label_keys a record only has a single key remaining, the log line sent to Loki will just be the value of the record key. - -### Buffer options - -`fluentd-plugin-loki` extends [Fluentd's builtin Output plugin](https://docs.fluentd.org/v1.0/articles/output-plugin-overview) and use `compat_parameters` plugin helper. It adds the following options: - -``` -buffer_type memory -flush_interval 10s -retry_limit 17 -retry_wait 1.0 -num_threads 1 -``` +[Fluentd](https://fluentd.org/) output plugin to ship logs to a Loki server. See [docs/client/fluentd/README.md](../../docs/clients/fluentd/README.md) for detailed information. ## Development