Skip to content

Commit

Permalink
[Filebeat] Add URL Encode template function for httpjson input (#30962)
Browse files Browse the repository at this point in the history
Add URL Encode template function for httpjson input.
  • Loading branch information
legoguy1000 authored and chrisberkhout committed Jun 1, 2023
1 parent 4dccb72 commit 75c3c52
Show file tree
Hide file tree
Showing 4 changed files with 33 additions and 1 deletion.
2 changes: 1 addition & 1 deletion CHANGELOG.next.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,6 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...main[Check the HEAD dif

- Add FIPS configuration option for all AWS API calls. {pull}28899[28899]
- Add support for kafka message headers. {pull}29940[29940]
- Add FIPS configuration option for all AWS API calls. {pull}[28899]
- Add support for non-unique Kafka headers for output messages. {pull}30369[30369]
- Add syslog parser and processor. {issue}30139[30139] {pull}30541[30541]
- Add action_input_type for the .fleet-actions-results {pull}30562[30562]
Expand All @@ -118,6 +117,7 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...main[Check the HEAD dif
- Improve recovery from corrupted registries. {issue}25135[25135] {pull}30994[30994]
- Add support in httpjson input for chain calls. {pull}29816[29816]
- checkpoint module: Add `network.transport` derived from IANA number. {pull}31076[31076]
- Add URL Encode template function for httpjson input. {pull}30962[30962]

*Auditbeat*

Expand Down
1 change: 1 addition & 0 deletions x-pack/filebeat/docs/inputs/input-httpjson.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -222,6 +222,7 @@ Some built-in helper functions are provided to work with the input state inside
- `uuid`: returns a random UUID such as `a11e8780-e3e7-46d0-8e76-f66e75acf019`. Example: `[[ uuid ]]`
- `userAgent`: generates the User Agent with optional additional values. If no arguments are provided, it will generate the default User Agent that is added to all requests by default. It is recommended to delete the existing User-Agent header before setting a new one. Example: `[[ userAgent "integration/1.2.3" ]]` would generate `Elastic-Filebeat/8.1.0 (darwin; amd64; 9b893e88cfe109e64638d65c58fd75c2ff695402; 2021-12-15 13:20:00 +0000 UTC; integration_name/1.2.3)`
- `beatInfo`: returns a map containing information about the Beat. Available keys in the map are `goos` (running operating system), `goarch` (running system architecture), `commit` (git commit of current build), `buildtime` (compile time of current build), `version` (version of current build). Example: `[[ beatInfo.version ]]` returns `{version}`.
- `urlEncode`: URL encodes the supplied string. Example `[[urlEncode "string1"]]`. Example `[[urlEncode "<string1>"]]` will return `%3Cstring1%3E`.

In addition to the provided functions, any of the native functions for https://golang.org/pkg/time/#Time[`time.Time`], https://golang.org/pkg/net/http/#Header[`http.Header`], and https://golang.org/pkg/net/url/#Values[`url.Values`] types can be used on the corresponding objects. Examples: `[[(now).Day]]`, `[[.last_response.header.Get "key"]]`

Expand Down
9 changes: 9 additions & 0 deletions x-pack/filebeat/input/httpjson/value_tpl.go
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ import (
"errors"
"fmt"
"hash"
"net/url"
"reflect"
"regexp"
"runtime"
Expand Down Expand Up @@ -71,6 +72,7 @@ func (t *valueTpl) Unpack(in string) error {
"uuid": uuidString,
"userAgent": userAgentString,
"beatInfo": beatInfo,
"urlEncode": urlEncode,
}).
Delims(leftDelim, rightDelim).
Parse(in)
Expand Down Expand Up @@ -379,3 +381,10 @@ func beatInfo() map[string]string {
"version": version.GetDefaultVersion(),
}
}

func urlEncode(value string) string {
if value == "" {
return ""
}
return url.QueryEscape(value)
}
22 changes: 22 additions & 0 deletions x-pack/filebeat/input/httpjson/value_tpl_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -460,6 +460,28 @@ func TestValueTpl(t *testing.T) {
paramTr: transformable{},
expectedVal: version.GetDefaultVersion(),
},
{
name: "func urlEncode blank",
value: `[[urlEncode ""]]`,
paramCtx: emptyTransformContext(),
paramTr: transformable{},
expectedVal: "",
expectedError: errEmptyTemplateResult.Error(),
},
{
name: "func urlEncode URL Safe",
value: `[[urlEncode "asdf"]]`,
paramCtx: emptyTransformContext(),
paramTr: transformable{},
expectedVal: "asdf",
},
{
name: "func urlEncode URL Safe",
value: `[[urlEncode "2022-02-17T04:37:10.406+0000"]]`,
paramCtx: emptyTransformContext(),
paramTr: transformable{},
expectedVal: "2022-02-17T04%3A37%3A10.406%2B0000",
},
}

for _, tc := range cases {
Expand Down

0 comments on commit 75c3c52

Please sign in to comment.