Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added top_level_domain #572

Merged
merged 6 commits into from
Oct 3, 2019
Merged

Added top_level_domain #572

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
88775bd
Added top_level_domain
mbudge Sep 6, 2019
e703c94
Removed dns top_level_domain
mbudge Sep 15, 2019
1d84a53
Apply PR #542 feedback
Oct 1, 2019
6567a84
Merge branch 'master' into top_level_domain
Oct 1, 2019
c36e1a9
Changelog entry
Oct 1, 2019
8ffb26a
Fix the addition in dns. It's dns.question.top_level_domain
Oct 1, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.next.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@ Thanks, you're awesome :-) -->
### Added

* Added fields in `log.*` to allow for full Syslog mapping. #525
* Added `top_level_domain` field to `url`, `dns.question`,
`source`, `destination`, `client`, and `server`. #562, #572
* Add group.domain field #547
* Added `error.stack_trace` field. #562
* Added `log.origin.file.name`, `log.origin.function` and `log.origin.file.line` fields. #563
Expand Down
9 changes: 9 additions & 0 deletions code/go/ecs/client.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

9 changes: 9 additions & 0 deletions code/go/ecs/destination.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

9 changes: 9 additions & 0 deletions code/go/ecs/dns.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

9 changes: 9 additions & 0 deletions code/go/ecs/server.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

9 changes: 9 additions & 0 deletions code/go/ecs/source.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

9 changes: 9 additions & 0 deletions code/go/ecs/url.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

78 changes: 78 additions & 0 deletions docs/field-details.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -339,6 +339,19 @@ example: `google.com`

// ===============================================================

| client.top_level_domain
| The effective top level domain (eTLD), also known as the domain suffix, is the last part of the domain name. For example, the top level domain for google.com is "com".

This value can be determined precisely with a list like the public suffix list (http://publicsuffix.org). Trying to approximate this by simply taking the last label will not work well for effective TLDs such as "co.uk".

type: keyword

example: `co.uk`

| extended

// ===============================================================

|=====

==== Field Reuse
Expand Down Expand Up @@ -690,6 +703,19 @@ example: `google.com`

// ===============================================================

| destination.top_level_domain
| The effective top level domain (eTLD), also known as the domain suffix, is the last part of the domain name. For example, the top level domain for google.com is "com".

This value can be determined precisely with a list like the public suffix list (http://publicsuffix.org). Trying to approximate this by simply taking the last label will not work well for effective TLDs such as "co.uk".

type: keyword

example: `co.uk`

| extended

// ===============================================================

|=====

==== Field Reuse
Expand Down Expand Up @@ -890,6 +916,19 @@ example: `google.com`

// ===============================================================

| dns.question.top_level_domain
| The effective top level domain (eTLD), also known as the domain suffix, is the last part of the domain name. For example, the top level domain for google.com is "com".

This value can be determined precisely with a list like the public suffix list (http://publicsuffix.org). Trying to approximate this by simply taking the last label will not work well for effective TLDs such as "co.uk".

type: keyword

example: `co.uk`

| extended

// ===============================================================

| dns.question.type
| The type of record being queried.

Expand Down Expand Up @@ -3038,6 +3077,19 @@ example: `google.com`

// ===============================================================

| server.top_level_domain
| The effective top level domain (eTLD), also known as the domain suffix, is the last part of the domain name. For example, the top level domain for google.com is "com".

This value can be determined precisely with a list like the public suffix list (http://publicsuffix.org). Trying to approximate this by simply taking the last label will not work well for effective TLDs such as "co.uk".

type: keyword

example: `co.uk`

| extended

// ===============================================================

|=====

==== Field Reuse
Expand Down Expand Up @@ -3326,6 +3378,19 @@ example: `google.com`

// ===============================================================

| source.top_level_domain
| The effective top level domain (eTLD), also known as the domain suffix, is the last part of the domain name. For example, the top level domain for google.com is "com".

This value can be determined precisely with a list like the public suffix list (http://publicsuffix.org). Trying to approximate this by simply taking the last label will not work well for effective TLDs such as "co.uk".

type: keyword

example: `co.uk`

| extended

// ===============================================================

|=====

==== Field Reuse
Expand Down Expand Up @@ -3543,6 +3608,19 @@ example: `https`

// ===============================================================

| url.top_level_domain
| The effective top level domain (eTLD), also known as the domain suffix, is the last part of the domain name. For example, the top level domain for google.com is "com".

This value can be determined precisely with a list like the public suffix list (http://publicsuffix.org). Trying to approximate this by simply taking the last label will not work well for effective TLDs such as "co.uk".

type: keyword

example: `co.uk`

| extended

// ===============================================================

| url.username
| Username of the request.

Expand Down
72 changes: 72 additions & 0 deletions generated/beats/fields.ecs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -285,6 +285,18 @@
list (http://publicsuffix.org). Trying to approximate this by simply taking
the last two labels will not work well for TLDs such as "co.uk".'
example: google.com
- name: top_level_domain
level: extended
type: keyword
ignore_above: 1024
description: 'The effective top level domain (eTLD), also known as the domain
suffix, is the last part of the domain name. For example, the top level domain
for google.com is "com".

This value can be determined precisely with a list like the public suffix
list (http://publicsuffix.org). Trying to approximate this by simply taking
the last label will not work well for effective TLDs such as "co.uk".'
example: co.uk
- name: user.domain
level: extended
type: keyword
Expand Down Expand Up @@ -578,6 +590,18 @@
list (http://publicsuffix.org). Trying to approximate this by simply taking
the last two labels will not work well for TLDs such as "co.uk".'
example: google.com
- name: top_level_domain
level: extended
type: keyword
ignore_above: 1024
description: 'The effective top level domain (eTLD), also known as the domain
suffix, is the last part of the domain name. For example, the top level domain
for google.com is "com".

This value can be determined precisely with a list like the public suffix
list (http://publicsuffix.org). Trying to approximate this by simply taking
the last label will not work well for effective TLDs such as "co.uk".'
example: co.uk
- name: user.domain
level: extended
type: keyword
Expand Down Expand Up @@ -749,6 +773,18 @@
list (http://publicsuffix.org). Trying to approximate this by simply taking
the last two labels will not work well for TLDs such as "co.uk".'
example: google.com
- name: question.top_level_domain
level: extended
type: keyword
ignore_above: 1024
description: 'The effective top level domain (eTLD), also known as the domain
suffix, is the last part of the domain name. For example, the top level domain
for google.com is "com".

This value can be determined precisely with a list like the public suffix
list (http://publicsuffix.org). Trying to approximate this by simply taking
the last label will not work well for effective TLDs such as "co.uk".'
example: co.uk
- name: question.type
level: extended
type: keyword
Expand Down Expand Up @@ -2302,6 +2338,18 @@
list (http://publicsuffix.org). Trying to approximate this by simply taking
the last two labels will not work well for TLDs such as "co.uk".'
example: google.com
- name: top_level_domain
level: extended
type: keyword
ignore_above: 1024
description: 'The effective top level domain (eTLD), also known as the domain
suffix, is the last part of the domain name. For example, the top level domain
for google.com is "com".

This value can be determined precisely with a list like the public suffix
list (http://publicsuffix.org). Trying to approximate this by simply taking
the last label will not work well for effective TLDs such as "co.uk".'
example: co.uk
- name: user.domain
level: extended
type: keyword
Expand Down Expand Up @@ -2586,6 +2634,18 @@
list (http://publicsuffix.org). Trying to approximate this by simply taking
the last two labels will not work well for TLDs such as "co.uk".'
example: google.com
- name: top_level_domain
level: extended
type: keyword
ignore_above: 1024
description: 'The effective top level domain (eTLD), also known as the domain
suffix, is the last part of the domain name. For example, the top level domain
for google.com is "com".

This value can be determined precisely with a list like the public suffix
list (http://publicsuffix.org). Trying to approximate this by simply taking
the last label will not work well for effective TLDs such as "co.uk".'
example: co.uk
- name: user.domain
level: extended
type: keyword
Expand Down Expand Up @@ -2757,6 +2817,18 @@

Note: The `:` is not part of the scheme.'
example: https
- name: top_level_domain
level: extended
type: keyword
ignore_above: 1024
description: 'The effective top level domain (eTLD), also known as the domain
suffix, is the last part of the domain name. For example, the top level domain
for google.com is "com".

This value can be determined precisely with a list like the public suffix
list (http://publicsuffix.org). Trying to approximate this by simply taking
the last label will not work well for effective TLDs such as "co.uk".'
example: co.uk
- name: username
level: extended
type: keyword
Expand Down
6 changes: 6 additions & 0 deletions generated/csv/fields.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ client.nat.port,long,extended,,1.2.0-dev
client.packets,long,core,12,1.2.0-dev
client.port,long,core,,1.2.0-dev
client.registered_domain,keyword,extended,google.com,1.2.0-dev
client.top_level_domain,keyword,extended,co.uk,1.2.0-dev
client.user.domain,keyword,extended,,1.2.0-dev
client.user.email,keyword,extended,,1.2.0-dev
client.user.full_name,keyword,extended,Albert Einstein,1.2.0-dev
Expand Down Expand Up @@ -72,6 +73,7 @@ destination.nat.port,long,extended,,1.2.0-dev
destination.packets,long,core,12,1.2.0-dev
destination.port,long,core,,1.2.0-dev
destination.registered_domain,keyword,extended,google.com,1.2.0-dev
destination.top_level_domain,keyword,extended,co.uk,1.2.0-dev
destination.user.domain,keyword,extended,,1.2.0-dev
destination.user.email,keyword,extended,,1.2.0-dev
destination.user.full_name,keyword,extended,Albert Einstein,1.2.0-dev
Expand All @@ -93,6 +95,7 @@ dns.op_code,keyword,extended,QUERY,1.2.0-dev
dns.question.class,keyword,extended,IN,1.2.0-dev
dns.question.name,keyword,extended,www.google.com,1.2.0-dev
dns.question.registered_domain,keyword,extended,google.com,1.2.0-dev
dns.question.top_level_domain,keyword,extended,co.uk,1.2.0-dev
dns.question.type,keyword,extended,AAAA,1.2.0-dev
dns.resolved_ip,ip,extended,"['10.10.10.10', '10.10.10.11']",1.2.0-dev
dns.response_code,keyword,extended,NOERROR,1.2.0-dev
Expand Down Expand Up @@ -291,6 +294,7 @@ server.nat.port,long,extended,,1.2.0-dev
server.packets,long,core,12,1.2.0-dev
server.port,long,core,,1.2.0-dev
server.registered_domain,keyword,extended,google.com,1.2.0-dev
server.top_level_domain,keyword,extended,co.uk,1.2.0-dev
server.user.domain,keyword,extended,,1.2.0-dev
server.user.email,keyword,extended,,1.2.0-dev
server.user.full_name,keyword,extended,Albert Einstein,1.2.0-dev
Expand Down Expand Up @@ -327,6 +331,7 @@ source.nat.port,long,extended,,1.2.0-dev
source.packets,long,core,12,1.2.0-dev
source.port,long,core,,1.2.0-dev
source.registered_domain,keyword,extended,google.com,1.2.0-dev
source.top_level_domain,keyword,extended,co.uk,1.2.0-dev
source.user.domain,keyword,extended,,1.2.0-dev
source.user.email,keyword,extended,,1.2.0-dev
source.user.full_name,keyword,extended,Albert Einstein,1.2.0-dev
Expand All @@ -348,6 +353,7 @@ url.port,long,extended,443,1.2.0-dev
url.query,keyword,extended,,1.2.0-dev
url.registered_domain,keyword,extended,google.com,1.2.0-dev
url.scheme,keyword,extended,https,1.2.0-dev
url.top_level_domain,keyword,extended,co.uk,1.2.0-dev
url.username,keyword,extended,,1.2.0-dev
user.domain,keyword,extended,,1.2.0-dev
user.email,keyword,extended,,1.2.0-dev
Expand Down
Loading