Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

patterns: add various detection patterns #1

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

patterns: add various detection patterns #1

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
129 changes: 129 additions & 0 deletions traffic-id.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,21 @@ tls-sni-patterns:
patterns:
- googlevideo.com

- id: google-tracking
labels:
- tracking
- stats
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We had discussed trying to stick close to Qosmos, or other tools that had already done some labelling. In which case, "stats" would be "analytics". They don't have one for tracking, so I think tracking is good.

patterns:
- www.google-analytics.com

- id: google-ads
labels:
- tracking
- ads
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similarily, I wonder if this should be "advertising".

patterns:
- .doubleclick.net
- www.googleadservices.com

# IRCCloud
- id: irccloud
label: im
Expand Down Expand Up @@ -102,6 +117,22 @@ tls-sni-patterns:
- feelinsonice-hrd.appspot.com
- snapchat.com

# Spotify
- id: spotify
labels:
- music
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should Spotify be streaming as well?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wonder if we need a 'streaming' as well as 'video' and 'music' in general?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or should we use "streaming-music" and "streaming-video"?

patterns:
- spclient.wg.spotify.com
- api-partner.spotify.com
- scdn.co

- id: mixcloud
labels:
- music
- streaming
patterns:
- mixcloud.com

# Twitter
- id: twitter
labels:
Expand All @@ -126,6 +157,84 @@ tls-sni-patterns:
- instagram.com
- cdninstagram.com

- id: feedly
labels:
- news
- rss
patterns:
- feedly.com

- id: nos
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a longer ID that can be used? My feeling is that this takes up a short name that could potentially be used for other IDs in the future. Not sure what.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

perhaps a country namespace? e.g. nl/nos or NL-nos or something along those lines. Guess the forward slash might confuse the flowbits that are set.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Namespace is a good idea. Not sure if a / would be an issue for not. The logger only looks for a prefix of "traffic/id/" then takes the rest as the ID. So "nl/nos" should be logged as:

{
  "traffic": {
    "id": ["nl/nos"]
  }
}

which I think could be a good option.

labels:
- news
patterns:
- nos.nl

- id: chartbeat
labels:
- stats
- tracking
patterns:
- chartbeat.net

- id: github
labels:
- dev
patterns:
- github.com

- id: github-tracking
labels:
- dev
- stats
- tracking
patterns:
- collector.githubapp.com

- id: mozilla
labels:
- browser
patterns:
- firefoxusercontent.com
- services.mozilla.com

- id: rememberthemilk
labels:
- todo
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

perhaps here we could have something like 'pim' (personal information management) ?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe. But again, but this would only apply to a subset as well. Asana for instance is more enterprisey todo/project management. Which remember the milk can do as well, so "personal" is misleading here.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if this is better labeled task-management or something. There is overlapping functionality with tools like Trello, Asana and Omnifocus, which are way more than "todo" apps, but close enough that I'd like them all labelled together.

patterns:
- rememberthemilk.com

- id: docker
labels:
- devops
patterns:
- registry-1.docker.io
- auth.docker.io

- id: rust-lang
labels:
- dev
patterns:
- static.crates.io

- id: oisf-dev-infra
labels:
- dev
patterns:
- buildbot.openinfosecfoundation.org

- id: apple
labels:
- software-update
patterns:
- .push.apple.com

- id: apple-icloud
labels:
- cloud
patterns:
- setup.icloud.com

# Traffic identification where more than simple lists of a specific
# pattern are required.
rules:
Expand All @@ -148,6 +257,24 @@ rules:
http_host: ubuntu.com
http_user_agent: Debian APT

# Ubuntu Updater
- id: ubuntu-swupdate
msg: "Ubuntu Software-Update"
labels:
- software-update
proto: http
http_host: changelogs.ubuntu.com
http_user_agent: Python-urllib/3.5

# Steam Game downloads
- id: steam
msg: "Steam"
labels:
- game
proto: http
http_host: .steamcontent.com
http_user_agent: Valve/Steam HTTP Client

# Map short ID names to full names (or description)
id-map:
bing: Bing
Expand All @@ -159,6 +286,7 @@ id-map:
netflix: Netflix
skype: Skype
snapchat: Snapchat
spotify: Spotify
twitter: Twitter
whatsapp: WhatsApp Messenger
whisper: Signal messaging application
Expand All @@ -171,4 +299,5 @@ labels:
chat: Chat
file-transfer: File Transfer
im: Instant Messaging
music: Music Stream Service
video: Video Stream Service