Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add explicit username and password option for Mongodb #2889

Merged
merged 1 commit into from
Oct 31, 2016
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Add explicit username and password option for Mongodb
We previously only had the option of specifying a user/pass in the
URL string, which is problematic because it results in the password
being indexed in Elasticsearch (#2888).

This adds the option to specify a username/password at the module
configuration. To make this happen, I had to copy some unexported
functions from the mgo driver.
Tudor Golubenco committed Oct 31, 2016

Verified

This commit was signed with the committer’s verified signature.
mbklein Michael B. Klein
commit c03da67b241fa0a868388d2467362a1934264767
1 change: 1 addition & 0 deletions CHANGELOG.asciidoc
Original file line number Diff line number Diff line change
@@ -53,6 +53,7 @@ https://github.com/elastic/beats/compare/v5.0.0...master[Check the HEAD diff]
- Add experimental filebeat metricset in the beats module. {pull}2297[2297]
- Add experimental libbeat metricset in the beats module. {pull}2339[2339]
- Add experimental docker module. Provided by Ingensi and @douaejeouit based on dockbeat.
- Add username and password config options to the MongoDB module. {pull}2889[2889]

*Packetbeat*
- Define `client_geoip.location` as geo_point in the mappings to be used by the GeoIP processor in the Ingest Node pipeline.
28 changes: 26 additions & 2 deletions metricbeat/docs/modules/mongodb.asciidoc
Original file line number Diff line number Diff line change
@@ -14,7 +14,7 @@ servers.
When configuring the `hosts` option, you must use MongoDB URLs of the following format:

-----------------------------------
[mongodb://][user:pass@]host[:port]
[mongodb://host[:port][?options]
-----------------------------------

The URL can be as simple as:
@@ -33,6 +33,21 @@ Or more complex like:
hosts: ["mongodb://myuser:mypass@localhost:40001", "otherhost:40001"]
----------------------------------------------------------------------

WARNING: In case you use username and password in the hosts array, this
information will be sent with each event as part of the `metricset.host` field.
To prevent sending username and password the config options `username` and
`password` can be used.

[source,yaml]
----
- module: mongodb
metricsets: ["status"]
hosts: ["localhost:27017"]
username: root
password: test
----



[float]
=== Compatibility
@@ -55,9 +70,18 @@ metricbeat.modules:
#period: 10s

# The hosts must be passed as MongoDB URLs in the format:
# [mongodb://][user:pass@]host[:port]
# [mongodb://][user:pass@]host[:port].
# Warning: specifying the user/password in the hosts array is possible
# but it will result in the password being present in the output documents.
# We recommend using the username and password options instead.
#hosts: ["localhost:27017"]

# Username to use when connecting to MongoDB. Empty by default.
#username: user

# Password to use when connecting to MongoDB. Empty by default.
#password: pass

----

[float]
11 changes: 10 additions & 1 deletion metricbeat/etc/beat.full.yml
Original file line number Diff line number Diff line change
@@ -105,9 +105,18 @@ metricbeat.modules:
#period: 10s

# The hosts must be passed as MongoDB URLs in the format:
# [mongodb://][user:pass@]host[:port]
# [mongodb://][user:pass@]host[:port].
# Warning: specifying the user/password in the hosts array is possible
# but it will result in the password being present in the output documents.
# We recommend using the username and password options instead.
#hosts: ["localhost:27017"]

# Username to use when connecting to MongoDB. Empty by default.
#username: user

# Password to use when connecting to MongoDB. Empty by default.
#password: pass


#-------------------------------- MySQL Module -------------------------------
#- module: mysql
11 changes: 10 additions & 1 deletion metricbeat/metricbeat.full.yml
Original file line number Diff line number Diff line change
@@ -105,9 +105,18 @@ metricbeat.modules:
#period: 10s

# The hosts must be passed as MongoDB URLs in the format:
# [mongodb://][user:pass@]host[:port]
# [mongodb://][user:pass@]host[:port].
# Warning: specifying the user/password in the hosts array is possible
# but it will result in the password being present in the output documents.
# We recommend using the username and password options instead.
#hosts: ["localhost:27017"]

# Username to use when connecting to MongoDB. Empty by default.
#username: user

# Password to use when connecting to MongoDB. Empty by default.
#password: pass


#-------------------------------- MySQL Module -------------------------------
#- module: mysql
11 changes: 10 additions & 1 deletion metricbeat/module/mongodb/_meta/config.yml
Original file line number Diff line number Diff line change
@@ -4,6 +4,15 @@
#period: 10s

# The hosts must be passed as MongoDB URLs in the format:
# [mongodb://][user:pass@]host[:port]
# [mongodb://][user:pass@]host[:port].
# Warning: specifying the user/password in the hosts array is possible
# but it will result in the password being present in the output documents.
# We recommend using the username and password options instead.
#hosts: ["localhost:27017"]

# Username to use when connecting to MongoDB. Empty by default.
#username: user

# Password to use when connecting to MongoDB. Empty by default.
#password: pass

17 changes: 16 additions & 1 deletion metricbeat/module/mongodb/_meta/docs.asciidoc
Original file line number Diff line number Diff line change
@@ -9,7 +9,7 @@ servers.
When configuring the `hosts` option, you must use MongoDB URLs of the following format:

-----------------------------------
[mongodb://][user:pass@]host[:port]
[mongodb://host[:port][?options]
-----------------------------------

The URL can be as simple as:
@@ -28,6 +28,21 @@ Or more complex like:
hosts: ["mongodb://myuser:mypass@localhost:40001", "otherhost:40001"]
----------------------------------------------------------------------

WARNING: In case you use username and password in the hosts array, this
information will be sent with each event as part of the `metricset.host` field.
To prevent sending username and password the config options `username` and
`password` can be used.

[source,yaml]
----
- module: mongodb
metricsets: ["status"]
hosts: ["localhost:27017"]
username: root
password: test
----



[float]
=== Compatibility
123 changes: 123 additions & 0 deletions metricbeat/module/mongodb/parseurl.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,123 @@
package mongodb

import (
"errors"
"fmt"
"net/url"
"strings"

mgo "gopkg.in/mgo.v2"
)

/*
* Functions copied from the mgo driver to help with parsing the URL.
*
* http://bazaar.launchpad.net/+branch/mgo/v2/view/head:/session.go#L382
*/

type urlInfo struct {
addrs []string
user string
pass string
db string
options map[string]string
}

func parseURL(s string) (*urlInfo, error) {
if strings.HasPrefix(s, "mongodb://") {
s = s[10:]
}
info := &urlInfo{options: make(map[string]string)}
if c := strings.Index(s, "?"); c != -1 {
for _, pair := range strings.FieldsFunc(s[c+1:], isOptSep) {
l := strings.SplitN(pair, "=", 2)
if len(l) != 2 || l[0] == "" || l[1] == "" {
return nil, errors.New("connection option must be key=value: " + pair)
}
info.options[l[0]] = l[1]
}
s = s[:c]
}
if c := strings.Index(s, "@"); c != -1 {
pair := strings.SplitN(s[:c], ":", 2)
if len(pair) > 2 || pair[0] == "" {
return nil, errors.New("credentials must be provided as user:pass@host")
}
var err error
info.user, err = url.QueryUnescape(pair[0])
if err != nil {
return nil, fmt.Errorf("cannot unescape username in URL: %q", pair[0])
}
if len(pair) > 1 {
info.pass, err = url.QueryUnescape(pair[1])
if err != nil {
return nil, fmt.Errorf("cannot unescape password in URL")
}
}
s = s[c+1:]
}
if c := strings.Index(s, "/"); c != -1 {
info.db = s[c+1:]
s = s[:c]
}
info.addrs = strings.Split(s, ",")
return info, nil
}

func isOptSep(c rune) bool {
return c == ';' || c == '&'
}

// ParseURL parses the given URL and returns a DialInfo structure ready
// to be passed to DialWithInfo
func ParseURL(host, username, pass string) (*mgo.DialInfo, error) {
uinfo, err := parseURL(host)
if err != nil {
return nil, err
}
direct := false
mechanism := ""
service := ""
source := ""
for k, v := range uinfo.options {
switch k {
case "authSource":
source = v
case "authMechanism":
mechanism = v
case "gssapiServiceName":
service = v
case "connect":
if v == "direct" {
direct = true
break
}
if v == "replicaSet" {
break
}
fallthrough
default:
return nil, errors.New("unsupported connection URL option: " + k + "=" + v)
}
}

info := &mgo.DialInfo{
Addrs: uinfo.addrs,
Direct: direct,
Database: uinfo.db,
Username: uinfo.user,
Password: uinfo.pass,
Mechanism: mechanism,
Service: service,
Source: source,
}

if len(username) > 0 {
info.Username = username
}
if len(pass) > 0 {
info.Password = pass
}

return info, nil
}
78 changes: 78 additions & 0 deletions metricbeat/module/mongodb/parseurl_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
package mongodb

import (
"testing"

"github.com/stretchr/testify/assert"
)

func TestParseURL(t *testing.T) {
tests := []struct {
Name string
URL string
Username string
Password string
ExpectedAddr string
ExpectedUsername string
ExpectedPassword string
}{
{
Name: "basic test",
URL: "localhost:40001",
Username: "user",
Password: "secret",

ExpectedAddr: "localhost:40001",
ExpectedUsername: "user",
ExpectedPassword: "secret",
},
{
Name: "with schema",
URL: "mongodb://localhost:40001",
Username: "user",
Password: "secret",

ExpectedAddr: "localhost:40001",
ExpectedUsername: "user",
ExpectedPassword: "secret",
},
{
Name: "user password in url",
URL: "mongodb://user:secret@localhost:40001",
Username: "",
Password: "",

ExpectedAddr: "localhost:40001",
ExpectedUsername: "user",
ExpectedPassword: "secret",
},
{
Name: "user password overwride",
URL: "mongodb://user:secret@localhost:40001",
Username: "anotheruser",
Password: "anotherpass",

ExpectedAddr: "localhost:40001",
ExpectedUsername: "anotheruser",
ExpectedPassword: "anotherpass",
},
{
Name: "with options",
URL: "mongodb://localhost:40001?connect=direct&authSource=me",
Username: "anotheruser",
Password: "anotherpass",

ExpectedAddr: "localhost:40001",
ExpectedUsername: "anotheruser",
ExpectedPassword: "anotherpass",
},
}

for _, test := range tests {
info, err := ParseURL(test.URL, test.Username, test.Password)
assert.NoError(t, err, test.Name)
assert.Equal(t, info.Addrs[0], test.ExpectedAddr, test.Name)
assert.Equal(t, info.Username, test.ExpectedUsername, test.Name)
assert.Equal(t, info.Password, test.ExpectedPassword, test.Name)
}
}
Loading