Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[docker] TLS support in docker connection / docker-py #1299

Closed
gregtap opened this issue Jan 14, 2015 · 4 comments · Fixed by #1908
Closed

[docker] TLS support in docker connection / docker-py #1299

gregtap opened this issue Jan 14, 2015 · 4 comments · Fixed by #1908
Assignees
@hkaj
Labels
bugfix
Milestone
5.5.0

Comments

@gregtap
Copy link

gregtap commented Jan 14, 2015

The recommended way of running docker is with TLS support:

export DOCKER_HOST=tcp://foobar.com:2376
export DOCKER_TLS_VERIFY=1
export DOCKER_CERT_PATH=/etc/tls/docker

It seems that there is no way for dd-agent to pick up those env variables, the urllib call therefore fails to establish a connection.
@remh remh added the bugfix label Jan 14, 2015
@gregtap
Copy link
Author

gregtap commented Jan 14, 2015

Bonus: I can connect to my docker daemon using docker-py:

from docker import Client, tls

tls_config = tls.TLSConfig(
    client_cert=('/home/foo/docker/cert.pem',    '/home/foo/docker/key.pem'),
    verify=False
)
c = Client(base_url='https://localhost:2376', tls=tls_config)
print c.containers()

Could dd-agent use docker-py instead of urllib?

@LotharSee
Copy link
Contributor

Thanks @coulix for this detailed explanation!

The main problem is that the current check doesn't support TLS. It shouldn't be hard to add its support to our current code (by using httplib.HTTPSConnection) but it's certainly the good moment to switch to docker-py.

Concerning the env. variables, it's a wider issue (the Agent doesn't use env. variables in its checks) which should be addressed more globally.

So initially we will have these TCP_VERIFY and CERT_PATH options in the configuration docker.yaml.

@LeoCavaille LeoCavaille added this to the 5.3.0 milestone Jan 23, 2015
@LeoCavaille LeoCavaille changed the title Docker agent reporting with TLS [docker] TLS support in docker connection / docker-py Jan 28, 2015
@gregtap
Copy link
Author

gregtap commented Feb 12, 2015

News

With the 1.5 release Docker exposes a stats endpoint that could maybe be used by data-dog to collect containers metrics.

https://blog.docker.com/2015/02/docker-1-5-ipv6-support-read-only-containers-stats-named-dockerfiles-and-more/

moby/moby#9984

@adzuci adzuci mentioned this issue Feb 25, 2015
Closed
@remh
Copy link

remh commented Apr 2, 2015

@LotharSee Changing milestone to 5.4.0

@remh remh modified the milestones: 5.4.0, 5.3.0 Apr 2, 2015
@yannmh yannmh modified the milestones: MacOSApp, 5.4.0, To be determined Jun 8, 2015
@hkaj hkaj modified the milestones: 5.5.0, To be determined Jun 24, 2015
@hkaj hkaj self-assigned this Jun 24, 2015
remh pushed a commit that referenced this issue Sep 11, 2015
[docker] New docker check
8219191 
This is a new version of the docker check, it's called docker_daemon (name can be changed).
The previous check "docker" is now deprecated and won't receive further support.

In terms of features, this adds:
- Support for TLS connections to the daemon
- New metrics:
    - Network metrics
    - Memory limits
    - Container size (rootfs)
    - Image size
- Support for labels (convert them into tags). Off by default, uses a list of labels that should be converted.
- Support for ECS tags: task name and task version

Backward incompatible changes:

    - docker.disk.size metric is renamed to docker.container.size_rw
    - Old optional metrics: https://github.com/DataDog/dd-agent/blob/5.4.x/checks.d/docker.py#L29-L38 Are not collected anymore
    - Old old tags are not supported anymore (e.g. `name` instead of container_name)

fix: #1299 #1648 #1739 #1742 #1896
remh pushed a commit that referenced this issue Sep 11, 2015
[docker] New docker check
e977301 
This is a new version of the docker check, it's called docker_daemon (name can be changed).
The previous check "docker" is now deprecated and won't receive further support.

In terms of features, this adds:
- Support for TLS connections to the daemon
- New metrics:
    - Network metrics
    - Memory limits
    - Container size (rootfs)
    - Image size
- Support for labels (convert them into tags). Off by default, uses a list of labels that should be converted.
- Support for ECS tags: task name and task version

Backward incompatible changes:

    - docker.disk.size metric is renamed to docker.container.size_rw
    - Old optional metrics: https://github.com/DataDog/dd-agent/blob/5.4.x/checks.d/docker.py#L29-L38 Are not collected anymore
    - Old old tags are not supported anymore (e.g. `name` instead of container_name)

fix: #1299 #1648 #1739 #1742 #1896
@remh remh mentioned this issue Sep 11, 2015
Merged
remh pushed a commit that referenced this issue Sep 11, 2015
[docker] New docker check
6921325 
This is a new version of the docker check, it's called docker_daemon (name can be changed).
The previous check "docker" is now deprecated and won't receive further support.

In terms of features, this adds:
- Support for TLS connections to the daemon
- New metrics:
    - Network metrics
    - Memory limits
    - Container size (rootfs)
    - Image size
- Support for labels (convert them into tags). Off by default, uses a list of labels that should be converted.
- Support for ECS tags: task name and task version

Backward incompatible changes:

    - docker.disk.size metric is renamed to docker.container.size_rw
    - Old optional metrics: https://github.com/DataDog/dd-agent/blob/5.4.x/checks.d/docker.py#L29-L38 Are not collected anymore
    - Old old tags are not supported anymore (e.g. `name` instead of container_name)

fix: #1299 #1648 #1739 #1742 #1896
remh pushed a commit that referenced this issue Sep 11, 2015
[docker] New docker check
7efba53 
This is a new version of the docker check, it's called docker_daemon (name can be changed).
The previous check "docker" is now deprecated and won't receive further support.

In terms of features, this adds:
- Support for TLS connections to the daemon
- New metrics:
    - Network metrics
    - Memory limits
    - Container size (rootfs)
    - Image size
- Support for labels (convert them into tags). Off by default, uses a list of labels that should be converted.
- Support for ECS tags: task name and task version

Backward incompatible changes:

    - docker.disk.size metric is renamed to docker.container.size_rw
    - Old optional metrics: https://github.com/DataDog/dd-agent/blob/5.4.x/checks.d/docker.py#L29-L38 Are not collected anymore
    - Old old tags are not supported anymore (e.g. `name` instead of container_name)

fix: #1299 #1648 #1739 #1742 #1896
remh pushed a commit that referenced this issue Sep 14, 2015
[docker] New docker check
71d1b5b 
This is a new version of the docker check, it's called docker_daemon (name can be changed).
The previous check "docker" is now deprecated and won't receive further support.

In terms of features, this adds:
- Support for TLS connections to the daemon
- New metrics:
    - Network metrics
    - Memory limits
    - Container size (rootfs)
    - Image size
- Support for labels (convert them into tags). Off by default, uses a list of labels that should be converted.
- Support for ECS tags: task name and task version

Backward incompatible changes:

    - docker.disk.size metric is renamed to docker.container.size_rw
    - Old optional metrics: https://github.com/DataDog/dd-agent/blob/5.4.x/checks.d/docker.py#L29-L38 Are not collected anymore
    - Old old tags are not supported anymore (e.g. `name` instead of container_name)

fix: #1299 #1648 #1739 #1742 #1896
remh pushed a commit that referenced this issue Sep 14, 2015
[docker] New docker check
0aac132 
This is a new version of the docker check, it's called docker_daemon (name can be changed).
The previous check "docker" is now deprecated and won't receive further support.

In terms of features, this adds:
- Support for TLS connections to the daemon
- New metrics:
    - Network metrics
    - Memory limits
    - Container size (rootfs)
    - Image size
- Support for labels (convert them into tags). Off by default, uses a list of labels that should be converted.
- Support for ECS tags: task name and task version

Backward incompatible changes:

    - docker.disk.size metric is renamed to docker.container.size_rw
    - Old optional metrics: https://github.com/DataDog/dd-agent/blob/5.4.x/checks.d/docker.py#L29-L38 Are not collected anymore
    - Old old tags are not supported anymore (e.g. `name` instead of container_name)

fix: #1299 #1648 #1739 #1742 #1896
remh pushed a commit that referenced this issue Sep 14, 2015
[docker] New docker check
c430b41 
This is a new version of the docker check, it's called docker_daemon (name can be changed).
The previous check "docker" is now deprecated and won't receive further support.

In terms of features, this adds:
- Support for TLS connections to the daemon
- New metrics:
    - Network metrics
    - Memory limits
    - Container size (rootfs)
    - Image size
- Support for labels (convert them into tags). Off by default, uses a list of labels that should be converted.
- Support for ECS tags: task name and task version

Backward incompatible changes:

    - docker.disk.size metric is renamed to docker.container.size_rw
    - Old optional metrics: https://github.com/DataDog/dd-agent/blob/5.4.x/checks.d/docker.py#L29-L38 Are not collected anymore
    - Old old tags are not supported anymore (e.g. `name` instead of container_name)

fix: #1299 #1648 #1739 #1742 #1896
remh pushed a commit that referenced this issue Sep 15, 2015
[docker] New docker check
96c0305 
This is a new version of the docker check, it's called docker_daemon (name can be changed).
The previous check "docker" is now deprecated and won't receive further support.

In terms of features, this adds:
- Support for TLS connections to the daemon
- New metrics:
    - Network metrics
    - Memory limits
    - Container size (rootfs)
    - Image size
- Support for labels (convert them into tags). Off by default, uses a list of labels that should be converted.
- Support for ECS tags: task name and task version

Backward incompatible changes:

    - docker.disk.size metric is renamed to docker.container.size_rw
    - Old optional metrics: https://github.com/DataDog/dd-agent/blob/5.4.x/checks.d/docker.py#L29-L38 Are not collected anymore
    - Old old tags are not supported anymore (e.g. `name` instead of container_name)

fix: #1299 #1648 #1739 #1742 #1896
urosgruber pushed a commit to urosgruber/dd-agent that referenced this issue Dec 23, 2015
@urosgruber
[docker] New docker check
5fd887f 
This is a new version of the docker check, it's called docker_daemon (name can be changed).
The previous check "docker" is now deprecated and won't receive further support.

In terms of features, this adds:
- Support for TLS connections to the daemon
- New metrics:
    - Network metrics
    - Memory limits
    - Container size (rootfs)
    - Image size
- Support for labels (convert them into tags). Off by default, uses a list of labels that should be converted.
- Support for ECS tags: task name and task version

Backward incompatible changes:

    - docker.disk.size metric is renamed to docker.container.size_rw
    - Old optional metrics: https://github.com/DataDog/dd-agent/blob/5.4.x/checks.d/docker.py#L29-L38 Are not collected anymore
    - Old old tags are not supported anymore (e.g. `name` instead of container_name)

fix: DataDog#1299 DataDog#1648 DataDog#1739 DataDog#1742 DataDog#1896
urosgruber pushed a commit to urosgruber/dd-agent that referenced this issue Dec 23, 2015
@urosgruber
[docker] New docker check
602be7c 
This is a new version of the docker check, it's called docker_daemon (name can be changed).
The previous check "docker" is now deprecated and won't receive further support.

In terms of features, this adds:
- Support for TLS connections to the daemon
- New metrics:
    - Network metrics
    - Memory limits
    - Container size (rootfs)
    - Image size
- Support for labels (convert them into tags). Off by default, uses a list of labels that should be converted.
- Support for ECS tags: task name and task version

Backward incompatible changes:

    - docker.disk.size metric is renamed to docker.container.size_rw
    - Old optional metrics: https://github.com/DataDog/dd-agent/blob/5.4.x/checks.d/docker.py#L29-L38 Are not collected anymore
    - Old old tags are not supported anymore (e.g. `name` instead of container_name)

fix: DataDog#1299 DataDog#1648 DataDog#1739 DataDog#1742 DataDog#1896
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@hkaj hkaj

Labels
bugfix
Projects
None yet
Milestone
5.5.0
Development

Successfully merging a pull request may close this issue.

[Docker check] V2 DataDog/dd-agent
6 participants
@gregtap @remh @LeoCavaille @LotharSee @hkaj @yannmh