integration with docker and CoreOS

[gregwebs]

Not sure what the best approach is here. One could attempt to run packetbeat on the host that runs docker images. But on CoreOS the OS is read-only and everything is supposed to be ran from a container. So it might make more sense to run this inside a docker container.

[dansowter]

Started down this road -- I'm keen to run Packetbeat up in a container on CoreOS.

Had a crack based on ubuntu:trusty, and the absence of syslog led to unhappiness.

Error opening syslog:  Unix syslog delivery error
Error opening syslog:  Unix syslog delivery error
Error opening syslog:  Unix syslog delivery error
Error opening syslog:  Unix syslog delivery error
Error opening syslog:  Unix syslog delivery error
Error opening syslog:  Unix syslog delivery error
Error opening syslog:  Unix syslog delivery error
Error opening syslog:  Unix syslog delivery error
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x0 pc=0x4a0e7b]

goroutine 1 [running]:
runtime.panic(0x6fd3c0, 0xd16908)
    /usr/local/go/src/pkg/runtime/panic.c:266 +0xb6
log.(*Logger).Output(0x0, 0x2, 0xc2100ae5b0, 0xc6, 0x0, ...)
    /usr/local/go/src/pkg/log/log.go:134 +0x46b
main.ERR(0x7ae7f0, 0x1c, 0x7ffa234f38f0, 0x1, 0x1)
    /home/vagrant/src/packetbeat/log_unix.go:82 +0x99
main.(*ElasticsearchOutputType).Init(0xd2f580, 0x101, 0xc21000b7c0, 0x3b, 0x241b, ...)
    /home/vagrant/src/packetbeat/output_elasticsearch.go:51 +0x2af
main.(*PublisherType).Init(0xd2f6e0, 0xc2100a0200, 0x62, 0x0)
    /home/vagrant/src/packetbeat/publish.go:279 +0x196
main.main()
    /home/vagrant/src/packetbeat/main.go:284 +0xe03

goroutine 3 [chan receive]:
github.com/golang/glog.(*loggingT).flushDaemon(0xd2fba0)
    /home/vagrant/src/github.com/golang/glog/glog.go:839 +0x50
created by github.com/golang/glog.init·1
    /home/vagrant/src/github.com/golang/glog/glog.go:406 +0x276

goroutine 4 [syscall]:
runtime.goexit()
    /usr/local/go/src/pkg/runtime/proc.c:1394

goroutine 6 [chan receive]:
github.com/bitly/go-hostpool.(*epsilonGreedyHostPool).epsilonGreedyDecay(0xc21009d980)
    /home/vagrant/src/github.com/bitly/go-hostpool/epsilon_greedy.go:78 +0x71
created by github.com/bitly/go-hostpool.NewEpsilonGreedy
    /home/vagrant/src/github.com/bitly/go-hostpool/epsilon_greedy.go:64 +0x240

Trying again based on the following Dockerfile (I'll share the rest soon)

FROM phusion/baseimage:0.9.12
MAINTAINER [email protected]

RUN apt-get update
RUN apt-get install libpcap0.8

ADD packetbeat_0.3.1-1_amd64.deb /tmp/packetbeat_0.3.1-1_amd64.deb
RUN dpkg -i /tmp/packetbeat_0.3.1-1_amd64.deb
RUN apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*

ADD packetbeat.conf /etc/packetbeat/packetbeat.conf

RUN mkdir /etc/service/packetbeat
ADD run /etc/service/packetbeat/run
RUN chmod +x /etc/service/packetbeat/run

CMD ["/sbin/my_init"]

Was much happier. I made sure to run the container with the "--net=host" flag, so Packetbeat should be able to spy on the network activity of all other containers on the box.

Some cursory curling around on the host OS was reporting successfully in the Kibana console, so it looks like this will work.

Anything else I should know before I dig further?

[packetb-old]

I don't know if it helps, but you can disable syslog with the -e flag. We didn't play yet with docker, but it's great news that you can sniff the traffic from inside a container. Thanks for sharing this.

[tsg]

I played with this during the DockerHackDay and created this repo: https://github.com/packetbeat/packetbeat-docker

[mingfang]

I just make a Dockerfile that runs the entire stack inside Docker.
https://github.com/mingfang/docker-packetbeat.git