Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Docker for mac compatibility #2838

Open
dcs3spp opened this issue Mar 25, 2021 · 8 comments
Open

Docker for mac compatibility #2838

dcs3spp opened this issue Mar 25, 2021 · 8 comments

Comments

@dcs3spp
Copy link

dcs3spp commented Mar 25, 2021 •
edited
Loading

Hi,

I am using docker image gcr.io/cadvisor/cadvisor:v0.39.0 with the following docker-compose service on macOS BigSur 11.2.3. I have included output from docker version and docker info below.

When I start up the docker-compose stack I receive the following warnings/errors:

cadvisor      | W0325 13:57:18.842825       1 machine_libipmctl.go:62] There are no NVM devices!
cadvisor      | W0325 13:57:18.847659       1 sysinfo.go:203] Nodes topology is not available, providing CPU topology
cadvisor      | W0325 13:57:18.854598       1 info.go:53] Couldn't collect info from any of the files in "/etc/machine-id,/var/lib/dbus/machine-id"
cadvisor      | W0325 13:57:18.980091       1 manager.go:288] Could not configure a source for OOM detection, disabling OOM events: open /dev/kmsg: no such file or directory
cadvisor      | time="2021-03-25T13:57:18Z" level=warning msg="falling back to securejoin: openat2 /sys/fs/cgroup: operation not permitted"
cadvisor      | W0325 13:57:19.087235       1 container.go:586] Failed to update stats for container "/kubepods": /sys/fs/cgroup/cpuset/kubepods/cpuset.cpus found to be empty, continuing to push stats
cadvisor      | W0325 13:57:39.321263       1 prometheus.go:1856] Couldn't get containers: partial failures: ["/kubepods": containerDataToContainerInfo: unable to find data in memory cache]
cadvisor      | W0325 13:57:54.300026       1 prometheus.go:1856] Couldn't get containers: partial failures: ["/kubepods": containerDataToContainerInfo: unable to find data in memory cache]
cadvisor      | W0325 13:58:09.309618       1 prometheus.go:1856] Couldn't get containers: partial failures: ["/kubepods": containerDataToContainerInfo: unable to find data in memory cache]

Does cadvisor have limited compatibility with docker for mac?

Docker-compose cadvisor service v0.39.0

cadvisor:
    image: gcr.io/cadvisor/cadvisor:v0.39.0
    container_name: cadvisor   
    ports:
      - "8081:8080"            
    links:
      - flask1
      - flask2
      - flask3
    volumes:
      - /var/run:/var/run:ro
      - /sys:/sys:ro
      - /var/lib/docker/:/var/lib/docker:ro
      - /var/run/docker.sock:/var/run/docker.sock:ro

If I explicitly add machine-id volumes, as suggested here then that seems to solve related error messages.

cadvisor:
    image: gcr.io/cadvisor/cadvisor:v0.39.0
    container_name: cadvisor   
    ports:
      - "8081:8080"            
    links:
      - flask1
      - flask2
      - flask3
    volumes:
      - /var/run:/var/run:ro
      - /sys:/sys:ro
      - /var/lib/docker/:/var/lib/docker:ro
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - /etc/machine-id:/etc/machine-id:ro
      - /var/lib/dbus/machine-id:/var/lib/dbus/machine-id:ro

How to solve the remaining warnings on macOS?

W0325 14:31:52.333856       1 machine_libipmctl.go:62] There are no NVM devices!
W0325 14:31:52.335974       1 sysinfo.go:203] Nodes topology is not available, providing CPU topology
W0325 14:31:52.438604       1 manager.go:288] Could not configure a source for OOM detection, disabling OOM events: open /dev/kmsg: no such file or directory
time="2021-03-25T14:31:52Z" level=warning msg="falling back to securejoin: openat2 /sys/fs/cgroup: operation not permitted"
W0325 14:31:52.492071       1 container.go:586] Failed to update stats for container "/kubepods": /sys/fs/cgroup/cpuset/kubepods/cpuset.cpus found to be empty, continuing to push stats
W0325 14:32:07.011444       1 prometheus.go:1856] Couldn't get containers: partial failures: ["/kubepods": containerDataToContainerInfo: unable to find data in memory cache]
W0325 14:32:21.991083       1 prometheus.go:1856] Couldn't get containers: partial failures: ["/kubepods": containerDataToContainerInfo: unable to find data in memory cache]
W0325 14:32:37.000379       1 prometheus.go:1856] Couldn't get containers: partial failures: ["/kubepods": containerDataToContainerInfo: unable to find data in memory cache]

docker version

Client: Docker Engine - Community
 Cloud integration: 1.0.9
 Version:           20.10.5
 API version:       1.41
 Go version:        go1.13.15
 Git commit:        55c4c88
 Built:             Tue Mar  2 20:13:00 2021
 OS/Arch:           darwin/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.5
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.13.15
  Git commit:       363e9a8
  Built:            Tue Mar  2 20:15:47 2021
  OS/Arch:          linux/amd64
  Experimental:     true
 containerd:
  Version:          1.4.3
  GitCommit:        269548fa27e0089a8b8278fc4fc781d7f65a939b
 runc:
  Version:          1.0.0-rc92
  GitCommit:        ff819c7e9184c13b7c2607fe6c30ae19403a7aff
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

docker info

Client:
 Context:    default
 Debug Mode: false
 Plugins:
  app: Docker App (Docker Inc., v0.9.1-beta3)
  buildx: Build with BuildKit (Docker Inc., v0.5.1-docker)
  scan: Docker Scan (Docker Inc., v0.5.0)

Server:
 Containers: 8
  Running: 8
  Paused: 0
  Stopped: 0
 Images: 170
 Server Version: 20.10.5
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 269548fa27e0089a8b8278fc4fc781d7f65a939b
 runc version: ff819c7e9184c13b7c2607fe6c30ae19403a7aff
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 4.19.121-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: x86_64
 CPUs: 4
 Total Memory: 5.071GiB
 Name: docker-desktop
 ID: LJD2:DQZQ:IRHN:3ZPK:LNVR:3FST:GXXZ:VKPZ:USX5:AEA4:CPKJ:RG7Q
 Docker Root Dir: /var/lib/docker
 Debug Mode: true
  File Descriptors: 108
  Goroutines: 100
  System Time: 2021-03-25T14:05:12.800558282Z
  EventsListeners: 5
 HTTP Proxy: gateway.docker.internal:3128
 HTTPS Proxy: gateway.docker.internal:3129
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: true
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false
@MarianaHiTi
Copy link

Did you solve it? :)

@dcs3spp
Copy link
Author

dcs3spp commented Apr 14, 2021

Hi @MarianaHiTi, Sadly not yet...still awaiting response.....

@philyuchkoff
Copy link

i'm don't solved this problem too, compelled revert to 0.38.8 :(

@MarianaHiTi
Copy link

Hi @MarianaHiTi, Sadly not yet...still awaiting response.....
I also continue with the problem :(

@MarianaHiTi
Copy link

i'm don't solved this problem too, compelled revert to 0.38.8 :(

please don't say me that ;(

@nicofr83
Copy link

Same problem here, with OSX 11.14 beta, current version of docker desktop for Mac

(containers do not pop up in the UI)

I tried with v0.39, and v0.38.8, same result

Docker output:

Creating cadvisor      ... done
Attaching to promtail, cadvisor, node-exporter, prometheus, climatodb, loki, pgadmin, grafana, django, jaeger
... Output from other containers removed...
cadvisor         | W0517 12:09:11.969978       1 sysinfo.go:203] Nodes topology is not available, providing CPU topology
cadvisor         | W0517 12:09:11.972793       1 sysfs.go:348] unable to read /sys/devices/system/cpu/cpu0/online: open /sys/devices/system/cpu/cpu0/online: no such file or directory
cadvisor         | W0517 12:09:11.974622       1 info.go:53] Couldn't collect info from any of the files in "/etc/machine-id,/var/lib/dbus/machine-id"
cadvisor         | W0517 12:09:12.036111       1 manager.go:288] Could not configure a source for OOM detection, disabling OOM events: open /dev/kmsg: no such file or directory
cadvisor         | E0517 12:09:12.046829       1 manager.go:1123] Failed to create existing container: /docker/1e468521518510555331a6a5e205fd01414de237e13fb46a23de6ed648f58bca: failed to identify the read-write layer ID for container "1e468521518510555331a6a5e205fd01414de237e13fb46a23de6ed648f58bca". - open /var/lib/docker/image/overlay2/layerdb/mounts/1e468521518510555331a6a5e205fd01414de237e13fb46a23de6ed648f58bca/mount-id: no such file or directory
cadvisor         | E0517 12:09:12.055347       1 manager.go:1123] Failed to create existing container: /docker/d2961c50c19e9276a928234b8a3a1fbcc8a836f20a3640c94f9d341bbd337166: failed to identify the read-write layer ID for container "d2961c50c19e9276a928234b8a3a1fbcc8a836f20a3640c94f9d341bbd337166". - open /var/lib/docker/image/overlay2/layerdb/mounts/d2961c50c19e9276a928234b8a3a1fbcc8a836f20a3640c94f9d341bbd337166/mount-id: no such file or directory
cadvisor         | E0517 12:09:12.714089       1 manager.go:1123] Failed to create existing container: /docker/34fefedc2ce2999aeb66e8aafe8b98415a1a790e24567568d8dc675aad60acca: failed to identify the read-write layer ID for container "34fefedc2ce2999aeb66e8aafe8b98415a1a790e24567568d8dc675aad60acca". - open /var/lib/docker/image/overlay2/layerdb/mounts/34fefedc2ce2999aeb66e8aafe8b98415a1a790e24567568d8dc675aad60acca/mount-id: no such file or directory
cadvisor         | E0517 12:09:12.743700       1 manager.go:1123] Failed to create existing container: /docker/1e468521518510555331a6a5e205fd01414de237e13fb46a23de6ed648f58bca: failed to identify the read-write layer ID for container "1e468521518510555331a6a5e205fd01414de237e13fb46a23de6ed648f58bca". - open /var/lib/docker/image/overlay2/layerdb/mounts/1e468521518510555331a6a5e205fd01414de237e13fb46a23de6ed648f58bca/mount-id: no such file or directory
cadvisor         | E0517 12:09:12.748076       1 manager.go:1123] Failed to create existing container: /docker/d2961c50c19e9276a928234b8a3a1fbcc8a836f20a3640c94f9d341bbd337166: failed to identify the read-write layer ID for container "d2961c50c19e9276a928234b8a3a1fbcc8a836f20a3640c94f9d341bbd337166". - open /var/lib/docker/image/overlay2/layerdb/mounts/d2961c50c19e9276a928234b8a3a1fbcc8a836f20a3640c94f9d341bbd337166/mount-id: no such file or directory
cadvisor         | E0517 12:09:12.751387       1 manager.go:1123] Failed to create existing container: /docker/34fefedc2ce2999aeb66e8aafe8b98415a1a790e24567568d8dc675aad60acca: failed to identify the read-write layer ID for container "34fefedc2ce2999aeb66e8aafe8b98415a1a790e24567568d8dc675aad60acca". - open /var/lib/docker/image/overlay2/layerdb/mounts/34fefedc2ce2999aeb66e8aafe8b98415a1a790e24567568d8dc675aad60acca/mount-id: no such file or directory
cadvisor         | W0517 12:09:12.754633       1 manager.go:1176] Failed to process watch event {EventType:0 Name:/docker/34fefedc2ce2999aeb66e8aafe8b98415a1a790e24567568d8dc675aad60acca WatchSource:0}: failed to identify the read-write layer ID for container "34fefedc2ce2999aeb66e8aafe8b98415a1a790e24567568d8dc675aad60acca". - open /var/lib/docker/image/overlay2/layerdb/mounts/34fefedc2ce2999aeb66e8aafe8b98415a1a790e24567568d8dc675aad60acca/mount-id: no such file or directory
... other messages 'Failed to process watch event' removed

I tried with different docker-compose file, here is the result with the same definition as dcs3spp gave in the first post

Docker info

Client:
 Context:    default
 Debug Mode: false
 Plugins:
  app: Docker App (Docker Inc., v0.9.1-beta3)
  buildx: Build with BuildKit (Docker Inc., v0.5.1-docker)
  compose: Docker Compose (Docker Inc., 2.0.0-beta.1)
  scan: Docker Scan (Docker Inc., v0.8.0)

Server:
 Containers: 10
  Running: 10
  Paused: 0
  Stopped: 0
 Images: 11
 Server Version: 20.10.6
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 05f951a3781f4f2c1911b05e61c160e9c30eaa8e
 runc version: 12644e614e25b05da6fd08a38ffa0cfe1903fdec
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 5.10.25-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: x86_64
 CPUs: 8
 Total Memory: 1.94GiB
 Name: docker-desktop
 ID: KXLL:7JMJ:X3ZO:4H3J:PXFD:Y4LP:NIFT:W3OR:RRVK:PW3F:LV4B:KZOX
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Docker version

 Client:
 Cloud integration: 1.0.14
 Version:           20.10.6
 API version:       1.41
 Go version:        go1.16.3
 Git commit:        370c289
 Built:             Fri Apr  9 22:46:57 2021
 OS/Arch:           darwin/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.6
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.13.15
  Git commit:       8728dd2
  Built:            Fri Apr  9 22:44:56 2021
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.4.4
  GitCommit:        05f951a3781f4f2c1911b05e61c160e9c30eaa8e
 runc:
  Version:          1.0.0-rc93
  GitCommit:        12644e614e25b05da6fd08a38ffa0cfe1903fdec
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

@genkobar
Copy link 

cadvisor:
    image: gcr.io/cadvisor/cadvisor:v0.39.0
    container_name: cadvisor   
    privileged: true            <--------------- Add this
    ports:
      - "8081:8080"            
    links:
      - flask1
      - flask2
      - flask3
    volumes:
      - /var/run:/var/run:ro
      - /sys:/sys:ro
      - /var/lib/docker/:/var/lib/docker:ro
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - /etc/machine-id:/etc/machine-id:ro
      - /var/lib/dbus/machine-id:/var/lib/dbus/machine-id:ro

Have you tried running cadvisor with privileges? I don't see the NVM error you ran into, although I am having other issues with cadvisor on Docker for Mac.

@nightah nightah mentioned this issue Dec 2, 2021
Closed
@clayroach
Copy link

Appears to be resolved in my case where I was running cadvisor, prometheus, grafana, along with my other services using the same docker-compose.yaml by moving these to separate networks...

Docker for Mac running on Monterey

  redis:
    image: redis:latest
    container_name: redis
    networks:
      - myservices-net
    ports:
    - 6379:6379

  cadvisor:
    image: gcr.io/cadvisor/cadvisor:v0.45.0
    container_name: cadvisor
    privileged: true
    devices:
      - /dev/kmsg:/dev/kmsg
    volumes:
      - /var/run:/var/run:ro
      - /sys:/sys:ro
      - /var/lib/docker/:/var/lib/docker:ro
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - /etc/machine-id:/etc/machine-id:ro
      - /var/lib/dbus/machine-id:/var/lib/dbus/machine-id:ro
    restart: unless-stopped
    expose:
      - 8080
    networks:
      - mon-net
    labels:
      org.label-schema.group: "monitoring"

networks:
  myservices-net:
    driver: bridge
  mon-net:
    driver: bridge

@frapsky frapsky mentioned this issue Feb 9, 2023
Open
vasconsaurus added a commit to vasconsaurus/ada-observability that referenced this issue Feb 15, 2023
@vasconsaurus
added cadvisor
9cd5b64 
Cadvisors suggested setup didn't work. I followed this suggestion:
google/cadvisor#2838 (comment)
vasconsaurus added a commit to vasconsaurus/ada-observability that referenced this issue Feb 15, 2023
@vasconsaurus
added cadvisor
7a2081b 
Cadvisors suggested setup didn't work. I followed this suggestion:
google/cadvisor#2838 (comment)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@nicofr83 @clayroach @genkobar @philyuchkoff @dcs3spp @MarianaHiTi