ENV for Remote datalogger is not used

[Ankh-]

Support guidelines

• I've read the support guidelines

I've found an issue and checked that ...

• ... the documentation does not mention anything about my problem
• ... there are no open or closed issues that are related to my problem

Description

Got a new docker installation running with the following snippet:

      - DSMRREADER_REMOTE_DATALOGGER_MODE=standalone
      - DSMRREADER_REMOTE_DATALOGGER_INPUT_METHOD=ipv4
      - DSMRREADER_REMOTE_DATALOGGER_NETWORK_HOST=192.168.30.10
      - DSMRREADER_REMOTE_DATALOGGER_NETWORK_PORT=23

I've got a P1 E20 ethernet (Zuidwijk) running nicely. When checking telnet for data, there is actual data. So something is wrong regarding the docker compose ENV.

Expected behaviour

Actual data!

Actual behaviour

Errors :(

dsmr | 2022-12-07 20:38:06,236 ERROR mixins run_once 107 | dsmr_datalogger.management.commands.dsmr_datalogger: [!] Exception raised. Traceback (most recent call last):
dsmr | File "/usr/local/lib/python3.11/site-packages/serial/serialposix.py", line 322, in open
dsmr | self.fd = os.open(self.portstr, os.O_RDWR | os.O_NOCTTY | os.O_NONBLOCK)
dsmr | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
dsmr | FileNotFoundError: [Errno 2] No such file or directory: '/dev/ttyUSB0'
dsmr |
dsmr | During handling of the above exception, another exception occurred:
dsmr |
dsmr | Traceback (most recent call last):
dsmr | File "/app/dsmr_datalogger/scripts/dsmr_datalogger_api_client.py", line 39, in read_telegram
dsmr | serial_handle = serial.serial_for_url(
dsmr | ^^^^^^^^^^^^^^^^^^^^^^
dsmr | File "/usr/local/lib/python3.11/site-packages/serial/init.py", line 90, in serial_for_url
dsmr | instance.open()
dsmr | File "/usr/local/lib/python3.11/site-packages/serial/serialposix.py", line 325, in open
dsmr | raise SerialException(msg.errno, "could not open port {}: {}".format(self._port, msg))
dsmr | serial.serialutil.SerialException: [Errno 2] could not open port /dev/ttyUSB0: [Errno 2] No such file or directory: '/dev/ttyUSB0'
dsmr |
dsmr | The above exception was the direct cause of the following exception:
dsmr |
dsmr | Traceback (most recent call last):
dsmr | File "/app/dsmr_backend/mixins.py", line 96, in run_once
dsmr | self.run(data=self.data, **options)
dsmr | File "/app/dsmr_datalogger/management/commands/dsmr_datalogger.py", line 29, in run
dsmr | telegram = next(self.telegram_generator)
dsmr | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
dsmr | File "/app/dsmr_datalogger/scripts/dsmr_datalogger_api_client.py", line 43, in read_telegram
dsmr | raise RuntimeError("Failed to connect: {}", error) from error
dsmr | RuntimeError: ('Failed to connect: {}', SerialException(2, "could not open port /dev/ttyUSB0: [Errno 2] No such file or directory: '/dev/ttyUSB0'"))
dsmr |

Steps to reproduce

basic yaml with added remote datalogger

Docker info

root@box:/docker/dsmr# docker info
Client:
 Context:    default
 Debug Mode: false
 Plugins:
  app: Docker App (Docker Inc., v0.9.1-beta3)
  buildx: Docker Buildx (Docker Inc., v0.9.1-docker)
  compose: Docker Compose (Docker Inc., v2.13.0)
  scan: Docker Scan (Docker Inc., v0.21.0)

Server:
 Containers: 2
  Running: 0
  Paused: 0
  Stopped: 2
 Images: 3
 Server Version: 20.10.21
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 770bd0108c32f3fb5c73ae1264f7e503fe7b2661
 runc version: v1.1.4-0-g5fd4c4d
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: default
  cgroupns
 Kernel Version: 5.10.0-19-amd64
 Operating System: Debian GNU/Linux 11 (bullseye)
 OSType: linux
 Architecture: x86_64
 CPUs: 6
 Total Memory: 15.44GiB
 Name: box
 ID: BISR:P377:UHHH:HYO6:4JHT:XYT2:4JGN:XXEQ:OAKF:7WUK:VJ3T:XZAK
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Version

see docker info

Docker compose

root@box:/docker/dsmr# docker compose config
name: dsmr
services:
  dsmr:
    cap_add:
    - NET_ADMIN
    container_name: dsmr
    depends_on:
      dsmrdb:
        condition: service_healthy
    environment:
      DGID: "1000"
      DJANGO_TIME_ZONE: Europe/Amsterdam
      DSMRREADER_REMOTE_DATALOGGER_INPUT_METHOD: ipv4
      DSMRREADER_REMOTE_DATALOGGER_MODE: standalone
      DSMRREADER_REMOTE_DATALOGGER_NETWORK_HOST: 192.168.30.10
      DSMRREADER_REMOTE_DATALOGGER_NETWORK_PORT: "23"
      DUID: "1000"
      TZ: Europe/Amsterdam
      VIRTUAL_HOST: localhost
    healthcheck:
      test:
      - CMD
      - curl
      - -Lsf
      - http://127.0.0.1/about
      - -o
      - /dev/null
      - -w
      - HTTP_%{http_code}
      timeout: 5s
      interval: 10s
      retries: 10
    image: xirixiz/dsmr-reader-docker:latest
    links:
    - dsmrdb
    networks:
      default: null
    ports:
    - mode: ingress
      target: 80
      published: "7777"
      protocol: tcp
    - mode: ingress
      target: 443
      published: "7779"
      protocol: tcp
    restart: always
    volumes:
    - type: bind
      source: /etc/localtime
      target: /etc/localtime
      read_only: true
      bind:
        create_host_path: true
    - type: bind
      source: /docker/dsmr/dsmr_backups
      target: /app/backups
      bind:
        create_host_path: true
  dsmrdb:
    container_name: dsmrdb
    environment:
      DGID: "1000"
      DUID: "1000"
      PG_TZ: Europe/Amsterdam
      POSTGRES_DB: dsmrreader
      POSTGRES_PASSWORD: dsmrreader
      POSTGRES_USER: dsmrreader
      TZ: Europe/Amsterdam
    healthcheck:
      test:
      - CMD-SHELL
      - pg_isready -U dsmrreader
      timeout: 5s
      interval: 10s
      retries: 10
    image: postgres:14-alpine
    networks:
      default: null
    restart: always
    volumes:
    - type: bind
      source: /docker/dsmr/dsmrdb
      target: /var/lib/postgresql/data
      bind:
        create_host_path: true
networks:
  default:
    name: dsmr_default
root@box:/docker/dsmr#

Container logs

see errors

Additional info

No response

[xirixiz]

Hi, it seems DSMRREADER_REMOTE_DATALOGGER_MODE is not set correctly. This should be set to "receiver"

https://github.com/xirixiz/dsmr-reader-docker#dsmr-datalogger-related

Can you try and validate this? Thanks!

[Ankh-]

Hi.
Changed to receiver and removed all data (clean run)

root@box:/docker/dsmr# cat docker-compose.yaml | grep DATALOGGER
- DSMRREADER_REMOTE_DATALOGGER_MODE=receiver
- DSMRREADER_REMOTE_DATALOGGER_INPUT_METHOD=ipv4
- DSMRREADER_REMOTE_DATALOGGER_NETWORK_HOST=192.168.30.10
- DSMRREADER_REMOTE_DATALOGGER_NETWORK_PORT=23
root@box:/docker/dsmr# ls -al
total 12
drwxr-xr-x 2 paul paul 4096 Dec 7 21:03 .
drwxr-xr-x 5 root root 4096 Dec 7 15:53 ..
-rw-r--r-- 1 paul paul 1911 Dec 7 21:03 docker-compose.yaml
root@box:/docker/dsmr#

The errors are gone. But when checking the dsmr page it still gives no data. Or do I need to go the admin level and setup the datalogger settings there? I thought it would be forced, so after a docker reboot it isn't back to defaults.

[xirixiz]

Hi, maybe @zuidwijk can help himself. He's been active here as well. Maybe it requires a different set of configuration. I`m not sure.

[Ankh-]

Dont think the issue is with the P1 ethernet reader from zuidwijk.

Just switched back to standalone and manually setup the settings: http://x.x.x.x:7777/admin/dsmr_datalogger/dataloggersettings/

And now it works. Yet when stopping the compose and starting it will wipe this setting.
So something is going wrong with the environment in the docker-compose.yml file.

This is the current snippet:

dsmr:
image: xirixiz/dsmr-reader-docker:latest
depends_on:
dsmrdb:
condition: service_healthy
container_name: dsmr
links:
- dsmrdb
cap_add:
- NET_ADMIN
restart: always
volumes:
- /etc/localtime:/etc/localtime:ro
- /home/dsmr/dsmr_backups:/app/backups
environment:
- TZ=Europe/Amsterdam
- DJANGO_TIME_ZONE=Europe/Amsterdam
- VIRTUAL_HOST=localhost
- DSMRREADER_REMOTE_DATALOGGER_MODE=standalone
- DSMRREADER_REMOTE_DATALOGGER_INPUT_METHOD=ipv4
- DSMRREADER_REMOTE_DATALOGGER_NETWORK_HOST=192.168.30.10
- DSMRREADER_REMOTE_DATALOGGER_NETWORK_PORT=23
- DSMRREADER_REMOTE_DATALOGGER_DEBUG_LOGGING=true
- DUID=1000
- DGID=1000
ports:
- 7777:80
- 7779:443
# healthcheck:
# disable: true
healthcheck:
test:
[
"CMD",
"curl",
"-Lsf",
"http://127.0.0.1/about",
"-o",
"/dev/null",
"-w",
"HTTP_%{http_code}"
]
interval: 10s
timeout: 5s
retries: 10

I've setup the debug log for the datalogger aswell and it doesnt show any debug log....

[xirixiz]

What I meant is that Zuidwijk knows exactly how to set it up properly. I'm sure Zuidwijks solution is working fine. What is it you change after starting the container?

[Ankh-]

Ik switch even naar nederlands ;-)

De P1 ethernet verzorgt alleen maar een telegram/console. Dus de DSMR Reader moet daar z'n info vandaan halen. Ik heb het idee als ik de remote_datalogger_mode als receiver in stel, deze niet meer externe informatie gaat opvragen. En enkel als ontvanger (via de API) z'n gegevens kan ontvangen.

Momenteel heb ik de remote_datalogger_mode als standalone ingesteld en middels de admin pagina wel de P1 Ethernet reader ingesteld. En nu werkt het allemaal.

[zuidwijk]

Het ziet er allemaal ingewikkelder uit dan dat hoe ik het draai. Dit is mijn docker compose:

version: '3.8'

services:
  dsmrdb:
    image: postgres:12.4-alpine
    container_name: dsmrdb
    restart: always
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - ./dsmrdb:/var/lib/postgresql/data
    environment:
      - TZ=Europe/Amsterdam
      - PG_TZ=Europe/Amsterdam
      - POSTGRES_USER=dsmrreader
      - POSTGRES_PASSWORD=dsmrreader
      - POSTGRES_DB=dsmrreader
  dsmr:
    image: xirixiz/dsmr-reader-docker:latest
    container_name: dsmr
    depends_on:
      - dsmrdb
    cap_add:
      - NET_ADMIN    
    links:
      - dsmrdb
    restart: always
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - ./dsmr_backups:/dsmr/backups
    environment:
      - TZ=Europe/Amsterdam
      - DJANGO_TIME_ZONE=Europe/Amsterdam
      - VIRTUAL_HOST=localhost
      - DSMRREADER_ADMIN_USER=****
      - DSMRREADER_ADMIN_PASSWORD=****
    ports:
      - 7777:80
      - 7779:443

[zuidwijk]

Ik switch even naar nederlands ;-)

De P1 ethernet verzorgt alleen maar een telegram/console. Dus de DSMR Reader moet daar z'n info vandaan halen. Ik heb het idee als ik de remote_datalogger_mode als receiver in stel, deze niet meer externe informatie gaat opvragen. En enkel als ontvanger (via de API) z'n gegevens kan ontvangen.

Momenteel heb ik de remote_datalogger_mode als standalone ingesteld en middels de admin pagina wel de P1 Ethernet reader ingesteld. En nu werkt het allemaal.

Ik stel het ip adres van de ethernet module in op de config pagina van dsmr-reader.

[xirixiz]

Bedankt voor het delen 👍🏻. Ik zal de check aanpassen voor standalone. Ik merk dat het vaak verwarrend is wanneer /dev/ttyUSB niet gevonden wordt.

[xirixiz]

Readme updated.