Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SONOFF zigbee 3.0 usb dongle plus Keeps crashing #12566

Closed
Veldkornet opened this issue May 22, 2022 · 58 comments
Closed

SONOFF zigbee 3.0 usb dongle plus Keeps crashing #12566

Veldkornet opened this issue May 22, 2022 · 58 comments
Labels
problem Something isn't working stale Stale issues

Comments

@Veldkornet
Copy link

What happened?

As a continuation of #11216, every so often all devices become unavailable and I get messages like the following:

Debug Received MQTT message on 'zigbee2mqtt/bridge/request/permit_join' with data '{"device":null,"time":254,"transaction":"wjftl-3","value":true}'
Info Zigbee: allowing new devices to join.
Error Request 'zigbee2mqtt/bridge/request/permit_join' failed with error: 'SRSP - ZDO - mgmtPermitJoinReq after 6000ms'
Debug Error: SRSP - ZDO - mgmtPermitJoinReq after 6000ms at Timeout._onTimeout (/app/node_modules/zigbee-herdsman/src/utils/waitress.ts:64:35) at listOnTimeout (internal/timers.js:557:17) at processTimers (internal/timers.js:500:7)
Info MQTT publish: topic 'zigbee2mqtt/bridge/response/permit_join', payload '{"data":{},"error":"SRSP - ZDO - mgmtPermitJoinReq after 6000ms","status":"error","transaction":"wjftl-3"}'

I currently run both MQTT and Z2M on HomeAssistant OS on a RaspberryPi 4 with a ARGON one M.2. The Sonoff adapter is plugged into a Nedis Powered USB HUB, which is plugged into one of the usb 2.0 ports of the RPi.
The only items plugged into the hub are the SONOFF and an Aeotec ZStick.

This issue does not occur with the ConBee II.

I’ve been through the troubleshooting topic listed here; and I’ve added usbcore.autosuspend=-1, but this hasn’t seem to make any difference.

It can crash every few hours or every 1-2 days, I don’t find a clear pattern.

What did you expect to happen?

Everything should stay running and the coordinator should not crash.

How to reproduce it (minimal and precise)

unclear, it just crashes on it’s own.

Zigbee2MQTT version

1.25.1

Adapter firmware version

20220507

Adapter

SONOFF Zigbee USB Dongle Plus (zStack3x0)

Debug log

No response

@Veldkornet Veldkornet added the problem Something isn't working label May 22, 2022
@Koenkk
Copy link
Owner

Koenkk commented May 22, 2022

Can you provide the zigbee herdsman log starting from the point where you start z2m until it fails?

See https://www.zigbee2mqtt.io/guide/usage/debug.html on how to enable the herdsman debug logging. Note that this is only logged to STDOUT and not to log files.

@Veldkornet
Copy link
Author

@Koenkk See log here. Looks to have started at 2022-05-25T03:48:50.628Z. I have about 2 days worth of that debug log before the issue arose, but let me know if you need more info.

@Koenkk
Copy link
Owner

Koenkk commented May 25, 2022

  • Could you try disabling the availability feature? I don't see any indication that something goes wrong in the coordinator firmware but this should reduce the stress on it.
  • Does anything appear in the dmesg logging when the crash happens?

@Veldkornet
Copy link
Author

I've disabled Availability before, but it didn't make a difference. Only that there is a less visual way for me to see that there is an issue.

A bit difficult to look back in dmesg...

However, I bought a new SONOFF and have moved everything over to it instead. Will see now if it works any better or has the same issues.

@dajomu1
Copy link

dajomu1 commented May 29, 2022

I had the same problem with my Electrolama stick. Check if you have the Zigbee Home Automation integration installed. When I disabled the integration everything worked as it should.

@Veldkornet
Copy link
Author

Nope, same thing happens with the new dongle. It was going strong for the first couple of days, but then last night... ☠️

Also, no I do not have ZHA.

Again, the issue does not occur when I use the ConBee II.

@Bascht74
Copy link

Bascht74 commented Jun 24, 2022

Same here. And some Philips motion sensors get partley disconnected (still shows connection, but it sends no updated sensor data). After hard restart (power off/on) everything is fine until it happens again after some days. Pairing of new devices is not possible as well in that state...
I had ConBee II before and it does not happen there...

@Veldkornet
Copy link
Author

At the moment for me, it happens once to twice per day. Most of the time, I just restart the add-on and it’s fine.
Now and then I also need to re-plug the stick.

It’s easy to spot because all of the devices become unavailable in HomeAssistant

@github-actions
Copy link
Contributor

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 7 days

@github-actions github-actions bot added the stale Stale issues label Jul 25, 2022
@Veldkornet
Copy link
Author

Issue still occurs. Still haven't really discovered why...

@github-actions github-actions bot removed the stale Stale issues label Jul 26, 2022
@github-actions
Copy link
Contributor

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 7 days

@github-actions github-actions bot added the stale Stale issues label Aug 26, 2022
@Veldkornet
Copy link
Author

FYI, I've updated the firmware to 20220726 and it seems to work better, but it still happens.

Ie, longer periods between when it breaks. Now it's maybe once a week.

@github-actions github-actions bot removed the stale Stale issues label Aug 27, 2022
@Minoux78
Copy link

Minoux78 commented Sep 5, 2022

Hello,
@Veldkornet
Where do you find the 20220726 firmware?
For me the last update for SONOFF Zigbee USB Dongle Plus (zStack3x0) is the 20220219.

Thanks for your feedback

@Veldkornet
Copy link
Author

Hello,

@Veldkornet

Where do you find the 20220726 firmware?

For me the last update for SONOFF Zigbee USB Dongle Plus (zStack3x0) is the 20220219.

Thanks for your feedback

Here -> https://github.com/Koenkk/Z-Stack-firmware/tree/develop/coordinator/Z-Stack_3.x.0/bin

@popy2k14
Copy link

@Veldkornet Exact same issue here!
The cherry on top, i have bought the sonoff to replace the conbee II because sometimes deconz addon has stabilitty issues :-)

I am also running; 20220219
Did you got the stick stable?

@popy2k14
Copy link

@Koenkk is this a known issue and solved with an newer firmware?

@Koenkk
Copy link
Owner

Koenkk commented Sep 15, 2022

@popy2k14 can you provide the herdsman debug logging starting from the moment you start z2m until the crash?

See https://www.zigbee2mqtt.io/guide/usage/debug.html on how to enable the herdsman debug logging. Note that this is only logged to STDOUT and not to log files.

@popy2k14
Copy link

@Koenkk i have enabled the herdsman logging on my hass.io supervised addon and tried to gather the logs as described in the link. There are now herdsman entries in it.

When the issue occured i will grab them an come back here.

Did you found an issue in @Veldkornet log above?

@Veldkornet is the new beta firmware 20220726 stable for you?

@Veldkornet
Copy link
Author

The new firmware works better, but it still happens unfortunately.

@popy2k14
Copy link

@Veldkornet thx for the info.

@Koenkk it happened again, here is the log file: https://drive.google.com/file/d/1NjKBKp9Qz-TPUm7kYP6pZS8MHRjekf8a/view?usp=sharing

@Koenkk
Copy link
Owner

Koenkk commented Sep 16, 2022

@popy2k14 the log doesn't stop with a crash, just with:

Zigbee2MQTT:error 2022-09-16 06:24:48: Failed to execute LQI for 'Balkon indirekt'
Zigbee2MQTT:info  2022-09-16 06:24:48: Network scan finished
Zigbee2MQTT:info  2022-09-16 06:24:48: MQTT publish: topic 'zigbee2mqtt/bridge/response/networkmap', payload '{"data":{"routes":false,"type":"raw","value":{"links":[],"nodes":[{"definition":null,"failed":["lqi"],"friendlyName":"Coordinator","ieeeAddr":"0x00124b00257cb272","lastSeen":null,"networkAddress":0,"type":"Coordinator"},{"definition":{"description":"Hue dimmer switch","model":"324131092621","supports":"battery, action, action_duration, linkquality","vendor":"Philips"},"friendlyName":"Dimmschalter Balkon","ieeeAddr":"0x0017880103e5e9a3","lastSeen":1663270959464,"manufacturerName":"Philips","modelID":"RWL021","networkAddress":24406,"type":"EndDevice"},{"definition":{"description":"Hue White A60 Single bulb E27/B22","model":"8718696449691","supports":"light (state, brightness), effect, linkquality","vendor":"Philips"},"failed":["lqi"],"friendlyName":"Balkon indirekt","ieeeAddr":"0x0017880103169f9c","lastSeen":1663267144444,"manufacturerName":"Signify Netherlands B.V.","modelID":"LWB010","networkAddress":43725,"type":"Router"},{"definition":{"description":"Hue motion outdoor sensor","model":"9290019758","supports":"temperature, occupancy, battery, illuminance_lux, illuminance, motion_sensitivity, led_indication, occupancy_timeout, linkquality","vendor":"Philips"},"friendlyName":"Bewegungsmelder Balkon","ieeeAddr":"0x00178801064432a9","lastSeen":1663272061394,"manufacturerName":"Philips","modelID":"SML002","networkAddress":49383,"type":"EndDevice"}]}},"status":"ok","transaction":"hhopd-1"}'
  • I expect that the real crash is logged after this, can you provide the log of this?
  • I see you are doing a network scan, does the crash also happen when not doing network scans?

@Veldkornet do you by any chance of have SONOFF sensors on your network (I mean battery powered sensors; not the coordinator itself)

@inertia666 I need herdsman debug logs of the crash, otherwise I cannot help.

@popy2k14
Copy link

@Koenkk
That's the whole log file. There isn't anything else in the log!
Network scans working over the day without an problem.
I am using this to see the LQI's and when there are no lines between the devices Z2M crashed.
In the morning i have done an scan to see if it crashed.

When it help's i can change my "test" and do not do an network scan the comming days and just try to see if the devices are offline.

Please let me know how i should proceed with my test.

@inertia666
Copy link

inertia666 commented Sep 19, 2022

I'm happy to report that my USB stick is working perfectly well now. I had misunderstood the instructions and exposed the device to two different Docker containers (Both Home Assistant and Zigbee2MQTT). Presumably, this was causing the crash. Sorry for reporting that I was experiencing crashing.

@Koenkk
Copy link
Owner

Koenkk commented Sep 21, 2022

@popy2k14 does z2m still answer the network map scan? If yes then z2m it self has no crashed. I do not see the finish of the network scan in your logs.

@popy2k14
Copy link

@Koenkk yes, it returns an error in red.

@srvoets
Copy link

srvoets commented Nov 22, 2022

Hey guys,

need to join in here. Same Stick, same problem (at least i think): usb-ITead_Sonoff_Zigbee_3.0_USB_Dongle_Plus_089e9136cd3aec11a38ba4957a0af07f-if00-port0

Z2M won't start. Full Log here:

me@rasPi3B $ ./start_z2m_w-debug.sh 


To detach the container press ctrl-p ctrl-q


c02b55caa1a31be6284961409f4347c9699fc3b43e2ed0b1b88db55a8f7a0003
Zigbee2MQTT:debug 2022-10-22 14:52:10: Loaded state from file /app/data/state.json
Zigbee2MQTT:info  2022-10-22 14:52:10: Logging to console and directory: '/app/data/log/2022-10-22.14-52-10' filename: log.txt
Zigbee2MQTT:debug 2022-10-22 14:52:10: Removing old log directory '/app/data/log/2022-10-22.14-47-04'
Zigbee2MQTT:info  2022-10-22 14:52:10: Starting Zigbee2MQTT version 1.28.0 (commit #03ba647)
Zigbee2MQTT:info  2022-10-22 14:52:10: Starting zigbee-herdsman (0.14.62)
Zigbee2MQTT:debug 2022-10-22 14:52:10: Using zigbee-herdsman with settings: '{"adapter":{"concurrent":null,"delay":null,"disableLED":false},"backupPath":"/app/data/coordinator_backup.json","databaseBackupPath":"/app/data/database.db.backup","databasePath":"/app/data/database.db","network":{"channelList":[11],"extendedPanID":[221,221,221,221,221,221,221,221],"networkKey":"HIDDEN","panID":6754},"serialPort":{"path":"/dev/serial/by-id/usb-ITead_Sonoff_Zigbee_3.0_USB_Dongle_Plus_089e9136cd3aec11a38ba4957a0af07f-if00-port0","rtscts":false}}'
  zigbee-herdsman:controller:log Starting with options '{"network":{"networkKeyDistribute":false,"networkKey":[1,3,5,7,9,11,13,15,0,2,4,6,8,10,12,13],"panID":6754,"extendedPanID":[221,221,221,221,221,221,221,221],"channelList":[11]},"serialPort":{"rtscts":false,"path":"/dev/serial/by-id/usb-ITead_Sonoff_Zigbee_3.0_USB_Dongle_Plus_089e9136cd3aec11a38ba4957a0af07f-if00-port0"},"databasePath":"/app/data/database.db","databaseBackupPath":"/app/data/database.db.backup","backupPath":"/app/data/coordinator_backup.json","adapter":{"disableLED":false,"concurrent":null,"delay":null}}' +0ms
  zigbee-herdsman:adapter:zStack:znp:log Opening SerialPort with /dev/serial/by-id/usb-ITead_Sonoff_Zigbee_3.0_USB_Dongle_Plus_089e9136cd3aec11a38ba4957a0af07f-if00-port0 and {"baudRate":115200,"rtscts":false,"autoOpen":false} +0ms
  zigbee-herdsman:adapter:zStack:znp:log Serialport opened +24ms
  zigbee-herdsman:adapter:zStack:znp:SREQ --> SYS - ping - {"capabilities":1} +0ms
  zigbee-herdsman:adapter:zStack:unpi:writer --> frame [254,0,33,1,32] +0ms
  zigbee-herdsman:adapter:zStack:znp:log Writing CC2530/CC2531 skip bootloader payload +274ms
  zigbee-herdsman:adapter:zStack:unpi:writer --> buffer [239] +261ms
  zigbee-herdsman:adapter:zStack:znp:SREQ --> SYS - ping - {"capabilities":1} +1s
  zigbee-herdsman:adapter:zStack:unpi:writer --> frame [254,0,33,1,32] +1s
  zigbee-herdsman:adapter:zStack:znp:log Skip bootloader for CC2652/CC1352 +1s
  zigbee-herdsman:adapter:zStack:znp:SREQ --> SYS - ping - {"capabilities":1} +819ms
  zigbee-herdsman:adapter:zStack:unpi:writer --> frame [254,0,33,1,32] +819ms
  zigbee-herdsman:adapter:zStack:znp:SREQ --> SYS - ping - {"capabilities":1} +6s
  zigbee-herdsman:adapter:zStack:unpi:writer --> frame [254,0,33,1,32] +6s
  zigbee-herdsman:adapter:zStack:znp:SREQ --> SYS - ping - {"capabilities":1} +6s
  zigbee-herdsman:adapter:zStack:unpi:writer --> frame [254,0,33,1,32] +6s
Zigbee2MQTT:error 2022-10-22 14:52:31: Error while starting zigbee-herdsman
Zigbee2MQTT:error 2022-10-22 14:52:31: Failed to start zigbee
Zigbee2MQTT:error 2022-10-22 14:52:31: Check https://www.zigbee2mqtt.io/guide/installation/20_zigbee2mqtt-fails-to-start.html for possible solutions
Zigbee2MQTT:error 2022-10-22 14:52:31: Exiting...
Zigbee2MQTT:error 2022-10-22 14:52:32: Error: Failed to connect to the adapter (Error: SRSP - SYS - ping after 6000ms)
    at ZStackAdapter.start (/app/node_modules/zigbee-herdsman/src/adapter/z-stack/adapter/zStackAdapter.ts:103:27)
    at Controller.start (/app/node_modules/zigbee-herdsman/src/controller/controller.ts:132:29)
    at Zigbee.start (/app/lib/zigbee.ts:58:27)
    at Controller.start (/app/lib/controller.ts:101:27)
    at start (/app/index.js:109:5)

What is interesting in my case, it has to be an issue with the RasPi3b, because same dongle with same container works like a charme on my LapTop. I copied config and databases to my laptop and pulled new z2m:latest. I then tried to start older versions of z2m without success on my rPi. I was thinking of permissions but sadly they're all the same. Same id, groups and so on. No other Container is using the adapter in first place. Nothing else running.

Any help VERY appreciated as I'm out of ideas where to look next.

add the following to your config

adapter: ezsp

^^ I saw someone post this, and it fixed 4 days of trying to make it work, but now Z2M crashes every 15 minutes.

@adfx
Copy link

adfx commented Nov 22, 2022

Thanks for your help. I'll try this later and come back then

@adfx
Copy link

adfx commented Dec 1, 2022

So now is later :)

Finally found another container interfering with z2m. In my case raspberrymatic (Homematic container'd) needs to be switched off before starting z2m. For any reason, it looks like Homematic is taking over the stick. So it's a matter of the starting order. I'll write another docker-compose.yml which will start both container in there respective order. Clear to me now why the container was working on my laptop. No other container there interfering with z2m.

@srvoets your idea with adapter: ezsp didn't help. If set, adapter will be accessible but z2m won't start. I removed the entry and now it works like a charme. Thanks for your help anyways.

@srvoets
Copy link

srvoets commented Dec 1, 2022

So now is later :)

Finally found another container interfering with z2m. In my case raspberrymatic (Homematic container'd) needs to be switched off before starting z2m. For any reason, it looks like Homematic is taking over the stick. So it's a matter of the starting order. I'll write another docker-compose.yml which will start both container in there respective order. Clear to me now why the container was working on my laptop. No other container there interfering with z2m.

@srvoets your idea with adapter: ezsp didn't help. If set, adapter will be accessible but z2m won't start. I removed the entry and now it works like a charme. Thanks for your help anyways.

Ahhh, I pulled my hair out for days seeing the same exact timeout error over and over again, and saw people mention make sure nothing else is running that could use stick, removed z-wave uninstalled it, had same issue, so I tried it in a docker container in unraid, same issue.

only thing that made it work was adapter: ezsp

So you are running this adapter without the above 'adapter' option in your serial config? Very interesting that yours works, and mine does not, is yours running stock firmware? I tried for several nights to flash it using the TI tool, python app, remove case holding buttons, never got it into bootloader mode to take the new firmware.

I really wish I did though because my zigbee network runs like dogshit now, I might go back to conbee until I can get this flashed and then set it up as a router.

My other consideration is migrating Z2m from from my unraid build inside an HA VM to a spare Pi that could be placed more centrally.

However, in the original location, conbee worked without issue, I blame this whacky stick.

I even unplugged my steelseries headphone\mouse wireless adapter, but no difference. At least it "runs"

@adfx
Copy link

adfx commented Dec 11, 2022

@srvoets FYI my docker-compose.yaml which is working:

version: "3.0" services: zigbee2mqtt: container_name: zig2mq restart: always image: koenkk/zigbee2mqtt volumes: - /home/eddy/docker/zigbee2mqtt/data:/app/data:rw - /run/udev:/run/udev:ro ports: - 8082:8080 environment: - TZ=Europe/Berlin devices: - /dev/serial/by-id/usb-ITead_Sonoff_Zigbee_3.0_USB_Dongle_Plus_089e9136cd3aec11a38ba4957a0af07f-if00-port0:/dev/ttyUSB0

and configuration.yaml:

homeassistant: false permit_join: true frontend: true mqtt: base_topic: zigbee2mqtt server: mqtt://xxx.xxx.xxx.xxx user: MYUSER password: MYPASSWD serial: port: /dev/ttyUSB0 advanced: log_level: debug devices: '0xGeneratedByZ2M: friendly_name: '0xGeneratedByZ2M'

And take care of the starting order. As said in my case Z2M needs to be started first. Then Stick is attached to z2m. Hope that helps.

Just had a look to the preview. Looks like format of .yaml's lost to copy paste. Sorry for that

@klaus1k
Copy link

klaus1k commented Dec 15, 2022

Hello, I also switched to Sonoff Dongle-P, which ran like a charm for a week. But it stopped working almost yesterday. now its crashing almost immediately after start. Curious thing ist - didnt change anything...

Some information (everything the same for working state and crashing state):

  • i followed all the instructions in Z2M FAQ if stick is crashing
  • I use it with a USB extension cable.
  • I have disabled access from homeassistant container to ttyUSB0
  • is use it with the newest dev firmware 20221102

dmesg log shows the following:

at start:
[ 492.124327] usb 1-1.1: new full-speed USB device number 3 using xhci_hcd
[ 492.233880] usb 1-1.1: New USB device found, idVendor=10c4, idProduct=ea60, bcdDevice= 1.00
[ 492.233884] usb 1-1.1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[ 492.233887] usb 1-1.1: Product: Sonoff Zigbee 3.0 USB Dongle Plus
[ 492.233888] usb 1-1.1: Manufacturer: Silicon Labs
[ 492.233890] usb 1-1.1: SerialNumber: 0001
[ 492.790586] usbcore: registered new interface driver usbserial_generic
[ 492.790602] usbserial: USB Serial support registered for generic
[ 492.794352] usbcore: registered new interface driver cp210x
[ 492.794377] usbserial: USB Serial support registered for cp210x
[ 492.794432] cp210x 1-1.1:1.0: cp210x converter detected
[ 492.799162] usb 1-1.1: cp210x converter now attached to ttyUSB0
...
[ 539.416965] cp210x ttyUSB0: usb_serial_generic_write_start - error submitting urb: -19
[ 539.418737] cp210x ttyUSB0: failed to get comm status: -19
[ 539.418750] cp210x ttyUSB0: failed set request 0x7 status: -19
[ 539.418763] cp210x ttyUSB0: failed set request 0x12 status: -19
[ 539.418767] cp210x ttyUSB0: failed set request 0x0 status: -19
...
then recurring:
[ 547.177746] cp210x ttyUSB0: failed set request 0x0 status: -19
[ 547.177753] cp210x ttyUSB0: cp210x_open - Unable to enable UART
...
[ 548.934547] cp210x ttyUSB0: failed set request 0x0 status: -19
[ 548.934557] cp210x ttyUSB0: cp210x_open - Unable to enable UART

Any ideas what i could try next? Am a little frustrated right now - migrated from deconz with conbeeII to Z2M with conbeeII (still some problems with lights not responding) and finally to sonoff Dongle-P. Was so happy that finally everything was working as it was supposed to. (including availability)

@Koenkk
Copy link
Owner

Koenkk commented Dec 15, 2022

Please try 20221214 from Koenkk/Z-Stack-firmware#383 (comment) . 20221102 is known to be unstable (sometimes)

@klaus1k
Copy link

klaus1k commented Dec 15, 2022

Thanks for the reply! Installed it right away, but sadly the errors didn't go away. I have the feeling that the problem may be at driver/linux usb side. Will look into it, i really hope to find a solution!

@adfx
Copy link

adfx commented Dec 17, 2022

@klaus1k
Indeed this looks like a driver issue. Have a try starting z2m with root privileges. If that helps you'll need to dig deeper in how to start containers as non-root.

@srvoets
I finally found the root cause. My z2m instance restarted right after homematic instance was running. So starting order wasn't the right way to fix my issue.
For all who are having same issues and using cuxd as addon for homematic I'll advice finding cuxd.ini. There's a simple line needed which disallows making use of tty devices. Just add: TTYPARAM=NONE anywhere to that cuxd.ini. Later if a serial device is needed to be obtained by cuxd then simply define the new device in cuxd.ini and that's it. Hope this helps others avoiding a long story of pulling hairs.

@klaus1k
Copy link

klaus1k commented Dec 17, 2022

Tried running with root privileges, sadly didn't help. So i tried unloading and reloading usbserial and cp210x modules. Also with no effect.

Then i noticed, that i get an error from xhci_hcd right before the first error with cp210x pops up. Maybe the issue lies there? any ideas what i could try? I'm running Z2M in docker on Ubuntu 20.04 on a Minisforum HM50.

[ 151.536962] xhci_hcd 0000:05:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x000f address=0xfff9e000 flags=0x0000]
[ 151.536974] xhci_hcd 0000:05:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x000f address=0xfff9e000 flags=0x0000]
[ 151.536978] xhci_hcd 0000:05:00.0: WARNING: Host System Error
[ 151.536984] xhci_hcd 0000:05:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x000f address=0xfff9e000 flags=0x0000]
[ 151.537474] cp210x ttyUSB0: usb_serial_generic_write_start - error submitting urb: -19
[ 151.538729] cp210x ttyUSB0: failed to get comm status: -19
[ 151.538736] cp210x ttyUSB0: failed set request 0x7 status: -19
[ 151.538743] cp210x ttyUSB0: failed set request 0x12 status: -19
[ 151.538745] cp210x ttyUSB0: failed set request 0x0 status: -19

EDIT: apparently adding GRUB_CMDLINE_LINUX="iommu=pt" to grub fixed the problem at least for now. I don't understand why, but i am not asking ;)

@lnagel
Copy link

lnagel commented Dec 18, 2022

Do you by any chance have a CC2531 sniffer and can sniff the traffic when the coordinator crashes? https://www.zigbee2mqtt.io/advanced/zigbee/04_sniff_zigbee_traffic.html#with-cc2531

I just joined this thread due to having crashes with the SONOFF ZigBee 3.0 adapter. I've upgraded today to 20221214 firmware. I also bought a zigbee sniffer adapter, so let's see. If I get crashes with the new firmware, then I'll setup the sniffer as well. If you have any other tips for how to monitor / debug this, then let us know.

The nice news is that after 8 hours of downtime and then upgrading to 20221214, then all my devices reconnected automatically. I was already dreading the network rebuild whilst driving to location.

@Veldkornet
Copy link
Author

Please try 20221214 from Koenkk/Z-Stack-firmware#383 (comment) . 20221102 is known to be unstable (sometimes)

I've applied 20221220, will let it run for a while and let you know how it goes.

@Veldkornet
Copy link
Author

Veldkornet commented Dec 23, 2022

Please try 20221214 from Koenkk/Z-Stack-firmware#383 (comment) . 20221102 is known to be unstable (sometimes)

I've applied 20221220, will let it run for a while and let you know how it goes.

Nope, already crashed.

**About**
| Zigbee2MQTT version | 1.28.4 commit: unknown |
| Coordinator type | zStack3x0 |
| Coordinator revision | 20221220 |
| Frontend version | 0.6.119 |

**Stats**
- Total: 12

**By device type**
- Router: 6
- End devices: 6

**By power source**
- Mains (single phase): 6
- Battery: 6

**By vendor**
- frient A/S: 10
- _TZ3210_eymunffl: 1
- innr: 1

**By model**
- SPLZB-131: 5
- SMSZB-120: 4
- TS0101: 1
- WISZB-120: 1
- SP 220: 1

@Koenkk
Copy link
Owner

Koenkk commented Dec 23, 2022

@Veldkornet can you provide the herdsman debug log starting from the moment you start z2m until it fails?

See https://www.zigbee2mqtt.io/guide/usage/debug.html on how to enable the herdsman debug logging. Note that this is only logged to STDOUT and not to log files.

@github-actions
Copy link
Contributor

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 7 days

@github-actions github-actions bot added the stale Stale issues label Jan 23, 2023
@lnagel
Copy link

lnagel commented Jan 23, 2023

Do you by any chance have a CC2531 sniffer and can sniff the traffic when the coordinator crashes? https://www.zigbee2mqtt.io/advanced/zigbee/04_sniff_zigbee_traffic.html#with-cc2531

I just joined this thread due to having crashes with the SONOFF ZigBee 3.0 adapter. I've upgraded today to 20221214 firmware. I also bought a zigbee sniffer adapter, so let's see. If I get crashes with the new firmware, then I'll setup the sniffer as well. If you have any other tips for how to monitor / debug this, then let us know.

The nice news is that after 8 hours of downtime and then upgrading to 20221214, then all my devices reconnected automatically. I was already dreading the network rebuild whilst driving to location.

The 20221214 version has been stable without crashes for a month. But I remember now that I added additional routers at the same time as the upgrade which may have increased the network reliability or otherwise altered the used topology. I will try to restore the original network topology and continue testing.

@github-actions github-actions bot removed the stale Stale issues label Jan 24, 2023
@Veldkornet
Copy link
Author

@Veldkornet can you provide the herdsman debug log starting from the moment you start z2m until it fails?

See https://www.zigbee2mqtt.io/guide/usage/debug.html on how to enable the herdsman debug logging. Note that this is only logged to STDOUT and not to log files.

Right after that, I went full nuclear ☢️ Completely removed anything Z2M related, reset all devices, started from scratch.
New PAN ID, Ext PAN ID etc. Still on that newer firmware.

It hasn't gone down once since then, more than a month already. I've even started to revive some old sensors and add them back to the network. Going strong 💪🏼

I was hesitant to report back at first, in case I jinx it 😄

@github-actions
Copy link
Contributor

github-actions bot commented Mar 3, 2023

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 7 days

@github-actions github-actions bot added the stale Stale issues label Mar 3, 2023
@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Mar 11, 2023
@popy2k14
Copy link

popy2k14 commented Mar 12, 2023

@Koenkk Back in business. I have now again an Sonoff Zigbee 3.0 USB Dongle Plus with latest 20221226 firmware.
Added the three devices back, which made the stick crash back than.

And yes, also with the new firmware, it crashes!

PS.: after an addon restart, all is working again. the container/addon is stopped but the Zigbee2MQTT frontend is reachable. It just shows no lines on the network scan, and no command (on/off) is working nor data is received from the devices.

Here is the herdsman log:

Mar 12 18:54:02 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T18:54:02.394Z zigbee-herdsman:adapter:zStack:znp:SRSP <-- AF - dataRequest - {"status":0}
Mar 12 18:54:02 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T18:54:02.394Z zigbee-herdsman:adapter:zStack:unpi:parser --- parseNext []
Mar 12 18:54:02 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T18:54:02.397Z zigbee-herdsman:adapter:zStack:unpi:parser <-- [254,3,68,128,0,1,137,79]
Mar 12 18:54:02 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T18:54:02.398Z zigbee-herdsman:adapter:zStack:unpi:parser --- parseNext [254,3,68,128,0,1,137,79]
Mar 12 18:54:02 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T18:54:02.398Z zigbee-herdsman:adapter:zStack:unpi:parser --> parsed 3 - 2 - 4 - 128 - [0,1,137] - 79
Mar 12 18:54:02 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T18:54:02.398Z zigbee-herdsman:adapter:zStack:znp:AREQ <-- AF - dataConfirm - {"status":0,"endpoint":1,"transid":137}
Mar 12 18:54:02 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T18:54:02.398Z zigbee-herdsman:adapter:zStack:unpi:parser --- parseNext []
Mar 12 18:54:02 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T18:54:02.411Z zigbee-herdsman:adapter:zStack:unpi:parser <-- [254,25,68,129,0,0,8,0,48,127,11,1,0,127,0,171,159,47,0,0,5,24,60,11,4,0,48,127,29,137]
Mar 12 18:54:02 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T18:54:02.411Z zigbee-herdsman:adapter:zStack:unpi:parser --- parseNext [254,25,68,129,0,0,8,0,48,127,11,1,0,127,0,171,159,47,0,0,5,24,60,11,4,0,48,127,29,137]
Mar 12 18:54:02 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T18:54:02.411Z zigbee-herdsman:adapter:zStack:unpi:parser --> parsed 25 - 2 - 4 - 129 - [0,0,8,0,48,127,11,1,0,127,0,171,159,47,0,0,5,24,60,11,4,0,48,127,29] - 137
Mar 12 18:54:02 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T18:54:02.411Z zigbee-herdsman:adapter:zStack:znp:AREQ <-- AF - incomingMsg - {"groupid":0,"clusterid":8,"srcaddr":32560,"srcendpoint":11,"dstendpoint":1,"wasbroadcast":0,"linkquality":127,"securityuse":0,"timestamp":3121067,"transseqnumber":0,"len":5,"data":{"type":"Buffer","data":[24,60,11,4,0]}}
Mar 12 18:54:02 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T18:54:02.412Z zigbee-herdsman:controller:log Received 'zcl' data '{"frame":{"Header":{"frameControl":{"frameType":0,"manufacturerSpecific":false,"direction":1,"disableDefaultResponse":true,"reservedBits":0},"transactionSequenceNumber":60,"manufacturerCode":null,"commandIdentifier":11},"Payload":{"cmdId":4,"statusCode":0},"Command":{"ID":11,"name":"defaultRsp","parameters":[{"name":"cmdId","type":32},{"name":"statusCode","type":32}]}},"address":32560,"endpoint":11,"linkquality":127,"groupID":0,"wasBroadcast":false,"destinationEndpoint":1}'
Mar 12 18:54:02 homeassistant addon_45df7312_zigbee2mqtt[461]: Zigbee2MQTT:info  2023-03-12 19:54:02: MQTT publish: topic 'zigbee2mqtt/Balkon indirekt (Z2M)', payload '{"brightness":254,"color_mode":"xy","last_seen":"2023-03-12T19:54:02+01:00","linkquality":127,"power_on_behavior":null,"state":"OFF","update":{"installed_version":-1,"latest_version":-1,"state":null},"update_available":null}'
Mar 12 18:54:02 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T18:54:02.414Z zigbee-herdsman:adapter:zStack:unpi:parser --- parseNext []
Mar 12 18:54:02 homeassistant addon_45df7312_zigbee2mqtt[461]: Zigbee2MQTT:info  2023-03-12 19:54:02: MQTT publish: topic 'zigbee2mqtt/Alle', payload '{"brightness":254,"color_mode":"xy","state":"ON"}'
Mar 12 18:54:02 homeassistant addon_45df7312_zigbee2mqtt[461]: Zigbee2MQTT:info  2023-03-12 19:54:02: MQTT publish: topic 'zigbee2mqtt/Balkon indirekt (Z2M)', payload '{"brightness":254,"color_mode":"xy","last_seen":"2023-03-12T19:54:02+01:00","linkquality":127,"power_on_behavior":null,"state":"ON","update":{"installed_version":-1,"latest_version":-1,"state":null},"update_available":null}'
Mar 12 18:54:02 homeassistant addon_45df7312_zigbee2mqtt[461]: Zigbee2MQTT:info  2023-03-12 19:54:02: MQTT publish: topic 'zigbee2mqtt/Alle ohne Weihnachtsbeleuchtung', payload '{"brightness":254,"color_mode":"xy","state":"ON"}'
Mar 12 18:54:02 homeassistant addon_45df7312_zigbee2mqtt[461]: Zigbee2MQTT:info  2023-03-12 19:54:02: MQTT publish: topic 'zigbee2mqtt/Balkon', payload '{"brightness":254,"color_mode":"xy","state":"ON"}'
Mar 12 18:54:03 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T18:54:03.231Z zigbee-herdsman:adapter:zStack:unpi:parser <-- [254,33,68,129,0,0,0,252,95,239,2,242,0,142,0,206,103,48,0,0,13,29,11,16,101,0,4,0,0,48,0,33,0,0,95,239,29,153]
Mar 12 18:54:03 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T18:54:03.231Z zigbee-herdsman:adapter:zStack:unpi:parser --- parseNext [254,33,68,129,0,0,0,252,95,239,2,242,0,142,0,206,103,48,0,0,13,29,11,16,101,0,4,0,0,48,0,33,0,0,95,239,29,153]
Mar 12 18:54:03 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T18:54:03.231Z zigbee-herdsman:adapter:zStack:unpi:parser --> parsed 33 - 2 - 4 - 129 - [0,0,0,252,95,239,2,242,0,142,0,206,103,48,0,0,13,29,11,16,101,0,4,0,0,48,0,33,0,0,95,239,29] - 153
Mar 12 18:54:03 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T18:54:03.232Z zigbee-herdsman:adapter:zStack:znp:AREQ <-- AF - incomingMsg - {"groupid":0,"clusterid":64512,"srcaddr":61279,"srcendpoint":2,"dstendpoint":242,"wasbroadcast":0,"linkquality":142,"securityuse":0,"timestamp":3172302,"transseqnumber":0,"len":13,"data":{"type":"Buffer","data":[29,11,16,101,0,4,0,0,48,0,33,0,0]}}
Mar 12 18:54:03 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T18:54:03.232Z zigbee-herdsman:controller:log Received 'zcl' data '{"frame":{"Header":{"frameControl":{"frameType":1,"manufacturerSpecific":true,"direction":1,"disableDefaultResponse":true,"reservedBits":0},"transactionSequenceNumber":101,"manufacturerCode":4107,"commandIdentifier":0},"Payload":{"button":4,"unknown1":3145728,"type":0,"unknown2":0,"time":0},"Command":{"ID":0,"parameters":[{"name":"button","type":32},{"name":"unknown1","type":34},{"name":"type","type":32},{"name":"unknown2","type":32},{"name":"time","type":32},{"name":"unknown2","type":32}],"name":"hueNotification"}},"address":61279,"endpoint":2,"linkquality":142,"groupID":0,"wasBroadcast":false,"destinationEndpoint":242}'
Mar 12 18:54:03 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T18:54:03.234Z zigbee-herdsman:adapter:zStack:unpi:parser --- parseNext []
Mar 12 18:54:03 homeassistant addon_45df7312_zigbee2mqtt[461]: Zigbee2MQTT:info  2023-03-12 19:54:03: MQTT publish: topic 'zigbee2mqtt/Balkon Dimmerschalter', payload '{"action":"off_press","action_duration":null,"battery":10,"last_seen":"2023-03-12T19:54:03+01:00","linkquality":142,"update":{"installed_version":1107324829,"latest_version":1107324829,"state":"idle"},"update_available":null}'
Mar 12 18:54:03 homeassistant addon_45df7312_zigbee2mqtt[461]: Zigbee2MQTT:info  2023-03-12 19:54:03: MQTT publish: topic 'zigbee2mqtt/Balkon Dimmerschalter', payload '{"action":"","action_duration":null,"battery":10,"last_seen":"2023-03-12T19:54:03+01:00","linkquality":142,"update":{"installed_version":1107324829,"latest_version":1107324829,"state":"idle"},"update_available":null}'
Mar 12 18:54:03 homeassistant addon_45df7312_zigbee2mqtt[461]: Zigbee2MQTT:info  2023-03-12 19:54:03: MQTT publish: topic 'zigbee2mqtt/Balkon Dimmerschalter/action', payload 'off_press'
Mar 12 18:54:03 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T18:54:03.239Z zigbee-herdsman:adapter:zStack:unpi:parser <-- [254,25,68,129,0,0,6,0,95,239,1,242,0,142,0,190,104,48,0,0,5,1,102,64,0,0,95,239,29,126]
Mar 12 18:54:03 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T18:54:03.239Z zigbee-herdsman:adapter:zStack:unpi:parser --- parseNext [254,25,68,129,0,0,6,0,95,239,1,242,0,142,0,190,104,48,0,0,5,1,102,64,0,0,95,239,29,126]
Mar 12 18:54:03 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T18:54:03.239Z zigbee-herdsman:adapter:zStack:unpi:parser --> parsed 25 - 2 - 4 - 129 - [0,0,6,0,95,239,1,242,0,142,0,190,104,48,0,0,5,1,102,64,0,0,95,239,29] - 126
Mar 12 18:54:03 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T18:54:03.239Z zigbee-herdsman:adapter:zStack:znp:AREQ <-- AF - incomingMsg - {"groupid":0,"clusterid":6,"srcaddr":61279,"srcendpoint":1,"dstendpoint":242,"wasbroadcast":0,"linkquality":142,"securityuse":0,"timestamp":3172542,"transseqnumber":0,"len":5,"data":{"type":"Buffer","data":[1,102,64,0,0]}}
Mar 12 18:54:03 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T18:54:03.240Z zigbee-herdsman:controller:log Received 'zcl' data '{"frame":{"Header":{"frameControl":{"frameType":1,"manufacturerSpecific":false,"direction":0,"disableDefaultResponse":false,"reservedBits":0},"transactionSequenceNumber":102,"manufacturerCode":null,"commandIdentifier":64},"Payload":{"effectid":0,"effectvariant":0},"Command":{"ID":64,"parameters":[{"name":"effectid","type":32},{"name":"effectvariant","type":32}],"name":"offWithEffect"}},"address":61279,"endpoint":1,"linkquality":142,"groupID":0,"wasBroadcast":false,"destinationEndpoint":242}'
Mar 12 18:54:03 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T18:54:03.242Z zigbee-herdsman:controller:endpoint DefaultResponse 0x0017880103e5e9a3/1 6(64, {"sendWhen":"immediate","timeout":10000,"disableResponse":false,"disableRecovery":false,"disableDefaultResponse":true,"direction":1,"srcEndpoint":null,"reservedBits":0,"manufacturerCode":null,"transactionSequenceNumber":null,"writeUndiv":false})
Mar 12 18:54:03 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T18:54:03.243Z zigbee-herdsman:adapter:zStack:unpi:parser --- parseNext []
Mar 12 18:54:03 homeassistant addon_45df7312_zigbee2mqtt[461]: Zigbee2MQTT:info  2023-03-12 19:54:03: MQTT publish: topic 'zigbee2mqtt/Balkon Dimmerschalter', payload '{"action":null,"action_duration":null,"battery":10,"last_seen":"2023-03-12T19:54:03+01:00","linkquality":142,"update":{"installed_version":1107324829,"latest_version":1107324829,"state":"idle"},"update_available":null}'
Mar 12 18:54:03 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T18:54:03.288Z zigbee-herdsman:controller:endpoint Command 0x0017880103169f9c/11 genLevelCtrl.moveToLevelWithOnOff({"level":0,"transtime":10}, {"sendWhen":"immediate","timeout":10000,"disableResponse":false,"disableRecovery":false,"disableDefaultResponse":false,"direction":0,"srcEndpoint":null,"reservedBits":0,"manufacturerCode":null,"transactionSequenceNumber":null,"writeUndiv":false})
Mar 12 18:54:03 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T18:54:03.288Z zigbee-herdsman:adapter:zStack:adapter sendZclFrameToEndpointInternal 0x0017880103169f9c:32560/11 (0,0,3)
Mar 12 18:54:03 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T18:54:03.288Z zigbee-herdsman:adapter:zStack:znp:SREQ --> AF - dataRequest - {"dstaddr":32560,"destendpoint":11,"srcendpoint":1,"clusterid":8,"transid":138,"options":0,"radius":30,"len":6,"data":{"type":"Buffer","data":[1,61,4,0,10,0]}}
Mar 12 18:54:03 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T18:54:03.289Z zigbee-herdsman:adapter:zStack:unpi:writer --> frame [254,16,36,1,48,127,11,1,8,0,138,0,30,6,1,61,4,0,10,0,216]
Mar 12 18:54:03 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T18:54:03.294Z zigbee-herdsman:adapter:zStack:unpi:parser <-- [254,1,100,1,0,100]
Mar 12 18:54:03 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T18:54:03.294Z zigbee-herdsman:adapter:zStack:unpi:parser --- parseNext [254,1,100,1,0,100]
Mar 12 18:54:03 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T18:54:03.294Z zigbee-herdsman:adapter:zStack:unpi:parser --> parsed 1 - 3 - 4 - 1 - [0] - 100
Mar 12 18:54:03 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T18:54:03.294Z zigbee-herdsman:adapter:zStack:znp:SRSP <-- AF - dataRequest - {"status":0}
Mar 12 18:54:03 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T18:54:03.294Z zigbee-herdsman:adapter:zStack:unpi:parser --- parseNext []
Mar 12 18:54:03 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T18:54:03.298Z zigbee-herdsman:adapter:zStack:unpi:parser <-- [254,3,68,128,0,1,138,76]
Mar 12 18:54:03 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T18:54:03.298Z zigbee-herdsman:adapter:zStack:unpi:parser --- parseNext [254,3,68,128,0,1,138,76]
Mar 12 18:54:03 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T18:54:03.298Z zigbee-herdsman:adapter:zStack:unpi:parser --> parsed 3 - 2 - 4 - 128 - [0,1,138] - 76
Mar 12 18:54:03 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T18:54:03.298Z zigbee-herdsman:adapter:zStack:znp:AREQ <-- AF - dataConfirm - {"status":0,"endpoint":1,"transid":138}
Mar 12 18:54:03 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T18:54:03.299Z zigbee-herdsman:adapter:zStack:unpi:parser --- parseNext []
Mar 12 18:54:03 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T18:54:03.305Z zigbee-herdsman:adapter:zStack:unpi:parser <-- [254,33,68,129,0,0,0,252,95,239,2,242,0,142,0,246,121,48,0,0,13,29,11,16,103,0,4,0,0,48,2,33,0,0,95,239,29,191]
Mar 12 18:54:03 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T18:54:03.305Z zigbee-herdsman:adapter:zStack:unpi:parser --- parseNext [254,33,68,129,0,0,0,252,95,239,2,242,0,142,0,246,121,48,0,0,13,29,11,16,103,0,4,0,0,48,2,33,0,0,95,239,29,191]
Mar 12 18:54:03 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T18:54:03.306Z zigbee-herdsman:adapter:zStack:unpi:parser --> parsed 33 - 2 - 4 - 129 - [0,0,0,252,95,239,2,242,0,142,0,246,121,48,0,0,13,29,11,16,103,0,4,0,0,48,2,33,0,0,95,239,29] - 191
Mar 12 18:54:03 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T18:54:03.306Z zigbee-herdsman:adapter:zStack:znp:AREQ <-- AF - incomingMsg - {"groupid":0,"clusterid":64512,"srcaddr":61279,"srcendpoint":2,"dstendpoint":242,"wasbroadcast":0,"linkquality":142,"securityuse":0,"timestamp":3176950,"transseqnumber":0,"len":13,"data":{"type":"Buffer","data":[29,11,16,103,0,4,0,0,48,2,33,0,0]}}
Mar 12 18:54:03 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T18:54:03.306Z zigbee-herdsman:controller:log Received 'zcl' data '{"frame":{"Header":{"frameControl":{"frameType":1,"manufacturerSpecific":true,"direction":1,"disableDefaultResponse":true,"reservedBits":0},"transactionSequenceNumber":103,"manufacturerCode":4107,"commandIdentifier":0},"Payload":{"button":4,"unknown1":3145728,"type":2,"unknown2":0,"time":0},"Command":{"ID":0,"parameters":[{"name":"button","type":32},{"name":"unknown1","type":34},{"name":"type","type":32},{"name":"unknown2","type":32},{"name":"time","type":32},{"name":"unknown2","type":32}],"name":"hueNotification"}},"address":61279,"endpoint":2,"linkquality":142,"groupID":0,"wasBroadcast":false,"destinationEndpoint":242}'
Mar 12 18:54:03 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T18:54:03.308Z zigbee-herdsman:adapter:zStack:unpi:parser --- parseNext []
Mar 12 18:54:03 homeassistant addon_45df7312_zigbee2mqtt[461]: Zigbee2MQTT:info  2023-03-12 19:54:03: MQTT publish: topic 'zigbee2mqtt/Balkon Dimmerschalter', payload '{"action":"off_press_release","action_duration":null,"battery":10,"last_seen":"2023-03-12T19:54:03+01:00","linkquality":142,"update":{"installed_version":1107324829,"latest_version":1107324829,"state":"idle"},"update_available":null}'
Mar 12 18:54:03 homeassistant addon_45df7312_zigbee2mqtt[461]: Zigbee2MQTT:info  2023-03-12 19:54:03: MQTT publish: topic 'zigbee2mqtt/Balkon Dimmerschalter', payload '{"action":"","action_duration":null,"battery":10,"last_seen":"2023-03-12T19:54:03+01:00","linkquality":142,"update":{"installed_version":1107324829,"latest_version":1107324829,"state":"idle"},"update_available":null}'
Mar 12 18:54:03 homeassistant addon_45df7312_zigbee2mqtt[461]: Zigbee2MQTT:info  2023-03-12 19:54:03: MQTT publish: topic 'zigbee2mqtt/Balkon Dimmerschalter/action', payload 'off_press_release'
Mar 12 18:54:03 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T18:54:03.315Z zigbee-herdsman:adapter:zStack:unpi:parser <-- [254,25,68,129,0,0,8,0,48,127,11,1,0,116,0,216,123,48,0,0,5,24,61,11,4,0,48,127,29,11]
Mar 12 18:54:03 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T18:54:03.316Z zigbee-herdsman:adapter:zStack:unpi:parser --- parseNext [254,25,68,129,0,0,8,0,48,127,11,1,0,116,0,216,123,48,0,0,5,24,61,11,4,0,48,127,29,11]
Mar 12 18:54:03 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T18:54:03.316Z zigbee-herdsman:adapter:zStack:unpi:parser --> parsed 25 - 2 - 4 - 129 - [0,0,8,0,48,127,11,1,0,116,0,216,123,48,0,0,5,24,61,11,4,0,48,127,29] - 11
Mar 12 18:54:03 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T18:54:03.316Z zigbee-herdsman:adapter:zStack:znp:AREQ <-- AF - incomingMsg - {"groupid":0,"clusterid":8,"srcaddr":32560,"srcendpoint":11,"dstendpoint":1,"wasbroadcast":0,"linkquality":116,"securityuse":0,"timestamp":3177432,"transseqnumber":0,"len":5,"data":{"type":"Buffer","data":[24,61,11,4,0]}}
Mar 12 18:54:03 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T18:54:03.317Z zigbee-herdsman:controller:log Received 'zcl' data '{"frame":{"Header":{"frameControl":{"frameType":0,"manufacturerSpecific":false,"direction":1,"disableDefaultResponse":true,"reservedBits":0},"transactionSequenceNumber":61,"manufacturerCode":null,"commandIdentifier":11},"Payload":{"cmdId":4,"statusCode":0},"Command":{"ID":11,"name":"defaultRsp","parameters":[{"name":"cmdId","type":32},{"name":"statusCode","type":32}]}},"address":32560,"endpoint":11,"linkquality":116,"groupID":0,"wasBroadcast":false,"destinationEndpoint":1}'
Mar 12 18:54:03 homeassistant addon_45df7312_zigbee2mqtt[461]: Zigbee2MQTT:info  2023-03-12 19:54:03: MQTT publish: topic 'zigbee2mqtt/Balkon indirekt (Z2M)', payload '{"brightness":254,"color_mode":"xy","last_seen":"2023-03-12T19:54:03+01:00","linkquality":116,"power_on_behavior":null,"state":"ON","update":{"installed_version":-1,"latest_version":-1,"state":null},"update_available":null}'
Mar 12 18:54:03 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T18:54:03.321Z zigbee-herdsman:adapter:zStack:unpi:parser --- parseNext []
Mar 12 18:54:03 homeassistant addon_45df7312_zigbee2mqtt[461]: Zigbee2MQTT:info  2023-03-12 19:54:03: MQTT publish: topic 'zigbee2mqtt/Alle', payload '{"brightness":254,"color_mode":"xy","state":"OFF"}'
Mar 12 18:54:03 homeassistant addon_45df7312_zigbee2mqtt[461]: Zigbee2MQTT:info  2023-03-12 19:54:03: MQTT publish: topic 'zigbee2mqtt/Balkon indirekt (Z2M)', payload '{"brightness":254,"color_mode":"xy","last_seen":"2023-03-12T19:54:03+01:00","linkquality":116,"power_on_behavior":null,"state":"OFF","update":{"installed_version":-1,"latest_version":-1,"state":null},"update_available":null}'
Mar 12 18:54:03 homeassistant addon_45df7312_zigbee2mqtt[461]: Zigbee2MQTT:info  2023-03-12 19:54:03: MQTT publish: topic 'zigbee2mqtt/Alle ohne Weihnachtsbeleuchtung', payload '{"brightness":254,"color_mode":"xy","state":"OFF"}'
Mar 12 18:54:03 homeassistant addon_45df7312_zigbee2mqtt[461]: Zigbee2MQTT:info  2023-03-12 19:54:03: MQTT publish: topic 'zigbee2mqtt/Balkon', payload '{"brightness":254,"color_mode":"xy","state":"OFF"}'
Mar 12 18:54:07 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T18:54:07.531Z zigbee-herdsman:adapter:zStack:unpi:parser <-- [254,3,68,128,0,1,136,78]
Mar 12 18:54:07 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T18:54:07.531Z zigbee-herdsman:adapter:zStack:unpi:parser --- parseNext [254,3,68,128,0,1,136,78]
Mar 12 18:54:07 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T18:54:07.531Z zigbee-herdsman:adapter:zStack:unpi:parser --> parsed 3 - 2 - 4 - 128 - [0,1,136] - 78
Mar 12 18:54:07 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T18:54:07.531Z zigbee-herdsman:adapter:zStack:znp:AREQ <-- AF - dataConfirm - {"status":0,"endpoint":1,"transid":136}
Mar 12 18:54:07 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T18:54:07.532Z zigbee-herdsman:adapter:zStack:unpi:parser --- parseNext []
Mar 12 18:54:07 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T18:54:07.532Z zigbee-herdsman:adapter:zStack:adapter sendZclFrameToEndpointInternal 0x0017880103e5e9a3:61279/1 (0,0,1)
Mar 12 18:54:07 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T18:54:07.532Z zigbee-herdsman:adapter:zStack:znp:SREQ --> AF - dataRequest - {"dstaddr":61279,"destendpoint":1,"srcendpoint":1,"clusterid":6,"transid":139,"options":0,"radius":30,"len":5,"data":{"type":"Buffer","data":[24,102,11,64,0]}}
Mar 12 18:54:07 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T18:54:07.532Z zigbee-herdsman:adapter:zStack:unpi:writer --> frame [254,15,36,1,95,239,1,1,6,0,139,0,30,5,24,102,11,64,0,57]
Mar 12 18:54:07 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T18:54:07.536Z zigbee-herdsman:adapter:zStack:unpi:parser <-- [254,1,100,1,0,100]
Mar 12 18:54:07 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T18:54:07.537Z zigbee-herdsman:adapter:zStack:unpi:parser --- parseNext [254,1,100,1,0,100]
Mar 12 18:54:07 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T18:54:07.537Z zigbee-herdsman:adapter:zStack:unpi:parser --> parsed 1 - 3 - 4 - 1 - [0] - 100
Mar 12 18:54:07 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T18:54:07.537Z zigbee-herdsman:adapter:zStack:znp:SRSP <-- AF - dataRequest - {"status":0}
Mar 12 18:54:07 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T18:54:07.537Z zigbee-herdsman:adapter:zStack:unpi:parser --- parseNext []
Mar 12 18:54:12 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T18:54:12.781Z zigbee-herdsman:adapter:zStack:unpi:parser <-- [254,3,68,128,0,1,139,77]
Mar 12 18:54:12 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T18:54:12.781Z zigbee-herdsman:adapter:zStack:unpi:parser --- parseNext [254,3,68,128,0,1,139,77]
Mar 12 18:54:12 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T18:54:12.781Z zigbee-herdsman:adapter:zStack:unpi:parser --> parsed 3 - 2 - 4 - 128 - [0,1,139] - 77
Mar 12 18:54:12 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T18:54:12.781Z zigbee-herdsman:adapter:zStack:znp:AREQ <-- AF - dataConfirm - {"status":0,"endpoint":1,"transid":139}
Mar 12 18:54:12 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T18:54:12.781Z zigbee-herdsman:adapter:zStack:unpi:parser --- parseNext []
Mar 12 19:02:44 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T19:02:44.558Z zigbee-herdsman:controller:database:log Writing database to '/config/zigbee2mqtt/database.db'
Mar 12 19:04:04 homeassistant addon_45df7312_zigbee2mqtt[461]: Zigbee2MQTT:info  2023-03-12 20:04:04: Starting network scan (includeRoutes 'false')
Mar 12 19:04:05 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T19:04:05.208Z zigbee-herdsman:adapter:zStack:znp:SREQ --> ZDO - mgmtLqiReq - {"dstaddr":0,"startindex":0}
Mar 12 19:04:05 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T19:04:05.208Z zigbee-herdsman:adapter:zStack:unpi:writer --> frame [254,3,37,49,0,0,0,23]
Mar 12 19:04:16 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T19:04:16.213Z zigbee-herdsman:adapter:zStack:znp:SREQ --> ZDO - mgmtLqiReq - {"dstaddr":0,"startindex":0}
Mar 12 19:04:16 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T19:04:16.213Z zigbee-herdsman:adapter:zStack:unpi:writer --> frame [254,3,37,49,0,0,0,23]
Mar 12 19:04:22 homeassistant addon_45df7312_zigbee2mqtt[461]: Zigbee2MQTT:error 2023-03-12 20:04:22: Failed to execute LQI for 'Coordinator'
Mar 12 19:04:23 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T19:04:23.234Z zigbee-herdsman:adapter:zStack:znp:SREQ --> ZDO - mgmtLqiReq - {"dstaddr":32560,"startindex":0}
Mar 12 19:04:23 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T19:04:23.235Z zigbee-herdsman:adapter:zStack:unpi:writer --> frame [254,3,37,49,48,127,0,88]
Mar 12 19:04:34 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T19:04:34.243Z zigbee-herdsman:adapter:zStack:znp:SREQ --> ZDO - mgmtLqiReq - {"dstaddr":32560,"startindex":0}
Mar 12 19:04:34 homeassistant addon_45df7312_zigbee2mqtt[461]: 2023-03-12T19:04:34.243Z zigbee-herdsman:adapter:zStack:unpi:writer --> frame [254,3,37,49,48,127,0,88]
Mar 12 19:04:40 homeassistant addon_45df7312_zigbee2mqtt[461]: Zigbee2MQTT:error 2023-03-12 20:04:40: Failed to execute LQI for 'Balkon indirekt (Z2M)'
Mar 12 19:04:40 homeassistant addon_45df7312_zigbee2mqtt[461]: Zigbee2MQTT:info  2023-03-12 20:04:40: Network scan finished
Mar 12 19:04:40 homeassistant addon_45df7312_zigbee2mqtt[461]: Zigbee2MQTT:info  2023-03-12 20:04:40: MQTT publish: topic 'zigbee2mqtt/bridge/response/networkmap', payload '{"data":{"routes":false,"type":"raw","value":{"links":[],"nodes":[{"definition":null,"failed":["lqi"],"friendlyName":"Coordinator","ieeeAddr":"0x00124b002a2e96db","lastSeen":null,"networkAddress":0,"type":"Coordinator"},{"definition":{"description":"Hue dimmer switch","model":"324131092621","supports":"battery, action, action_duration, linkquality","vendor":"Philips"},"friendlyName":"Balkon Dimmerschalter","ieeeAddr":"0x0017880103e5e9a3","lastSeen":1678647243307,"manufacturerName":"Philips","modelID":"RWL021","networkAddress":61279,"type":"EndDevice"},{"definition":{"description":"Hue White A60 Single bulb E27/B22","model":"8718696449691","supports":"light (state, brightness), effect, power_on_behavior, linkquality","vendor":"Philips"},"failed":["lqi"],"friendlyName":"Balkon indirekt (Z2M)","ieeeAddr":"0x0017880103169f9c","lastSeen":1678647243317,"manufacturerName":"Signify Netherlands B.V.","modelID":"LWB010","networkAddress":32560,"type":"Router"},{"definition":{"description":"Hue motion outdoor sensor","model":"9290019758","supports":"temperature, occupancy, battery, illuminance_lux, illuminance, motion_sensitivity, led_indication, occupancy_timeout, linkquality","vendor":"Philips"},"friendlyName":"Balkon Bewegungsmelder","ieeeAddr":"0x00178801064432a9","lastSeen":1678647118488,"manufacturerName":"Philips","modelID":"SML002","networkAddress":51256,"type":"EndDevice"}]}},"status":"ok","transaction":"irv4v-1"}'

It crashed (doesnt work) around: 18:54:12
Yes, i have made an network scan afterwards to check if any device is responding, sadly not.

BUT this time i added the devices in the following order:

  • Philips 324131092621 (Dimmer RWL021)
  • Philips 8718696449691 (Light LWB010)
  • Philips 9290019758 (Motion sensor SML002)

It worked without an crash ~6h, but after i added the last device (SML002) it crashed twice in short time (minutes).

Maybe there is an issue with the hue motion sensor?
How can i help to debug this?

Ps.: The sensor is working without an issue on deconz! So it's NOT defect!

@popy2k14
Copy link

Found the following in my dmesg around that time it failed:

➜  ~ dmesg | grep cp210x
[12.03.2023 18:52:26] cp210x ttyUSB0: usb_serial_generic_read_bulk_callback - urb stopped: -32
[12.03.2023 18:52:26] cp210x ttyUSB0: usb_serial_generic_read_bulk_callback - urb stopped: -32
[12.03.2023 19:52:30] cp210x ttyUSB0: usb_serial_generic_read_bulk_callback - urb stopped: -32
[12.03.2023 19:52:30] cp210x ttyUSB0: usb_serial_generic_read_bulk_callback - urb stopped: -32
➜  ~

That seemed to be an long ongoing kernel bug which is not solved up until now.
See:

raspberrypi/linux#2406
raspberrypi/linux#5211
https://forum.odroid.com/viewtopic.php?t=40719

Ps.: Running an Odroid N2+

The issue seems to affect various manufactures of USB uarts, see the issues linkes above.

What seems to help is using an active powered usb hub in between, not because of the high power draw of the usb-uart devices, rather to wokaround the issue.

If the issue pops up again, i will throw an unpowered usb hub in between and see if the issue is gone.

What's a little bit wierd is, that my conbee II with deconz addon running good for months.
conbee II is using an ftdi.
Maybe deconz is doing some reopening when uart/stick is lost!?

Does zigbee2mqtt some retries of this sort?
If not, this would be a nice feature.

@popy2k14
Copy link

Today it crashed again, in dmesg are the exact same messages. Will migrate to an x86 hardware and see if the issue is gone. Sorry for spamming this thread. But maybe this info still help someone, with the same issues.

Will post an update after the migration with the exact same config (HA full backup) when it's done.

@popy2k14
Copy link

Running the new x86 hardware with an HA backup for an day and so far so good, no crash!
Seems the issue is really kernel arm based.

@popy2k14
Copy link

popy2k14 commented Apr 1, 2023

After 2 weeks, Z2M on x86 with sonoff dongle p is rock solid!

@MOST2K2
Copy link

MOST2K2 commented May 9, 2023

@klaus1k Indeed this looks like a driver issue. Have a try starting z2m with root privileges. If that helps you'll need to dig deeper in how to start containers as non-root.

@srvoets I finally found the root cause. My z2m instance restarted right after homematic instance was running. So starting order wasn't the right way to fix my issue. For all who are having same issues and using cuxd as addon for homematic I'll advice finding cuxd.ini. There's a simple line needed which disallows making use of tty devices. Just add: TTYPARAM=NONE anywhere to that cuxd.ini. Later if a serial device is needed to be obtained by cuxd then simply define the new device in cuxd.ini and that's it. Hope this helps others avoiding a long story of pulling hairs.

You saved my day @adfx , big thanks!

@EpicLPer
Copy link

This might be a "stupid" thing to blurt into this conversation here, but everything was fine and dandy for the past months without any hiccups or anything (apart from some huge delayed response times from devices but that's beyond the point).
A few days ago my grandpa did something on our electrical, which tripped the breaker and despite having a UPS somehow made my Sonoff Dongle Plus crash... and ever since then it tends to crash randomly without any warnings. 2 days ago it crashed without anyone tripping our breaker, and yesterday it did the same.

Just as a "this is my experience" kind of info dump here :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
problem Something isn't working stale Stale issues
Projects
None yet
Development

No branches or pull requests