org.bluez.Device1 Errors & Random System Crashes After OS 10 Upgrade

[jmc1983]

The problem

Since upgrading to OS 10, on a irregular basis (every 1-3 days) the log overflows with around three million entries of the following error message, then the system fails in various ways (locks up) and requires a power cycle reboot to fix:

2023-05-17 07:48:54.722 ERROR (MainThread) [root] A message handler raised an exception: 'org.bluez.Device1'.
Traceback (most recent call last):
File "src/dbus_fast/message_bus.py", line 811, in dbus_fast.message_bus.BaseMessageBus._process_message
File "/usr/local/lib/python3.10/site-packages/bleak/backends/bluezdbus/manager.py", line 854, in _parse_msg
condition_callback()
File "/usr/local/lib/python3.10/site-packages/bleak/backends/bluezdbus/manager.py", line 709, in callback
self._properties[device_path][defs.DEVICE_INTERFACE][property_name]
KeyError: 'org.bluez.Device1'

In the most recent instance, the first such error appeared at 07:48:54.722. The error repeated nearly 3 million times, until 17:54:09.337, at which point the log reflects "NUL" characters and ends.

At 17:54, Z-Wave Zigbee and BT all went offline; the Settings-System page could not be accessed ("Error while loading page system."); the Logbook stopped recording entries; automations stopped triggering; the device no longer was accessible via the network to view the log files; etc. I solve this issue with a hard reboot/power cycle. After the system came back online, the /config/home-assistant.log.1 file (timestamped 17:54) was 1.3GB in size. The above error message came from this file.

This issue started after upgrading to Operating System version 10. 10.1 did not fix the issue.

These errors do not seem to correlate with anything happening in the house. They have happened at various times of the day, including once at 2:21am when there were no automations/commands/etc.

The setup is: Running a RPi 4. Bluetooth adapter is an external Sena UD100-G03; the internal BT adapter is disabled. Also connected are a USB SSD; and a Nortek HUSBZB-1 dongle. Home Assistant 2023.5.3 - Supervisor 2023.04.1 - Operating System 10.1 - Frontend 20230503.3 - latest.

What version of Home Assistant Core has the issue?

core-2023.5.3

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant OS

Integration causing the issue

Bluetooth

Link to integration documentation on our website

https://www.home-assistant.io/integrations/bluetooth/

Diagnostics information

config_entry-bluetooth-25dd42f1ce05be8f45fb3d475a0f7e91.json.txt

Example YAML snippet

No response

Anything in the logs that might be useful for us?

IndexError: list index out of range
2023-05-17 07:48:54.638 ERROR (MainThread) [root] A message handler raised an exception: 'org.bluez.Device1'.
Traceback (most recent call last):
  File "src/dbus_fast/message_bus.py", line 811, in dbus_fast.message_bus.BaseMessageBus._process_message
  File "/usr/local/lib/python3.10/site-packages/bleak/backends/bluezdbus/manager.py", line 854, in _parse_msg
    condition_callback()
  File "/usr/local/lib/python3.10/site-packages/bleak/backends/bluezdbus/manager.py", line 709, in callback
    self._properties[device_path][defs.DEVICE_INTERFACE][property_name]
KeyError: 'org.bluez.Device1'

Additional information

No response

[home-assistant]

Hey there @bdraco, mind taking a look at this issue as it has been labeled with an integration (bluetooth) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of bluetooth can trigger bot actions by commenting:

• @home-assistant close Closes the issue.
• @home-assistant rename Awesome new title Renames the issue.
• @home-assistant reopen Reopen the issue.
• @home-assistant unassign bluetooth Removes the current integration label and assignees on the issue, add the integration domain after the command.

_{^{(message by CodeOwnersMention)}}

---

bluetooth documentation
bluetooth source
_{^{(message by IssueLinks)}}

[jmc1983]

For those experiencing the same, I resolved this by disabled the Bluetooth integration on my RPi instance and set up a couple of ESP32 Bluetooth proxy devices, which has resolved the crash caused by the BT integration while also maintaining connectivity to the BT devices in HA.

[bdraco]

I opened a PR to handle the case where the device is removed from the bus while we are waiting for a change hbldh/bleak#1329

[bdraco]

#99520 will fix the case where the device is removed from the bus while connecting. If the adapter itself it unstable, there isn't anything we can do about that though.

If you have an unstable adapter problem, please continue in #92379