Set up Presence Detection

[swann05]

Hi, I'm sorry if it isn't the right spot to ask this. I'm new in home automation.

I try to set up a Presence detection via Bluetooth and was thinking about detection through the lock. I've found that i can see some devices through mqtt > nuki/presence/devices.
But i can't understand why i can't see my phone. There are 4 devices : nuki lock, and 3 others that i guess to be my sonos devices.
That's it ! I can't see my phone or my wife's ...

I have HA and running HA companion with beacon activated.

[swann05]

I've done some other tests.
Using nrf connect to advertise my BLE, my address appears in presence/devices only when i select "complete name" to advertise.
Using an esp32 with espresense, i can see my ble advertised by HA companion but only with an id. No mac address and no name.
It seems that presence/devices needs a name and an address to be displayed but HA companion doesn't allow these features.

[technyon]

A bluetooth device has to advertise itself using beacons. It seems you've solved that part by using nrf connect. I'd suggest to get an actual ble beacon you can for example put on your keychain. Having for phone constantly sending beacons will drain its batteries. For me Gigaset G-Tag work quite well, but other tags should work too.

[swann05]

I understand but HA companion can simulate a beacon on my phone. But it seems that the infos which are advertise doesn't fit the infos asked by your code (name and address). Is it possible?
As i said in espresense, i can see my phone "beacon" but only with an id, no name or address.
Anyway i will look into an true beacon tag.

[technyon]

The thing is it's not exactly standardized what a ble beacon looks like. There many format iBeacon and eddystone being the two most known. I'm not exactly sure what kind of beacon HA companion simulates. You can try other apps, there are plenty that simulate beacons, and usually you can select what kind of beacon you want to simulate.

[swann05]

HA companion simulates iBeacon.
Espresense can identify it.
https://github.com/ESPresense/ESPresense/tree/master/lib/BleFingerprint
I don't know much about code but there's a line that seems to say "if no name, set id as name". Maybe that's the missing part.
As i said i'll try beacons but the idea of having only one item to wear (my phone) was sweet to me.

[technyon]

There's indeed a check that a device has to have a name. If I omit this check, I see a crazy amount of devices in my area - too many to transfer via MQTT due to memory constraints. I understand that just having one device looks appealing, but do you really want to drain your phone battery due to this? Those beacons are always transmitted, wether you're home or not. It could be made configurable to skip the name-check, maybe it works in less urban environments.

[mundschenk-at]

Maybe have a list of IDs that are passed through?

[swann05]

I simulate an ibeacon with HA companion for 2 days now and i don't notice a major difference on my battery.
Indeed espresense add a filter to only transmit ids you allow. In my area, i have only 5/6 devices detected and there are all mine ^^

[swann05]

I'm sorry if i ask stupid things but the "presence/devices" topics is not in json ? Am i right ?
Everything is explain in json format in HA so i'm a bit lost.
In espresence there are several topics "deviceid1", "adress_device1", "rssi_device1" ...etc
It's way more easy to read and put in a sensor.

I'm sorry, I realize I did not say thank you for the amazing work. As i said i'm new to automation and my english is poor so i went straight to the point...

[technyon]

Hi,

yes I know this is a bit inconsistent, but JSON has a lot of redundant information, and I need to keep the message short due to memory constraints. CSV is a lot more space efficient. The presence detection is somewhat of a byproduct since NUKI signals state changes via beacons, so they are received already, NUKI Hub is just publishing them to MQTT, but not its main functionality. Maybe the easy solution is to just run another ESP with espresence?

[swann05]

"The presence detection is somewhat of a byproduct since NUKI signals state changes via beacons, so they are received already"

I fear i miss something... how do you set your lock to open when you're tag is recognized?

[mundschenk-at]

@swann05 I think @technyon meant that presence detection is not the primary feature of Nuki Hub and so does not attempt to support every use-case.

[swann05]

Oh i understand if it's not the purpose of the development.
I think it's a cool feature to monitor presence without adding multiple sensor.
I've tested espresence but esp has limit range of bluetooth and the esp need to be near my door as the esp already running nuki hub (hidden in my doorbell and no more room to add another).
I'll try to find the code to use it as it is !
Thanks for the good work !

[swann05]

Someone gave me the code for HA. If someone is searching the same feature than me later, i put it here. But it doesn't work with HA companion because it doesn't advertise "name". Set the value higher than 50, if you want a wider range.

mqtt:

• device_tracker:
  name: "Tracker Name"
  state_topic: "your/bluetooth/devices/topic"
  value_template: >
  {% set data = value.split('\n')
  | select('contains', 'YOUR_DEVICE_NAME')
  | first | default(';;', true) %}
  {{ 'home' if data.split(';')[2] | int(-99) > -50 else 'not_home' }}

[technyon]

Why don't you just use another app that allows you to send the name?

[swann05]

Cause it's an other app using battery and space on my phone. Most of apps need to have a permanent notification. I like to keep it simple and invisible.
I've seen on github that HA companion not displaying a name is problem for many application. Maybe it will be change one day.
I will use espresense or beacon if it works better.

[swann05]

I've tried a Tile beacon. It doesn't advertise a name either, so doesn't work with presence detection on Nuki_hub.

[swann05]

Hello, I'm sorry to bother you again with that problem.
On HA, they add the same issue with their iBeacon tracker which doesn't catch no named devices.
They solved it by setting an exception on minor in address with a exotic number (40004 for them). So iBeacon tracker now catch no named devices with 40004 minor in address.
Is that an easy fix in the case of nuki hub?

[technyon]

Can you point to me to where you read that?

[swann05]

home-assistant/core#104419

[technyon]

Check if this binary works. Beacons will be added if they match that minor ID, which should be the case for the HA companion app as far as I understand.

nuki_hub-8.29-pre-1.zip

[swann05]

Sorry but it's not working.
Actually, I think i can't verify if the code for ibeacon tracker works in HA cause it's not pushed yet.

[technyon]

Do you know if they updated the android app already?

[swann05]

Yes but only in beta which i have

[swann05]

I don't think it matters too much because the only change is that 40004 minor value is set by default now (to match the future change in ibeacon tracker integration)

[technyon]

Well that is the important part. The binary above should check for that minor ID, and if found it should be added to presence detection.

[swann05]

I understand that but even in the not-updated app we can already set a custom UUID, major and minor.

[technyon]

ok I see

[swann05]

I don't know anything about code but it seems you have a "scanner" and then you build the mqtt message from address , name and rssi. Maybe it's not showing because it has no name to show ?

[technyon]

Correct, everything without a name is filtered out, this is intentional. It could be made configurable though, but you'll detect a lot of devices eventually. And there's a limit to how many devices can be published, due to limitations of using a microcontroller, so eventually if you live in an urban environment, you won't see your device because it's omitted because too many devices are detected.

[swann05]

It's exactly the same problem with iBeacon tracker in HA. So they set a exception on minor 40004 to allow no named devices to be discovered.
I don't know what modification you have done in the bin you sent me.
I'm not in a urban environment. I only have 5 named devices discovered in the list.

[swann05]

I'm curious. How do you open your door? You have a beacon with a name?

[technyon]

I'm using a Gigaset G-Tag, which I have on my keychain. Better than using an app I think, because it'll drain your phones battery if it's constantly broadcasting beacons.

[swann05]

Honestly, the HA app doesn't drain my battery at all.

[technyon]

@swann05 Please recheck with this binary, it works for me. It seems the special minor id is not beta anymore.

nuki_hub-8.29-pre-2.zip

[swann05]

It's working !
But as it has no name, I can only see " address ; ; RSSI " with a blank. It's hard to identify. Maybe you can set something like "if no name, set uuid instead" ?
Thank you very much for the work

[technyon]

That sure can be done.

nuki_hub-8.29-pre-3.zip

[swann05]

It's all good ! Thank you very much !