[BLE] Simplify the publication of advertisement data

boards.ini

@@ -1383,7 +1383,7 @@ build_flags =
   '-DLED_SEND_RECEIVE=2'
   '-DLED_SEND_RECEIVE_ON=0'
   '-DpubBLEManufacturerData=true'
-  '-DpubKnownBLEServiceData=true'
+  '-DpubBLEAdvData=true'
   '-DpubBLEServiceUUID=true'
   '-DGateway_Name="OpenMQTTGateway_ESP32_BLE"'
 custom_description = Default BLE gateway with additional servicedata, manufacturerdata and service uuid for analysing decoding issues

docs/use/ble.md

@@ -124,7 +124,7 @@ If you want to change the number of BLE scans that are done before a BLE connect
 
 The BLE connect will be done every 30 * (`TimeBtwRead` + `Scan_duration`), 30 * (55000 + 10000) = 1950000ms
 
-## Setting if the gateway publishes all the BLE devices scanned or only the detected sensors
+## Setting if the gateway publishes all the BLE devices scanned or only the detected sensors (default: false)
 
 If you want to change this characteristic:
 
@@ -232,37 +232,24 @@ If you want to enable this feature:
 
 `mosquitto_pub -t home/OpenMQTTGateway/commands/MQTTtoBT/config -m '{"filterConnectable":true}'`
 
-## ADVANCED: Publishing known service data
+## ADVANCED: Publishing advertisement data (default: false)
 
 If you want to enable this feature:
 
-`mosquitto_pub -t home/OpenMQTTGateway/commands/MQTTtoBT/config -m '{"pubKnownServiceData":true}'`
+`mosquitto_pub -t home/OpenMQTTGateway/commands/MQTTtoBT/config -m '{"pubadvdata":true}'`
 
-## ADVANCED: Publishing unknown service data
-
-If you want to change the default behaviour, in case you are having too heavy service data:
-
-`mosquitto_pub -t home/OpenMQTTGateway/commands/MQTTtoBT/config -m '{"pubUnknownServiceData":false}'`
-
-## ADVANCED: Publishing known manufacturer's data
-
-If you want to change the default behaviour:
-
-`mosquitto_pub -t home/OpenMQTTGateway/commands/MQTTtoBT/config -m '{"pubKnownManufData":true}'`
-
-## ADVANCED: Publishing unknown manufacturer's data
-
-If you want to change the default behaviour, in case you are having too heavy service data:
-
-`mosquitto_pub -t home/OpenMQTTGateway/commands/MQTTtoBT/config -m '{"pubUnknownManufData":false}'`
+This will publish extensive information about the device:
+```json
+{"id":"11:22:33:44:55:66","mac_type":0,"adv_type":0,"name":"Qingping Motion & Light","rssi":-93,"servicedata":"88121122334455660201520f0126090403000000","servicedatauuid":"0xfdcd","brand":"Qingping","model":"Motion & Light","model_id":"CGPR1","lux":3,"batt":82}
+```
 
-## ADVANCED: Publishing the service UUID data
+## ADVANCED: Not publishing advertisement data
 
-If you want to change the default behaviour:
+To stop publishing advertisement data:
 
-`mosquitto_pub -t home/OpenMQTTGateway/commands/MQTTtoBT/config -m '{"pubServiceDataUUID":true}'`
+`mosquitto_pub -t home/OpenMQTTGateway/commands/MQTTtoBT/config -m '{"pubadvdata":false}'`
 
-## Store BLE configuration into the gateway (only with ESP32 boards)
+## Store BLE configuration into the gateway
 
 Open MQTT Gateway has the capability to save the current configuration and reload it at startup.
 
@@ -284,14 +271,14 @@ By the way, if you want to load the default built-in configuration (on any board
 `mosquitto_pub -t home/OpenMQTTGateway/commands/MQTTtoBT/config -m '{"init":true}'`
 Note that it will not change the stored configuration, `erase` or `save` is still needed to overwrite the saved configuration.
 
-## Read/write BLE characteristics over MQTT (ESP32 only)
+## Read/write BLE characteristics over MQTT
 
 The gateway can read and write BLE characteristics from devices and provide the results in an MQTT message.  
 ::: tip
 These actions will be taken on the next BLE connection, which occurs after scanning and after the scan count is reached, [see above to set this](#setting-the-number-of-scans-between-connection-attempts).
 This can be overridden by providing an (optional) parameter `"immediate": true` within the command. This will cause the BLE scan to stop if currently in progress, allowing the command to be immediately processed. All other connection commands in queue will also be processed for the same device, commands for other devices will be deferred until the next normally scheduled connection.
 
-**Note** Some devices need to have the MAC address type specified. You can find this type by checking the log/MQTT data and looking for "mac_type". By default the type is 0 but some devices use different type values. You must specify the correct type to connect successfully.  
+**Note** Some devices need to have the MAC address type specified. You can find this type by checking the log/MQTT data and looking for "mac_type". The mac_type of your device can be seen by setting `pubadvdata` to `true` with an MQTT command (see Publishing advertisement data), or with the macro `pubBLEAdvData true`. By default the type is 0 but some devices use different type values. You must specify the correct type to connect successfully.  
 To specify the MAC address type add the parameter `"mac_type"` to the command. For example `"mac_type": 1` to connect with a device with the MAC address type of 1.
 :::
 
@@ -341,7 +328,7 @@ The `ttl` parameter is the number of attempts to connect (defaults to 1), which
 `value_type` can be one of: STRING, HEX, INT, FLOAT. Default is STRING if omitted in the message.
 :::
 
-## SwitchBot Bot control (ESP32 only)
+## SwitchBot Bot control
 
 SwitchBot Bot devices are automatically discovered and available as a device in the configuration menu of home assistant.
 

main/ZgatewayBT.ino

@@ -92,11 +92,7 @@ void BTConfig_init() {
   BTConfig.extDecoderEnable = UseExtDecoder;
   BTConfig.extDecoderTopic = MQTTDecodeTopic;
   BTConfig.filterConnectable = BLE_FILTER_CONNECTABLE;
-  BTConfig.pubKnownServiceData = pubKnownBLEServiceData;
-  BTConfig.pubUnknownServiceData = pubUnknownBLEServiceData;
-  BTConfig.pubKnownManufData = pubBLEManufacturerData;
-  BTConfig.pubUnknownManufData = pubUnknownBLEManufacturerData;
-  BTConfig.pubServiceDataUUID = pubBLEServiceUUID;
+  BTConfig.pubAdvData = pubBLEAdvData;
   BTConfig.pubBeaconUuidForTopic = useBeaconUuidForTopic;
   BTConfig.ignoreWBlist = false;
 }
@@ -141,16 +137,8 @@ void BTConfig_fromJson(JsonObject& BTdata, bool startup = false) {
   BTConfig_update(BTdata, "extDecoderTopic", BTConfig.extDecoderTopic);
   // Sets whether to filter publishing
   BTConfig_update(BTdata, "filterConnectable", BTConfig.filterConnectable);
-  // Publish service data belonging to recognised sensors
-  BTConfig_update(BTdata, "pubKnownServiceData", BTConfig.pubKnownServiceData);
-  // Publish service data belonging to unrecognised sensors
-  BTConfig_update(BTdata, "pubUnknownServiceData", BTConfig.pubUnknownServiceData);
-  // Publish known manufacturer's data
-  BTConfig_update(BTdata, "pubKnownManufData", BTConfig.pubKnownManufData);
-  // Publish unknown manufacturer's data
-  BTConfig_update(BTdata, "pubUnknownManufData", BTConfig.pubUnknownManufData);
-  // Publish the service UUID data
-  BTConfig_update(BTdata, "pubServiceDataUUID", BTConfig.pubServiceDataUUID);
+  // Publish advertisment data
+  BTConfig_update(BTdata, "pubadvdata", BTConfig.pubAdvData);
   // Use iBeacon UUID as topic, instead of sender (random) MAC address
   BTConfig_update(BTdata, "pubBeaconUuidForTopic", BTConfig.pubBeaconUuidForTopic);
   // Disable Whitelist & Blacklist
@@ -170,11 +158,7 @@ void BTConfig_fromJson(JsonObject& BTdata, bool startup = false) {
   jo["extDecoderEnable"] = BTConfig.extDecoderEnable;
   jo["extDecoderTopic"] = BTConfig.extDecoderTopic;
   jo["filterConnectable"] = BTConfig.filterConnectable;
-  jo["pubKnownServiceData"] = BTConfig.pubKnownServiceData;
-  jo["pubUnknownServiceData"] = BTConfig.pubUnknownServiceData;
-  jo["pubKnownManufData"] = BTConfig.pubKnownManufData;
-  jo["pubUnknownManufData"] = BTConfig.pubUnknownManufData;
-  jo["pubServiceDataUUID"] = BTConfig.pubServiceDataUUID;
+  jo["pubadvdata"] = BTConfig.pubAdvData;
   jo["pubBeaconUuidForTopic"] = BTConfig.pubBeaconUuidForTopic;
   jo["ignoreWBlist"] = BTConfig.ignoreWBlist;
 
@@ -544,6 +528,7 @@ void procBLETask(void* pvParameters) {
       mac_adress.toUpperCase();
       BLEdata["id"] = (char*)mac_adress.c_str();
       BLEdata["mac_type"] = advertisedDevice->getAddress().getType();
+      BLEdata["adv_type"] = advertisedDevice->getAdvType();
       Log.notice(F("Device detected: %s" CR), (char*)mac_adress.c_str());
       BLEdevice* device = getDeviceByMac(BLEdata["id"].as<const char*>());
 
@@ -827,13 +812,6 @@ bool BTtoMQTT() { // for on demand BLE scans
   return true;
 }
 
-void RemoveJsonPropertyIf(JsonObject& obj, const char* key, bool condition) {
-  if (condition) {
-    Log.trace(F("Removing %s" CR), key);
-    obj.remove(key);
-  }
-}
-
 boolean valid_service_data(const char* data, int size) {
   for (int i = 0; i < size; ++i) {
     if (data[i] != 48) // 48 correspond to 0 in ASCII table
@@ -946,14 +924,15 @@ void launchBTDiscovery(bool overrideDiscovery) {
 void PublishDeviceData(JsonObject& BLEdata, bool processBLEData) {
   if (abs((int)BLEdata["rssi"] | 0) < abs(BTConfig.minRssi)) { // process only the devices close enough
     if (processBLEData) process_bledata(BLEdata);
-    if (!BTConfig.pubOnlySensors || BLEdata.containsKey("model") || BLEdata.containsKey("distance")) {
-      RemoveJsonPropertyIf(BLEdata, "servicedatauuid", !BTConfig.pubServiceDataUUID && BLEdata.containsKey("model"));
-      RemoveJsonPropertyIf(BLEdata, "servicedata", !BTConfig.pubKnownServiceData && BLEdata.containsKey("model"));
-      RemoveJsonPropertyIf(BLEdata, "manufacturerdata", !BTConfig.pubKnownManufData && BLEdata.containsKey("model"));
+    if (!BTConfig.pubAdvData) {
+      BLEdata.remove("servicedatauuid");
+      BLEdata.remove("servicedata");
+      BLEdata.remove("manufacturerdata");
+      BLEdata.remove("mac_type");
+      BLEdata.remove("adv_type");
+    }
+    if (!BTConfig.pubOnlySensors || BLEdata.containsKey("model") || BLEdata.containsKey("distance")) { // Identified device
       pubBT(BLEdata);
-    } else {
-      RemoveJsonPropertyIf(BLEdata, "servicedata", !BTConfig.pubUnknownServiceData);
-      RemoveJsonPropertyIf(BLEdata, "manufacturerdata", !BTConfig.pubUnknownManufData && BLEdata.containsKey("model"));
     }
   } else if (BLEdata.containsKey("distance")) {
     pubBT(BLEdata);

main/ZmqttDiscovery.ino

@@ -997,6 +997,16 @@ void pubMqttDiscovery() {
                   stateClassNone, //State Class
                   "false", "true" //state_off, state_on
   );
+  createDiscovery("switch", //set Type
+                  subjectBTtoMQTT, "BT: Publish Advertisement data", (char*)getUniqueId("pubadvdata", "").c_str(), //set state_topic,name,uniqueId
+                  "", "", "{{ value_json.pubadvdata }}", //set availability_topic,device_class,value_template,
+                  "{\"pubadvdata\":true,\"save\":true}", "{\"pubadvdata\":false,\"save\":true}", "", //set,payload_on,payload_off,unit_of_meas,
+                  0, //set  off_delay
+                  Gateway_AnnouncementMsg, will_Message, true, subjectMQTTtoBTset, //set,payload_available,payload_not available   ,is a gateway entity, command topic
+                  "", "", "", "", false, // device name, device manufacturer, device model, device MAC, retain
+                  stateClassNone, //State Class
+                  "false", "true" //state_off, state_on
+  );
   createDiscovery("switch", //set Type
                   subjectBTtoMQTT, "BT: Connect to devices", (char*)getUniqueId("bleconnect", "").c_str(), //set state_topic,name,uniqueId
                   "", "", "{{ value_json.bleconnect }}", //set availability_topic,device_class,value_template,

main/config_BT.h

@@ -112,24 +112,8 @@ extern int btQueueLengthCount;
 
 unsigned long scanCount = 0;
 
-#ifndef pubKnownBLEServiceData
-#  define pubKnownBLEServiceData false // define true if you want to publish service data belonging to recognised sensors
-#endif
-
-#ifndef pubUnknownBLEServiceData
-#  define pubUnknownBLEServiceData true // define false if you don't want to publish service data to unrecognised sensors (in case you are having too heavy service data) https://github.com/1technophile/OpenMQTTGateway/issues/318#issuecomment-446064707
-#endif
-
-#ifndef pubBLEManufacturerData
-#  define pubBLEManufacturerData false // define true if you want to publish the manufacturer's data (sometimes contains characters that aren't valid with receiving client)
-#endif
-
-#ifndef pubUnknownBLEManufacturerData
-#  define pubUnknownBLEManufacturerData true // define true if you want to publish the manufacturer's data (sometimes contains characters that aren't valid with receiving client)
-#endif
-
-#ifndef pubBLEServiceUUID
-#  define pubBLEServiceUUID false // define true if you want to publish the service UUID data
+#ifndef pubBLEAdvData
+#  define pubBLEAdvData false // define true if you want to publish all advertisement data
 #endif
 
 #ifndef useBeaconUuidForTopic
@@ -157,11 +141,7 @@ struct BTConfig_s {
   bool extDecoderEnable; // Send undecoded device data to another gateway device for decoding
   String extDecoderTopic; // Topic to send undecoded device data on
   bool filterConnectable; // Sets whether to filter publishing of scanned devices that require a connection.
-  bool pubKnownServiceData; // Publish service data belonging to recognised sensors
-  bool pubUnknownServiceData; // Publish service data belonging to unrecognised sensors (in case you are having too heavy service data) https://github.com/1technophile/OpenMQTTGateway/issues/318#issuecomment-446064707
-  bool pubKnownManufData; // Publish the manufacturer's data (sometimes contains characters that aren't valid with receiving client)
-  bool pubUnknownManufData; // Publish the manufacturer's data (sometimes contains characters that aren't valid with receiving client)
-  bool pubServiceDataUUID; // Publish the service UUID data
+  bool pubAdvData; // Publish advertisement data
   bool pubBeaconUuidForTopic; // Use iBeacon UUID as topic, instead of sender (random) MAC address
   bool ignoreWBlist; // Disable Whitelist & Blacklist
 };