Configurable Update Interval

[LukeHandle]

After reading #15 , I saw you mention the sensors update every second (!)

For those using the recorder, this will generate a lot of data there.

I suggest allowing a configurable update frequency on this? Maybe:

• 1 second
• 15 second
• 1 minute
• 2 minutes
  (or just a "seconds" configurable option?)

For reference, in 2.5 hours (with 2 interfaces and 6 devices) that was 10174 recorded states (85%). It logs the same sort of stuff into the events table as well.

MariaDB [homeassistant]> select NOW();
+---------------------+
| NOW()               |
+---------------------+
| 2020-04-16 17:43:55 |
+---------------------+
1 row in set (0.000 sec)

MariaDB [homeassistant]> SELECT COUNT(last_updated) FROM states WHERE last_updated > "2020-04-16 15:08:55";
+---------------------+
| COUNT(last_updated) |
+---------------------+
|               11793 |
+---------------------+
1 row in set (0.075 sec)

MariaDB [homeassistant]> SELECT COUNT(last_updated) FROM states WHERE last_updated > "2020-04-16 15:08:55" AND entity_id LIKE "%edgeos%";
+---------------------+
| COUNT(last_updated) |
+---------------------+
|               10174 |
+---------------------+
1 row in set (0.191 sec)

MariaDB [homeassistant]> SELECT DISTINCT entity_id FROM states WHERE last_updated > "2020-04-16 15:08:55" AND entity_id LIKE "%edgeos%";
+-------------------------------------+
| entity_id                           |
+-------------------------------------+
| binary_sensor.edgeos_system_status  |
| sensor.edgeos_system_uptime         |
| sensor.edgeos_unknown_devices       |
| binary_sensor.edgeos_interface_eth0 |
| binary_sensor.edgeos_interface_eth1 |
| binary_sensor.edgeos_device_name1   |
| binary_sensor.edgeos_device_name2   |
| binary_sensor.edgeos_device_name3   |
| binary_sensor.edgeos_device_name4   |
| binary_sensor.edgeos_device_name5   |
| binary_sensor.edgeos_device_name6   |
+-------------------------------------+
11 rows in set (0.283 sec)

I understand part of the data is the transfer rates, but I doubt many need such granularity? (they should probably export it into Prometheus or something if they wanted proper tracking of it).

edit: It might also be worth documenting this will put a lot of data into the recorder. Maybe mention that users can also exclude the entities from the recorder

[elad-bar]

Part of what i'm doing with those components is to monitor traffic of devices / interfaces in Grafana,
In recorder I set the purging period to 7 days while in InfluxDB there is no purging defined,
the size of the DB in MariaDB is ~4GB (I have ~90 domains / integrations with ~850 components).

I will add the comment as you suggested and will add option to change the update interval from UI with default of 1 second.

thanks for the feedback

[LukeHandle]

Interesting. I haven't looked yet at pulling proper long terms stats from my Unifi or EdgeOS, but I lightly planned to dump them straight into the TSDB (likely Prometheus). While I'll certainly pull data from HA as well, I'll only use it for the IOT bits.

My HA is running on a Pi. I was trying to keep the IO limited there to preserve the SD card life as I'm aware that's likely where I'll see issues in the future. I'm much less concerned with the data volume/storage used, more so with the amount written.

I haven't got round to excluding any data in the recorder, and per SELECT COUNT(DISTINCT entity_id) FROM states WHERE last_updated > "2020-04-16 15:08:55"; I have ~138 entities reporting in that timeframe. I keep 30 days worth though (no idea why).

The IO change is certainly apparent:

$ sar -d

09:05:02          DEV       tps     rkB/s     wkB/s   areq-sz    aqu-sz     await     svctm     %util
09:15:01     dev179-0      5.52      0.05    121.71     22.05      0.05     40.32      5.59      3.08
09:25:01     dev179-0      5.15      0.00    111.36     21.60      0.08     48.65      9.10      4.69
09:35:01     dev179-0      5.83      0.21    126.64     21.75      0.05     22.81      4.89      2.85
09:45:01     dev179-0      6.26      0.03    135.85     21.72      0.06     36.53      6.05      3.79
09:55:01     dev179-0      5.37      0.05    116.31     21.67      0.05     29.04      5.59      3.00
10:05:01     dev179-0      5.85      0.03    127.89     21.88      0.06     34.36      6.59      3.85
10:15:02     dev179-0      5.68      0.03    123.85     21.82      0.05     35.94      4.98      2.83
10:25:01     dev179-0      5.19      0.21    109.83     21.20      0.06     58.40      6.28      3.26
10:35:01     dev179-0      5.35      0.00    117.33     21.93      0.05     31.54      6.37      3.41
10:45:01     dev179-0      5.36      0.02    119.15     22.24      0.04     19.29      5.91      3.17
10:55:01     dev179-0      4.86      0.07    109.32     22.49      0.04     33.57      5.91      2.87


18:15:01          DEV       tps     rkB/s     wkB/s   areq-sz    aqu-sz     await     svctm     %util
18:25:01     dev179-0     34.37     14.43    852.38     25.22      0.36     33.10      6.43     22.09
18:35:01     dev179-0     34.73      9.68    858.74     25.00      0.35     32.58      6.36     22.08
18:45:01     dev179-0     35.22      5.88    861.12     24.62      0.33     29.50      5.96     20.98
18:55:01     dev179-0     36.61      5.41    898.32     24.68      0.35     28.51      6.06     22.19
19:05:01     dev179-0     35.80      3.13    890.93     24.97      0.35     27.68      6.24     22.35
19:15:02     dev179-0     35.18      2.45    874.02     24.91      0.34     31.79      6.12     21.55
19:25:01     dev179-0     36.51      3.64    890.95     24.50      0.36     29.28      6.06     22.14
19:35:01     dev179-0     35.27      0.97    859.85     24.41      0.38     37.48      6.63     23.37
19:45:01     dev179-0     36.76      6.17    874.94     23.97      0.35     32.33      6.10     22.44
19:55:01     dev179-0     34.51      0.32    848.32     24.59      0.32     27.48      5.97     20.61
20:05:01     dev179-0     34.52      0.50    852.07     24.70      0.35     28.87      6.58     22.73

Thanks for your quick responses!

[elad-bar]

can you please check if now it works for you better?

[LukeHandle]

It didn't appear to be labelled in the UI - but changing this to 60 seems to have changed the frequency of the update 👍

THANKS, much appreciated!

[elad-bar]

@LukeHandle thanks for the feedback, fixed it