Weird error - EZSP_BACKUP #1
Comments
|
First issue, first pun of my name ;-) . Don't worry, nobody notices. I have no ezsp so I made an attempt at the implementation, so yes I expect it needs adjustments. I probably located that reason, you can update and test again. Edit : The title says "znp_backup: Backup ZNP network data" where ZNP suggests the dongle type. But this can always be repeated and clarified what that means. |
|
Okay, that looks like an improvement. The log now says: And I have a backup file. However, I suspect that it doesn't contain everything. Here's a screenshot of the zigbee network visualisation: {
"node_type": 1,
"node_id": 0,
"node_ieee": "00:0d:6f:00:0c:86:91:ba",
"panId": 57059,
"extendedPanId": "a0:7e:d0:9d:89:0c:c8:19",
"radioChannel": 11,
"radioTxPower": 255,
"nwkUpdateId": 0,
"channels": 134215680,
"tc_link_key": {
"bitmask": 26,
"type": 1,
"key": [
90,
105,
103,
66,
101,
101,
65,
108,
108,
105,
97,
110,
99,
101,
48,
57
],
"outgoingFrameCounter": 57344,
"incomingFrameCounter": 0,
"sequenceNumber": 0,
"partnerEUI64": "00:0d:6f:00:0c:86:91:ba"
},
"network_key": {
"bitmask": 3,
"type": 3,
"key": [
20,
198,
161,
199,
203,
224,
195,
24,
142,
124,
54,
48,
220,
173,
165,
150
],
"outgoingFrameCounter": 3827760,
"incomingFrameCounter": 0,
"sequenceNumber": 0,
"partnerEUI64": "00:00:00:00:00:00:00:00"
},
"key_table": []
}I think the backup should have more data? |
|
It's encouraging but the output does not look like the "open-coordinator-backup" format. It's encouraging because you were able to make the backup live, without having to specify the port or doing anything special. I dupped the code from https://github.com/zigpy/bellows/blob/dev/bellows/cli/backup.py#L85 and I now looked at the video mentioned here where we can notice the following output: So I wonder: is ZHA in zigpy not containing this code or should I use something else as a reference? The backup file should at least contain some devices, but only those that are known/managed by the coordinator - there is nothing in this file. And it should (eventually) record this in the open-coordinator-backup format. So maybe somebody could find out where the good reference code is. |
|
I backed up a bit in the video and apparently it requires another version of bellows to export it this way: So I can't do that in the custom_component, but at least there is a start. |
|
Strange, it actually needs an older version of bellows? Otherwise I'd say we should wait until the bellows version of ZHA is updated, but now I'm not sure what to do. I guess I have to go for the manual way, as demonstrated in the video... |
Exactly which Zigbee Coordinator adapter brand + model (including exact chip it uses) and which exact firmware does it have? Silicon Labs (a.k.a. Silabs) have a catalogue of Zigbee chips going back 20-years and some still ship very old firmware today. For reference, could you try to stop the ZHA integration and test performing a backup to file using bellows CLI directly? See: https://github.com/zigpy/zigpy/wiki/Coordinator-Backup-and-Migration https://github.com/zigpy/bellows/blob/dev/README.md#nvram-backup-and-restore https://www.youtube.com/watch?v=ql58ZfXso1M&ab_channel=EverythingSmartHome https://gist.github.com/EverythingSmartHome/990f2436608fd347cd20cf0009b5cef4 As mentioned, the goal is for zha-toolkit to produce a backup files compatible with Open ZigBee Coordinator Backup Format: https://github.com/zigpy/open-coordinator-backup Please see this sample file as an example how a such standard backup file should look like: https://github.com/zigpy/open-coordinator-backup/blob/main/samples/z2m-sample-1.json |
|
I think it's a development version. I did dig enough to find the code that corresponds to it - possibly it's all in the CLI part of the code and worst case that could be copied over to zha-toolkit until it is release in HA. |
|
@Hedda It's not a matter of stopping ZHA, but having the code generating the open format - that's apparently the case for the ZNP tools currently in HA, but not for the bellows tools. I haven't use all my means to find the bellows code in which 'zigpy/open-coordinator-backup' appears as we have it in the next lines for zigpy-znp: In zigpy-znp the CLI code seems to suffice on its own. |
git grep open-coordinator $(git rev-list --all)did not find anything in the bellows repository, but found several occurrences in the zigpy-znp repository. |
|
The code is in puddly's fork: https://github.com/puddly/bellows/blob/puddly/open-coordinator-backup/bellows/cli/backup.py . I update zha-toolkit with that code - it's likely compatible with HA's version. Needs to be tested. |
|
Let me know when it's ready, and I will happily run a test backup with it. |
|
You can try it - I've updated the code. |
|
Apparently a merge messed up services.yaml - hopefully that did not affect you. |
@puddly Do you think a better solution here would be to move to zha-toolkit using zigpy-cli with this PR? -> zigpy/zigpy-cli#2 That PR, of course, is dependent on all radio libraries implementing that new radio api for zigpy:
|
|
I'm now back at the original error: |
|
Sorry, I cleaned up the repository from metadata and that explains why it was reverted to an old state. |
|
@bartkummel Hourly backups are good for something ;-). It should have the update now. |
|
@Hedda @puddly - I think that more generally, the backup functions would ideally be part of the library. The CLI would not be implementing any logic other than providing the interface between these library functions and the command line. ZHA could evolve to provide the backup service amongst its default services.
It's ok for me if |
|
|
Ok, I added ezsp_backup.py (again). |
|
Next: (I think this is the essential info you need. Let me know if I need to copy-paste more from the logs.) |
|
Yes, that's enough. A new update is available; |
|
|
For the update the 'alias' t had to be the bellow types. That's updated. |
|
I get the same error again... :-( |
|
I get "Exception: 'ezsp_backup' is only available for BELLOWS/EZSP" ;-). When bypassing that limitation, and explicitally using t.EmberStatus.SUCCESS, there is no such complaint. I think the issue is due to the fact that I did not reload the backup module and your probably did not restart HA (which I prefer we avoid), so you might have been running the old (already loaded) code for the backup function. I added an extra line which should add "WARNING (MainThread) [custom_components.zha_toolkit.ezsp_backup] INFO! t.EmberStatus.SUCCESS=EmberStatus.SUCCESS" to the log, and I also reload the backup module. It's available for testing. |
The open coordinator backup branch of bellows is unmaintained, I wouldn't recommend relying on its correctness. All new development is happening in the "new radio API" branches, tracked by the zigpy-cli PR. Once zigpy-zigate is finished, zigpy will be able to do these backups natively. Why are frequent backups are even necessary? I ran hourly backups for many months on my own network and nothing ever changed other than the frame counter. Realistically, you should be fine taking a single backup, running your network for 24 hours, taking another one, and estimating your counter increment rate from that. Then just restore a backup later but increment the frame counter by 10x that rate, multiplied by the number of days since the last backup. |
|
Right, I do have a backup file now again. However, the contents are still not what I'd expect. Here's the full log: And here's the contents of my backup file: {
"metadata": {
"version": 1,
"format": "zigpy/open-coordinator-backup",
"source": "[email protected]",
"internal": {
"creation_time": "2022-01-21T09:16:25+01:00"
}
},
"coordinator_ieee": "000d6f000c8691ba",
"pan_id": "dee3",
"extended_pan_id": "a07ed09d890cc819",
"nwk_update_id": 0,
"security_level": 5,
"channel": 11,
"channel_mask": [
11,
12,
13,
14,
15,
16,
17,
18,
19,
20,
21,
22,
23,
24,
25,
26
],
"network_key": {
"key": "14c6a1c7cbe0c3188e7c3630dcada596",
"sequence_number": 0,
"frame_counter": 3845964
},
"devices": []
}The devices array is still empty... |
|
Looks pretty good. I suspect there is a format mismatch between the "addresses" list and the "keys" list - so I added a debug message with the following format: "Got key at index %s status: %s key_struct: %s" . |
|
I am having strange behavior with regards to my commits. I am checking that out and will let you know when it's ok. |
|
I worked out why the checks were failing, so you can test the export again so check the logs for the new information. |
|
This is my log output: Doesn't look different compared to the earlier one... |
|
The backup I restored from yesterday did not have the change to reload the backup library, the transcript confirms that it's not the current version (I removed one of the messages and it still appears). I re-applied the library reload - sorry for this extra step. |
|
@puddly FYI, this is indirectly related to this feature request until the point that ZHA developers implement as a built-in feature: My guess is that most ZHA users should want either daily/nightly, weekly, or monthly backup as a standard feature/function that is always enabled by default. As in one backup saved into a separate file and/or directory every 24-hours or once per week or month (or a combination of those) and rotate a selected number of backups so that it does not ever keep an archive of more X-numbers of backup at any one time. Normally backup systems support incremental backups and in those cases it is only necessary to save a full backup once per month (or even just verify/update that full backup if it already exists) then only perform incremental backups in between then. However when it is not possible to save incremental backups then one common best practice backup schema combination if is to keep one backup for one or two months back as well as keep three daily/nightly backups. That way you can always have the possibility and option to restore either one month back or 1-3 days back in time. Regardless, users should always want backups saved to file for disaster recovery purposes even if they don't see the reason why when no disaster has ever occurred for them before. PS: I believe that this is also part what Home Assistant founders indirectly talked about in their latest blog-post about "Streamlining Experiences": https://www.home-assistant.io/blog/2022/01/19/streamlining-experiences/ "With Home Assistant the last few years we’ve been focusing on making things easier, stable, and faster. More things can be managed via the UI, most YAML-based integrations can be reloaded without restarting and if something breaks, safe mode and built-in backups have your back." |
|
Ok, we no have the following:
So there are 4 entries in the table, and all for are indicated as "Erased". Only key entries for which the result is "EmberStatus.SUCCESS" are added to the list of devices. So while there are apparently 7 NWK nodes, they do not appear in the "KeyTable". I can't tell if this is normal for your situation, nor what this evolved to. I do notice that the "ezsp_get_keys" command also uses the getKeyTableEntry method from bellows. I suppose that that command will give the same "empty" result in your case. So we've gone as far as we can for now. I've added some extra output to ezsp_backup to see if there is a "children" attribute of the radio instance, and if there is any usefull information in there. In my case (znp), it's not filled with that information yet in my up-to-date HA version. |
|
OK, thanks. I guess I'm going to rebuild my Zigbee network manually with my new coordinator then. |
|
I am curious if you really need to rebuild it given that the key information stays the same an that you can update your TX counter on restore. |
|
Well, the key information is missing from the backup, isn't it? So If I put a new coordinator in, it won't know of the devices that the current coordinator does. |
|
The list of child devices is not crucial, most child devices will likely wake up while the coordinator is offline for the migration and switch to a new parent anyways. If you're trying to do a migration, use the CLI utility instead. It contains more recently updated code for EZSP and ZNP radios. |
|
@bartkummel I mainly meant 'key' information: network key, tx counter, ieee address of the router (which is kindof a key). @puddly Thanks for confirming that the child information is likely not needed. |
|
I updated to 0.5.0 and then the backup stopped working again. I needed to go on, so decided to manually re-add all my devices. Thanks for the effort, I hope this functionality will mature and end up being part of ZHA, including a nice UI for it. |
|
@bartkummel I had added some information regarding properties of the radio instance - so I remove those for the next users. There have been other code changes to comply with code checks that I added to the automatic release actions, there could be some new mistakes, especially when I can't test them. Good luck with our migration - you did post a file that you could try to restore from. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hi,
I tried to create a backup of my EZSP stick, but I got an error. Here's a snippet from my
home-assistant.log:Apart from the fact that the error is not very descriptive for an end user, I also suspect that somethings going wrong. I'm calling the EZSP backup procedure, so, I don't expect something with
znpin it. Can this be a bug? If not: what can I try to fix it?The text was updated successfully, but these errors were encountered: