Using Source Control for Backup

[sshaikh]

Is your feature request related to a problem? Please describe

I'm trying to figure out the best way to source control my configuration. My first try was to keep configuration.yaml in a git tree, but that left secret.yaml in the application git tree. The answer was to relocate the whole data directory, but this brings with it other files which may or may not be appropriate to keep in source control.

Describe the solution you'd like

A few things would help here:

1. Split configuration and working data into two locations.
2. Format all files for efficient diff'ing (I think only database.db has this problem where chunks of json appear as single lines).
3. Documentation that describes the various files so that we can determine whether they need backing up or not.
4. Restrcuture files so that static data is kept separately from volatile (recreatable?) data.

Describe alternatives you've considered

At the moment I am .gitignoring most things other than configuration.yaml, but I presume this isn't enough to restore after an loss event.

Additional context

I understand that config management isn't a backup, but Z2M does a few things already which I feel puts it in reach of this goal. I guess the first step is trying to understand what in data is recreatable/disposable, and what is real "config".

[sjorge]

#4806 might interest you I think

[sshaikh]

Thanks! I actually originally raised #4697 which that was spun out of. While still great, it doesn't actually help with this as the data folder will still be the same.

[bhopkinson]

I'm looking at doing the same as @sshaikh. Initially I'm going for a .gitignore that looks like the following

# Ignore all files and folders
/*
# Except
!.gitignore
!database.db
!*.yaml
# But definately secret.yaml!
secret.yaml

The only other file in the data folder that I'm not really sure about is coordinator_backup.json. I'm thinking it doesn't need including in the repo? Without it, is that enough to restore from backup?

[Koenkk]

@bhopkinson coordinator_backup.json is required to restore (otherwise you will need to re-pair everything if you have a fresh stick)

[bhopkinson]

@bhopkinson coordinator_backup.json is required to restore (otherwise you will need to re-pair everything if you have a fresh stick)

That's great - thank you 👍