Skip to content

Commit

Permalink
feat(docs): backup review pIII
Browse files Browse the repository at this point in the history 
Co-authored-by: Klaas Freitag <[email protected]>
Signed-off-by: jkoberg <[email protected]>
  • Loading branch information
@kobergj @dragotin
kobergj and dragotin committed May 23, 2024
1 parent 21e0fd4 commit da99f26
Showing 1 changed file with 19 additions and 19 deletions.
38 changes: 19 additions & 19 deletions docs/ocis/backup.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,17 +7,17 @@ geekdocEditPath: edit/master/docs/ocis
geekdocFilePath: backup.md
---

This small guide aims to shed some light on the internal ocis data structure. You can refer to it when you are trying to optimize your backups or if you are just curious about how ocis stores its data.
This small guide aims to shed some light on the internal Infinite Scale data structure. You can refer to it when you are trying to optimize your backups or if you are just curious about how Infinite Scale stores its data.

Note, as a prerequisite backing up ocis, the instance has to be fully shut down for the time being.
Note, as a prerequisite backing up Infinite Scale, the instance has to be fully shut down for the time being.

## Ocis Data Structure

Ocis stores its data in a folder that can be configured via the envvar `OCIS_BASE_DATA_PATH`. Without further configuration, services derive from that path when they store data, though individual settings can be configured.
Ocis stores its data in a folder that can be configured via the environment variable `OCIS_BASE_DATA_PATH`. Without further configuration, services derive from that path when they store data, though individual settings for certain data types can be configured.

The default value for the `OCIS_BASE_DATA_PATH` variable is `$HOME/.ocis` (or `/var/lib/ocis` when using the docker container. Note: Configuration data is by default stored in `/etc/ocis/` in the container.).

Inside this folder, ocis will store all its data separated by subdirectories. That includes metadata, configurations, queues and stores etc. The actual bytes of files which are currently uploaded are also stored here by default. If an s3 store is used as blobstore, the blobs need to be backed up separately. Note: See special case for the `config` folder in a docker container.
Inside this folder, Infinite Scale will store all its data in separate subdirectories. That includes metadata, configurations, queues and stores etc. The actual bytes of files (blobs) are handled by a so called blobstore, which also stores here by default. Depending on the used blobstore, the blobs need to be backed up separately, for example if S3 is used. Note: See special case for the `config` folder in a docker container.

### Base Data Path Overview

Expand All @@ -42,7 +42,7 @@ The following sections describe the content and background of the subdirectories

### `config`

Contains basic ocis configuration created by `ocis init`(Note: The location of the configuration folder can be specified with the `OCIS_CONFIG_DIR` envvar but for this readme we will assume this envvar is not set and the default is used.)
Contains basic Infinite Scale configuration created by `ocis init`(Note: The location of the configuration folder can be specified with the `OCIS_CONFIG_DIR` environment variable, but for this document we will assume this variable is not set and the default is used.)

```bash
~/.ocis/config/:tree
Expand All @@ -53,7 +53,7 @@ Contains basic ocis configuration created by `ocis init`(Note: The location of t
```

* `ocis.yaml`:\
BACKUP RECOMMENDED. Holds ocis configuration data. The contents can vary depending on your environment variables. In general, most of this file can be recreated again by running `ocis init`. This will recreate secrets and certificates. However, if not backed up completely, some fields MUST be copied over from the old config manually to regain data access after a restore:
BACKUP RECOMMENDED. Holds Infinite Scale configuration data. The contents can vary depending on your environment variables. In general, most of this file can be recreated again by running `ocis init`. This will recreate secrets and certificates. However, if not backed up completely, some fields MUST be copied over from the old config manually to regain data access after a restore:

| Field Name | Envvar Name | Description | If not backed up |
| --- | --- | --- | --- |
Expand All @@ -71,7 +71,7 @@ BACKUP RECOMMENDED. Holds ocis configuration data. The contents can vary dependi

Note: This folder will not appear if you use an external idm. Refer to your idms documentation for backup details in this case.

Contains the data for the internal ocis identity management. See the IDM README for more details.
Contains the data for the internal Infinite Scale identity management. See the [IDM README]({{< ref "../idm/_index.md" >}}) for more details.

```bash
~/.ocis/idm/:tree
Expand All @@ -84,7 +84,7 @@ Contains the data for the internal ocis identity management. See the IDM README
```

* `ocis.boltdb`:\
BACKUP REQUIRED. This is the boltdb database that stores user data. Use `IDM_DATABASE_PATH` to specify its path. If not backed up, ocis will have no users, therefore also all data is lost.
BACKUP REQUIRED. This is the boltdb database that stores user data. Use `IDM_DATABASE_PATH` to specify its path. If not backed up, Infinite Scale will have no users, therefore also all data is lost.
* `ldap.crt`:\
BACKUP OPTIONAL. This is the certificate for the idm. Use `IDM_LDAPS_CERT` to specify its path. Will be auto-generated if not backed up.
* `ldap.key`:\
Expand All @@ -95,7 +95,7 @@ BACKUP OPTIONAL. This is the certificate key for the idm. Use `IDM_LDAPS_KEY` to

Note: This folder will not appear if you use an external idp. Refer to your idp's documentation for backup details in this case.

Contains the data for the internal ocis identity provider. See the IDP README for more details.
Contains the data for the internal Infinite Scale identity provider. See the [IDP README]({{< ref "../idp/_index.md" >}}) for more details.

```bash
~/.ocis/idp/:tree
Expand All @@ -113,13 +113,13 @@ BACKUP RECOMMENDED. This is the encryption secret. Use `IDP_ENCRYPTION_SECRET_FI
* `private-key.pem`:\
BACKUP RECOMMENDED. This is the encryption key. Use `IDP_SIGNING_PRIVATE_KEY_FILES` to specify its paths. Not backing this up will force users to relogin.
* `identifier-registration.yml`:\
BACKUP OPTIONAL. It holds temporary data of active sessions. Will be recreated if not backed up.
BACKUP OPTIONAL. It holds configuration for oidc clients (web, desktop, ios, android). Will be recreated if not backed up.

### `nats`

Note: This folder will not appear if you use an external nats installation
Note: This folder will not appear if you use an external nats installation. In that case, data has to secured in alignment with the external installation.

Contains nats data for streams and stores. See the NATS README for more details.
Contains nats data for streams and stores. See the [NATS README]({{< ref "../nats/_index.md" >}}) for more details.

```bash
~/.ocis/nats/:tree -L 1
Expand All @@ -129,11 +129,11 @@ Contains nats data for streams and stores. See the NATS README for more details.
```

* `jetstream`:\
BACKUP RECOMMENDED. This folder contains nats data about streams and key-value stores. Use `NATS_NATS_STORE_DIR` to specify its path. Not backing it up can break history for multiple (non-vital) features such as history or notifications. The ocis functionality is not impacted if omitted.
BACKUP RECOMMENDED. This folder contains nats data about streams and key-value stores. Use `NATS_NATS_STORE_DIR` to specify its path. Not backing it up can break history for multiple (non-vital) features such as history or notifications. The Infinite Scale functionality is not impacted if omitted.

### `proxy`

Contains proxy service data. See the PROXY README for more details.
Contains proxy service data. See the [PROXY README]({{< ref "../proxy/_index.md" >}}) for more details.

```bash
~/.ocis/proxy/:tree
Expand All @@ -151,7 +151,7 @@ BACKUP OPTIONAL. This is the certificate key for the http services. Use `PROXY_T

### `search`

Contains the search index.
Contains the search index. See the [SEARCH README]({{< ref "../search/_index.md" >}}) for more details.

```bash
~/.ocis/search/:tree -L 1
Expand All @@ -166,7 +166,7 @@ BACKUP RECOMMENDED/OPTIONAL. This contains the search index. Can be specified vi

### `storage`

Contains ocis meta (and blob) data.
Contains Infinite Scale meta (and blob) data, depending on the blobstore. See the [STORAGE-USERS README]({{< ref "../storage-users/_index.md" >}}) for more details.

```bash
~/.ocis/storage/:tree -L 1
Expand All @@ -183,11 +183,11 @@ BACKUP REQUIRED. Contains system data. Path can be specified via `STORAGE_SYSTEM
* `ocm`:\
BACKUP REQUIRED/OMITABLE. Contains ocm share data. When not using ocm sharing, this folder does not need to be backed up.
* `users`:\
BACKUP REQUIRED. Contains user data. Path can be specified via `STORAGE_USERS_OCIS_ROOT`. Not backing it up will remove all spaces and all files. As result, you will have a configured but empty ocis instance, which is fully functional accepting new data. Old data is lost.
BACKUP REQUIRED. Contains user data. Path can be specified via `STORAGE_USERS_OCIS_ROOT`. Not backing it up will remove all spaces and all files. As result, you will have a configured but empty Infinite Scale instance, which is fully functional accepting new data. Old data is lost.

### `thumbnails`

Contains thumbnails data.
Contains thumbnails data. See the [THUMBNAILS README]({{< ref "../thumbnails/_index.md" >}}) for more details.

```bash
~/.ocis/thumbnails/:tree -L 1
Expand All @@ -200,7 +200,7 @@ OPTIONAL/RECOMMENDED. This folder contains prerendered thumbnails. Can be specif

### `web`

Contains web assets such as custom logos, themes etc.
Contains web assets such as custom logos, themes etc. See the [WEB README]({{< ref "../web/_index.md" >}}) for more details.

```bash
~/.ocis/web/:tree -L 1
Expand Down

0 comments on commit da99f26

Please sign in to comment.