Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add-db-upgrade-from-3.x--3.4 #1937

Merged
merged 3 commits into from
Feb 7, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,108 @@
# Upgrade NebulaGraph Enterprise Edition from version 3.x to 3.4.0

This topic takes the enterprise edition of NebulaGraph v3.1.0 as an example and describes how to upgrade to v3.4.0.

## Notes

- This upgrade is only applicable for upgrading the enterprise edition of NebulaGraph v3.x to v3.4.0. <!--Because in the Enterprise Edition of NebulaGraph 3.4, one partition corresponds to one RocksDB instance, which is different from one graph space corresponding to one RocksDB instance in versions before 3.4.--> If your version is below 3.0.0, please upgrade to enterprise edition 3.x before upgrading to v3.4.0. For details, see [Upgrade NebulaGraph Enterprise Edition 2.x to 3.1.0](https://docs.nebula-graph.io/3.1.0/4.deployment-and-installation/3.upgrade-nebula-graph/upgrade-nebula-graph-to-latest/).

- The IP address of the machine performing the upgrade operation must be the same as the original machine.

- The remaining disk space on the machine must be at least 1.5 times the size of the original data directory.


## Steps

1. [Contact us](https://www.nebula-graph.io/contact) to obtain the installation package of the enterprise edition of NebulaGraph v3.4.0 and install it.

!!! note

The upgrade steps are the same for different installation packages. This article uses the RPM package and the installation directory `/usr/local/nebulagraph-ent-3.4` as an example. See [Install with RPM packages](../2.compile-and-install-nebula-graph/2.install-nebula-graph-by-rpm-or-deb.md) for specific operations.

!!! caution

Please ensure that the number of storage paths set for the `--data_path` parameter in the Meta and Storage service configuration files of the 3.4.0 cluster is the same as that for the `--data_path` parameter in the configuration files of the 3.x cluster. Otherwise, the upgraded cluster will not start.

2. Stop the enterprise edition of v3.x services. For details see [Manage NebulaGraph services](../../2.quick-start/3.quick-start-on-premise/5.start-stop-service.md).

Run the `nebula.service status all` command to confirm that all services have been stopped after running the command.

3. In the installation directory of the Enterprise Edition NebulaGraph v3.4.0, run the following commands to upgrade the Storage and Meta services.

- Upgrade the Storage service:

Syntax:

```bash
sudo ./bin/db_upgrader --max_concurrent_parts=<num> --src_db_path=<source_storage_data_path> --dst_db_path=<destination_storage_data_path>
```

| Parameter | Description |
| :-------------- | :--------------------------- |
| `--max_concurrent_parts` | Specify the number of partitions to upgrade simultaneously, with the default value being 1.<br/>It is recommended to increase the value appropriately based on disk performance. |
| `--src_db_path` | Specify the absolute path to the source data directory. The following takes the source data directory `/usr/local/nebula-ent-3.1.0/data/storage` as an example. |
| `--dst_db_path` | Specify the absolute path to the target data directory. The example target data directory is `/usr/local/nebula-ent-3.4/data/storage`.|

Example:

```bash
sudo ./bin/db_upgrader --max_concurrent_parts=20 --src_db_path=/usr/local/nebula-ent-3.1.0/data/storage --dst_db_path=/usr/local/nebula-ent-3.4/data/storage
```

If there are multiple source data directories, specify each source data directory and target data directory and run the corresponding command. For example, there are two source data directories `/usr/local/nebula-ent-3.1.0/data/storage` and `/usr/local/nebula-ent-3.1.0/data2/storage`, run the following commands:

```bash
sudo ./bin/db_upgrader --src_db_path=/usr/local/nebula-ent-3.1.0/data/storage --dst_db_path=/usr/local/nebula-ent-3.4/data/storage

sudo ./bin/db_upgrader --src_db_path=/usr/local/nebula-ent-3.1.0/data2/storage --dst_db_path=/usr/local/nebula-ent-3.4/data2/storage
```

- Upgrade the Meta service:

Syntax:

```bash
sudo ./bin/meta_upgrader --src_meta_path=<source_meta_data_path> --dst_meta_path=<destination_meta_data_path>
```

| Parameter | Description |
| :-------------- | :--------------------------- |
| `--src_meta_path` | Specify the absolute path to the source meta data directory. The following takes the source data directory `/usr/local/nebula-ent-3.1.0/data/meta` as an example. |
| `--dst_meta_path` | Specify the absolute path to the target meta data directory. The example target data directory is `/usr/local/nebula-ent-3.4/data/meta`.|

Example:

```bash
sudo ./bin/meta_upgrader --src_meta_path=/usr/local/nebula-ent-3.1.0/data/meta --dst_meta_path=/usr/local/nebula-ent-3.4/data/meta
```

If there are multiple source meta data directories, specify each source meta data directory and target meta data directory and run the corresponding command.

After the upgrade, a `data` directory will be generated in the v3.4.0 installation directory, containing the upgraded data files.

4. Upload the license file to the `share/resources` directory under the v3.4.0 installation directory.

5. Start and connect to the NebulaGraph v3.4.0 enterprise edition service and verify that the data is correct. The following commands can be used as reference:

```
nebula> SHOW HOSTS;
nebula> SHOW HOSTS storage;
nebula> SHOW SPACES;
nebula> USE <space_name>
nebula> SHOW PARTS;
nebula> SUBMIT JOB STATS;
nebula> SHOW STATS;
nebula> MATCH (v) RETURN v LIMIT 5;
```

## Docker Compose Deployment

!!! caution

For NebulaGraph deployed using Docker Compose, it is recommended to redeploy the new version and import data.






Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ To upgrade NebulaGraph from historical versions to {{nebula.release}}:

* You must upgrade the old NebulaGraph services on the same machines they are deployed. **DO NOT** change the IP addresses, configuration files of the machines, and **DO NOT** change the cluster topology.

* The hard disk space left on each machine should be **two times** as much as the space taken by the original data directories. Half of the reserved space is for storing the manual backup of data. The other half is for storing the WALs that will be copied to the `dst_db_path` and the new keys supporting vertices with no tags.
<!-- * The hard disk space left on each machine should be **two times** as much as the space taken by the original data directories. Half of the reserved space is for storing the manual backup of data. The other half is for storing the WALs that will be copied to the `dst_db_path` and the new keys supporting vertices with no tags. -->

* Known issues that could cause data loss are listed on [GitHub known issues](https://github.com/vesoft-inc/nebula-graph/issues/857). The issues are all related to altering schema or default values.

Expand All @@ -35,15 +35,15 @@ To upgrade NebulaGraph from historical versions to {{nebula.release}}:

## Upgrade influences

- Data swelling
<!-- - Data swelling

The NebulaGraph 3.x version expands the original data format with one more key per vertex, so the data takes up more space after the upgrade.

The format of the new key is:

Type (1 byte) + Partition ID (3 bytes) + VID (size depends on the data type).

The value of the new key is empty. The extra space taken can be calculated based on the number of vertices and the data type of the VID. For example, if there are 100 million vertices in the dataset and the VIDs are INT64, the new key will take 100 million x (1 + 3 + 8) = 1.2 billion bytes, i.e., about 1.2 GB.
The value of the new key is empty. The extra space taken can be calculated based on the number of vertices and the data type of the VID. For example, if there are 100 million vertices in the dataset and the VIDs are INT64, the new key will take 100 million x (1 + 3 + 8) = 1.2 billion bytes, i.e., about 1.2 GB. -->

- Client compatibility

Expand Down
6 changes: 4 additions & 2 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -455,8 +455,10 @@ nav:
- Manage Storage host: 4.deployment-and-installation/manage-storage-host.md
# - Manage zone: 4.deployment-and-installation/5.zone.md
- Upgrade:
- Upgrade NebulaGraph to the latest version: 4.deployment-and-installation/3.upgrade-nebula-graph/upgrade-nebula-graph-to-latest.md
- Upgrade NebulaGraph from v3.x to the current version: 4.deployment-and-installation/3.upgrade-nebula-graph/upgrade-nebula-from-300-to-latest.md
- Upgrade NebulaGraph from v2.x to v3.4 (Community Edition): 4.deployment-and-installation/3.upgrade-nebula-graph/upgrade-nebula-graph-to-latest.md
- Upgrade NebulaGraph from v3.x to v3.4 (Community Edition): 4.deployment-and-installation/3.upgrade-nebula-graph/upgrade-nebula-from-300-to-latest.md
#ent
- Upgrade NebulaGraph from v3.x to v3.4 (Enterprise Edition): 4.deployment-and-installation/3.upgrade-nebula-graph/upgrade-nebula-ent-from-3.x-3.4.md
- Uninstall NebulaGraph: 4.deployment-and-installation/4.uninstall-nebula-graph.md

- Configurations and logs:
Expand Down