Deduplication configuration nt shown in volume menu

[WAdama]

Hi,

it seems, the script has set everything correct, but I get not the menu point shown:

Output from syno_enable_dedupe.sh -c:
`Synology_enable_Deduplication v1.4.28
DS420+ DSM 7.2.1-69057-5

StorageManager 1.0.0-0017

Using options: -c
Running from: /volume1/Work/Scripts/SH/syno_enable_dedupe.sh
WARNING Don't store this script on an NVMe volume!

NAS has 10 GB of memory.

Btrfs Data Deduplication is not enabled.

Tiny Btrfs Data Deduplication is enabled.

Checking non-Synology drive supported.
File is already edited.

Checking value in backup file.
Backup file is okay.
`
But I see in Volume global settings the entry for setting automatic deduplication, which of course says, that there is no deduplication activated for a volume.

[007revad]

Did you close and re-open Storage Manager.

If yes, then I suspect the SSD cache is preventing the deduplication menu from appearing because deduplication is only officially supported on SSD volumes, and an SSD volume would not have an SSD cache.

[WAdama]

Hi Dave,

I restarted the system before this. That would be enough, wouldn't it?

Have I misunderstood something here? I have a system with hard drives and ssd's as cache. Is this not supported then?

[007revad]

Yes, as you restarted the NAS you do not need to close and re-open storage manager.

I have a system with hard drives and ssd's as cache. Is this not supported then?

I think yes.

One way to be sure would be to:

1. Remove the cache from the volume.
2. Reboot.
3. Check if deduplication is now available.

I could maybe allow running deduplication on a volume with a cache but I don't know if your data would be safe.

[WAdama]

Hi Dave,

will check this, may need some time.

I can't remember any hint from Synology that deduplication with cache is not possible.

[007revad]

No need to do those 3 steps. I just confirmed the cache is NOT the problem.

I added a read-only SSD cache to a HDD volume and was able set deduplication.

Then I deleted the read-only cache and created a read/write cache and was able set deduplication.

Read/write cache:

Read only cache:

[007revad]

Have you previously run https://github.com/007revad/Synology_HDD_db

[WAdama]

Hi Dave,

thought that was just optional, will try that...

Regards
Ingo

[WAdama]

Hi Dave,

just tested it again with Synology_HDD_db, but I'm afraid, that made no difference.

Still only this part:

Output from -c:
`Synology_enable_Deduplication v1.4.28
DS420+ DSM 7.2.1-69057-5

StorageManager 1.0.0-0017

Using options: -c
Running from: /volume1/Work/Scripts/SH/syno_enable_dedupe.sh
WARNING Don't store this script on an NVMe volume!

NAS has 10 GB of memory.

Btrfs Data Deduplication is not enabled.

Tiny Btrfs Data Deduplication is enabled.

Checking non-Synology drive supported.
File is already edited.

Checking value in backup file.
Backup file is okay.
`

[007revad]

I just remembered it can take up to 10 minutes before the deduplication menu appears. Can you check if the menu is still missing.

Another thought is that my Synology NAS have either a NVMe volume or a SATA SSD volume. When you have time can you try:

• Just the HDD volume and no SSD cache.
• The HDD volume and a SSD volume.

Make sure to check after 10 minutes.

[sheckandar]

I had the same issue.

We have our synology configured in a HA cluster and when I ran the script on one of them only, nothing showed up in the Storage Manager. However, after I ran the script on both devices and rebooted both one by one, the option showed up the Storage Manager.

[WAdama]

Hi Dave,

sorry for answering so late. Even waiting more than 10 minutes and several restarts does not do the trick.

I have to admit, this feature is not that important for me to reconfigure my working NAS so massive.

I had hoped for a simplier reason for this error.

By the way, this is usable on an already configured and used file system? Not that that's the problem.

[007revad]

By the way, this is usable on an already configured and used file system?

Yes.

I just remembered that deduplication requires:

1. Data deduplication is only supported on Btrfs volumes. (which you have).
2. Data deduplication can only run when the volume status is Healthy. (which yours is).
3. Data deduplication requires you to Enable usage detail analysis for the Btrfs volume.

If you don't have "Usage detail analysis" enabled:

1. Go to Storage Manager > Storage.
2. Click the upper-right icon of the Btrfs volume you want to configure.
3. Select Settings.
4. Go to the Usage Details section and tick the Enable usage detail analysis checkbox.
5. Click Save to confirm.

[WAdama]

Hi Dave,

all three prerequisites are fulfilled, so yes, usage detail analysis is active..

The full configuration of my volume:
RAID5 (no SHR) with 4 hdd's
Btrfs
Read/write ssd cache

Regards
Ingo

[MaximumFish]

Btrfs Data Deduplication is not enabled

A stupid question maybe, but are you definitely running the script with the --hdd option?

[007revad]

A stupid question maybe, but are you definitely running the script with the --hdd option?

Well spotted!

I just noticed the -c or --check option on it's own does not check if dedupe on hdds is enabled or not.

[007revad]

I've just released v1.4.29

• Changed to show state of "Dedupe config menu for HDDs and 2.5 inch SSDs" even if --hdd option not used.
  • Now also shows "Run the script with the --hdd option if you want it enabled".

https://github.com/007revad/Synology_enable_Deduplication/releases

[WAdama]

Hi Dave,

you're right, I started it without --hdd

Reason in your description: Enable data deduplication for HDDs (dangerous)

I thought tiny would be enough...

What is tiny then for?

[007revad]

I wrote dangerous because:

• Synology must have a reason for only allowing deduplication on SSDs.
• I don't want to be responsible if people run deduplication on HDDs and something goes wrong.

There's enough people who do run deduplication on HDDs without any reported issues that I should remove that (dangerous) comment.

What is tiny then for?

DSM can do 2 different types of deduplication:

• "btrfs data deduplication" which requires 16GB of memory or more.
• "tiny btrfs data deduplication" which requires 4GB of memory or more.

If your Synology has more than 3GB of memory but less than 16GB the script enables "tiny btrfs data deduplication".

If your Synology has 16GB or more memory the script enables "btrfs data deduplication".

What is the difference between tiny and regular deduplication? Apart from the required amount of memory I honestly don't know. I assume tiny deduplication takes longer.

I should run some tests and time them.

EDIT The --tiny option only exists in case someone has a Synology with 16GB or more memory but wants to use tiny deduplication instead of regular deduplication. Say if they have 16GB of memory but some of it is allocated to VMs and regular deduplication slows down the NAS too much.

[MaximumFish]

I think it's worth keeping some sort of warning there to distance yourself from any issues that may arise from an unsupported configuration. Maybe instead of "dangerous", change it to something along the lines of "HDD dedupe is a hack that is not supported by Synology. No known stability issues but use at your own risk".

Edit: Or just "unsupported - use at your own risk" if you want to keep it short and sweet. 😄

[007revad]

Edit: Or just "unsupported - use at your own risk" if you want to keep it short and sweet. 😄

I had just removed (dangerous) minutes before your reply.

Then I saw your reply and changed it to "HDD data deduplication is not supported by Synology. No known stability issues but use at your own risk".

And now you've edited your reply 😄

[WAdama]

Hi Dave,

did use --hdd and now the missing point is shown. As my DS420+ have 10GB RAM, your script enabled tiny dedup.

Just running a dedup analysis...