feat: update to badger v4

[kruskall]

Motivation/summary

badger v2 is severely out of date and newer versions have important performance improvements

Checklist

• Update CHANGELOG.asciidoc
  - [ ] Update package changelog.yml (only if changes to apmpackage have been made)
  - [ ] Documentation has been updated

For functional changes, consider:

• Is it observable through the addition of either logging or metrics?
• Is its use being published in telemetry to enable product improvement?
• Have system tests been added to avoid regression?

How to test these changes

Related issues

[mergify]

This pull request does not have a backport label. Could you fix it @kruskall? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

• backport-7.x is the label to automatically backport to the 7.x branch.
• backport-7./d is the label to automatically backport to the 7./d branch. /d is the digit

NOTE: backport-skip has been added to this pull request.

[kruskall]

For reviewers: this will fail because the meaning of vlog files changed, see dgraph-io/badger#1555

[axw]

Thinking about this again, I'm not sure we should just remove the test. GC is still needed, it's just that how it works has changed. Can we instead update the test?

[axw]

Looks good!

IIUC this would be a breaking change as the data format has changed, so old files wouldn't be able to be loaded. Is that correct? If so then we should list it in the changelog. Also, have you confirmed the behaviour when upgrading from an older apm-server version?

[mergify]

This pull request is now in conflicts. Could you fix it @kruskall? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b feat/badgerv4 upstream/feat/badgerv4
git merge upstream/main
git push upstream feat/badgerv4

[kruskall]

This is what happened when trying to load a directory on the previous version:

panic: manifest has unsupported version: 7 (we support 8).
Please see https://dgraph.io/docs/badger/faq/#i-see-manifest-has-unsupported-version-x-we-support-y-error on how to fix this.

I don't think the migration procedure is feasible. Could we change the directory path ? e.g. tail_sampling -> tail_sampling_v4 ? WDYT ?

[mergify]

This pull request is now in conflicts. Could you fix it @kruskall? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b feat/badgerv4 upstream/feat/badgerv4
git merge upstream/main
git push upstream feat/badgerv4

[axw]

I don't think the migration procedure is feasible. Could we change the directory path ? e.g. tail_sampling -> tail_sampling_v4 ? WDYT ?

That sounds good. The alternative would be to automatically delete the old data, but I think it would be better to leave it there and give the user an option to manually backup/restore.

OTOH that would mean the old data will be left behind forever. Maybe we could check the modification time of files in the tail_sampling directory at startup (if it exists), and remove it if all files are older than the TTL. WDYT?

[kruskall]

Sorry for the late reply!

OTOH that would mean the old data will be left behind forever. Maybe we could check the modification time of files in the tail_sampling directory at startup (if it exists), and remove it if all files are older than the TTL. WDYT?

Good idea! Updated!

[axw]

Thanks for the updates @kruskall. Looks good overall, my main concern now is the removed test.

[axw]

We should mention why it's breaking - what the implications are for users.

[axw]

Thinking about this again, I'm not sure we should just remove the test. GC is still needed, it's just that how it works has changed. Can we instead update the test?

[kruskall]

I'll close this in the interest of time and open an issue that we can schedule. Sorry, I underestimated the amount of work needed for this 🙇