Refactor badger storage dependency parsing for better performance

[burmanm]

Which problem is this PR solving?

Currently getting the dependencies window can take a lot of time if there are significant amount of active spans in the badger.

With 1.6 million active spans this took 10 seconds to return on my machine. After this patch, it takes 1.1s on my machine.

Short description of the changes

• Adds new index for dependency writing (one extra write - but inside same transaction)
• Does not decode the protobufs or touch the values in badger and instead uses key only parsing to find these results.
• Does not store a lot of stuff in the memory, instead it parses them while getting them from badger
• Adds new key to the storage which indicates the used version
• Adds schema migration from version 0 to version 1

[codecov]

Codecov Report

Merging #1694 into master will decrease coverage by 0.1%.
The diff coverage is 93.33%.

@@            Coverage Diff             @@
##           master    #1694      +/-   ##
==========================================
- Coverage   98.36%   98.26%   -0.11%     
==========================================
  Files         193      194       +1     
  Lines        9364     9495     +131     
==========================================
+ Hits         9211     9330     +119     
- Misses        119      125       +6     
- Partials       34       40       +6

Impacted Files	Coverage Δ
plugin/storage/badger/spanstore/writer.go	97.59% <100%> (+0.36%)	⬆️
plugin/storage/badger/dependencystore/storage.go	87.5% <100%> (-7.24%)	⬇️
plugin/storage/badger/spanstore/reader.go	96.89% <100%> (+0.25%)	⬆️
plugin/storage/badger/factory.go	98.14% <100%> (+0.05%)	⬆️
plugin/storage/badger/spanstore/schema_manager.go	86.58% <86.58%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2724a58...88aeff0. Read the comment docs.

[burmanm]

Seems Travis is failing to github timeouts today..

[jpkrohling]

This latest failure is actually a real one:

Go fmt, license check, or import ordering failures, run 'make fmt'

[burmanm]

make fmt gives me nothing related to this PR:

[michael@nina jaeger]$ git status -s
 M examples/hotrod/services/frontend/gen_assets.go
 M jaeger-ui
[michael@nina jaeger]$ make fmt
./scripts/import-order-cleanup.sh inplace
Running go fmt...
./scripts/updateLicenses.sh
[michael@nina jaeger]$ git status -s
 M examples/hotrod/services/frontend/gen_assets.go
 M jaeger-ui
[michael@nina jaeger]

Nor make test-ci ?

[burmanm]

Lets see if rebase fixes Travis

[burmanm]

codecov never ceases to amaze me. I'm getting coverage diff rejection because of change in processor.go. Yet, this PR doesn't even have that file.

[yurishkuro]

I see most uncovered code coming from storage.go: https://codecov.io/gh/jaegertracing/jaeger/compare/e5f9953251b50643c5c4479cc8941395c75faa76...5f361bd82dbe0f1e18db9f4819ce365510e8d741/diff

[jpkrohling]

Closing due to inactivity.