current package layout hurts compile time and binary sizes

[mvdan]

Since all the DB libraries are imported from a single package, using a single one of them requires building all of them. And since each is registered globally, they must be bundled in the final binary too.

This will get worse as more DB implementations are added (#38). I made a write-up about this, and potential solutions, on ChainSafe/chaindb#2 since that module has a similar issue.

Similarly, I think a database implementation called foo should live at github.com/tendermint/tm-db/foo. Then, to use it, one would need to call a constructor like foo.New(...) instead of db.NewDB("foo", ...). This way, because the user only imports the database implementations they need, Go only builds and links the libraries as needed, not all of them at once.

[erikgrinaker]

We're currently using build tags for this, at least for a couple of the backends (see e.g. BoltDB and RocksDB). Would this work for BadgerDB as well?

[mvdan]

Hm, I guess it works, but then every project depending on one of these databases must specify the build tag every single time they build their project.

In comparison, the "import as needed" approach doesn't require specifying build tags every time.

[erikgrinaker]

Hm, I guess it works, but then every project depending on one of these databases must specify the build tag every single time they build their project.

In comparison, the "import as needed" approach doesn't require specifying build tags every time.

Sure, but that's sort of by design - in e.g. Tendermint the database backend is configurable at runtime, and so build tags are a mechanism for controlling which backends should be available in the Tendermint binary.

[mvdan]

You could still implement that if you wanted, though - it would probably be fifty or so lines on top of the "import as needed" mechanism.

My point of view is someone who uses tendermint libraries, so I want to be able to use tm-db without having to use build tags for my normal build or having to build five DB backends when I just need one.

[erikgrinaker]

That's true, we could have the backends as independent subpackages and use build-tags in the root package to control which ones to include in a "full" build - although it might get a bit confusing. What do you think @melekes?

[mvdan]

You'll probably need a separate package from the root, though, because if the interface is also at the root package you would have import cycles. You could have a new package at /full or /metadb, which would import /foo, which would import the root package for the interface and shared code.

[erikgrinaker]

Makes sense, metadb or something similar sounds good. Would you be able to submit a PR for this?

[mvdan]

Sure, I can send a PR after my badger one.