Support for PostgreSQL

[vishalkuo]

Following a conversation with @RJPercival on the trillian slack channel, I think it'd be helpful to add support for a postgres backed data store. I'd like to use this issue for further discussion/tracking progress.

I'll likely borrow heavily from https://github.com/Martin2112/trillian/tree/pgsql/storage/pgsql as a starting point

[pav-kv]

What's the status of this work?

@vishalkuo Is the postgres Trillian storage used anywhere?

[vishalkuo]

Apologies - I have no had time to look into this at all. It looks like @bigjimnolan has also started some work on this. It might be worth following up on his PRs.

[pav-kv]

I see only one commit from @bigjimnolan here. Jim, could you comment whether this is actually being used / tested / developed?

[pav-kv]

Could you also clarify whether you use the trillian_log_server and trillian_log_signer binaries that we provide in this repository, or you have your own main.go files?

Just to let you know, the PR #1919 disables postgres storage provider in main.go files of trillian_log_server and trillian_log_signer by default given the development status, until we know that it needs to be enabled. Feel free to submit the corresponding "enabling" PR, or ask me/team to do that.

[vishalkuo]

Got it. I believe I do use the provided binaries. Apologies for the delay on this and I appreciate the patience/follow up

[pav-kv]

Closing this for now. Feel free to reopen if you actively work on PostgreSQL support.

[marcopeereboom]

Boo!

Am I the only one waiting for postgres support? In my experience maria doesn't quite hack it under load.

Would you guys merge a PR if it magically appeared?

[Martin2112]

Hi,

It's not that we're opposed to Postgres per se. Our concerns are around the costs of ongoing maintenance, support and CI / build infrastructure, which will multiply by the number of supported storage types.

We did look at Postgres when we started the project but could not find an equivalent to Galera in terms of replication characteristics and the tests with what was available at the time were not very encouraging.

However, it should be possible to use a Postgres storage implementation hosted outside of our repository. It will require the main,go files for the binaries to be forked and a storage provider to be written to wrap up the flags but that should be more or less it. If there are obstacles to doing this then we'd probably be willing to look at making smaller changes to support it.

[aPaleBlueDot]

Hi @Martin2112, Trillian newbie here. A Postgres compatible and scalable database out there is CockroachDB / CockroachCloud.

[Martin2112]

Yes we looked at that when the project started but it wasn't stable enough to be used in production. Maybe it's moved on now but the issue of us having limited resources for maintenance still exists.

[robstradling]

PostgreSQL support has been added to Trillian in #3644.