Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

added/updated postgresql implementation of log_storage #1571

Merged
merged 36 commits into from
May 21, 2019
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
7abee92
added/updated postgresql implementation of log_storage
bigjimnolan May 9, 2019
c9ac881
reverted some test code to make commit compatible with integration te…
bigjimnolan May 9, 2019
af2cb5a
Typo fix in db code
bigjimnolan May 9, 2019
d12bba1
udpated queue batching code to work for postgres
bigjimnolan May 9, 2019
b79e24e
updated code using gofmt and goimport
bigjimnolan May 9, 2019
86a2de4
more format fixing
bigjimnolan May 9, 2019
93301be
Made updates referenced in pull request. Added test coverage for log…
bigjimnolan May 9, 2019
733e86f
Updated test code to explicitly declare funtion for AddLeaf call
bigjimnolan May 9, 2019
ed19758
Updated test code and fixed methodology for pushing sql code into dat…
bigjimnolan May 9, 2019
55b8240
Updated log storage and added some more testing
bigjimnolan May 10, 2019
7509c43
Trying to fix this odd error with compact.VisitFn
bigjimnolan May 10, 2019
628a6a3
ignoring test for now
bigjimnolan May 10, 2019
5e21ca2
used goimports
bigjimnolan May 10, 2019
b3d4fcd
updated for lint checks
bigjimnolan May 10, 2019
44b41c4
fixing goimport random gripe
bigjimnolan May 10, 2019
bde1d03
must be a different copy of goimports on the server
bigjimnolan May 10, 2019
b3767ad
lint fixes?
bigjimnolan May 10, 2019
a028c8b
Seeing if this is the issue with goimports
bigjimnolan May 10, 2019
d80b158
will this pass the lint check?
bigjimnolan May 10, 2019
8c98f37
will this pass the lint check?
bigjimnolan May 10, 2019
a424960
will this pass the lint check 3/?
bigjimnolan May 10, 2019
620357d
removed some more commented code; added some more error checking
bigjimnolan May 13, 2019
bab9462
removed some more old code
bigjimnolan May 13, 2019
33a71fc
updating storage_test
bigjimnolan May 13, 2019
a23c318
manual update to master
bigjimnolan May 13, 2019
1004e83
Added warning about experimental nature of PostgreSQL support
bigjimnolan May 14, 2019
a750242
Update storage/postgres/README.md
bigjimnolan May 17, 2019
1acdaa0
Update storage/postgres/README.md
bigjimnolan May 17, 2019
59837d6
Update storage/postgres/storage.sql
bigjimnolan May 17, 2019
16137cd
Update storage/postgres/storage_test.go
bigjimnolan May 17, 2019
e70d3ea
Update storage/postgres/storage_test.go
bigjimnolan May 17, 2019
7bfb409
Apply suggestions from code review
bigjimnolan May 17, 2019
e5807c5
Made updates based on pull request comments
bigjimnolan May 17, 2019
714e7fb
updated readme.txt
bigjimnolan May 20, 2019
96f9796
Merge branch 'master' of https://github.com/bigjimnolan/trillian
bigjimnolan May 20, 2019
3ff3939
udpated readme to restart build
bigjimnolan May 21, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion server/postgres_storage_provider.go
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ func newPGProvider(mf monitoring.MetricFactory) (StorageProvider, error) {
}

func (s *pgProvider) LogStorage() storage.LogStorage {
panic("Not Implemented")
return postgres.NewLogStorage(s.db, s.mf) //panic("Not Implemented")
bigjimnolan marked this conversation as resolved.
Show resolved Hide resolved
}

func (s *pgProvider) MapStorage() storage.MapStorage {
Expand Down
17 changes: 17 additions & 0 deletions storage/postgres/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# Postgres LogStorage

## Notes and Caveats
The current LogStorage part of the Postgres implementation was based off what
was alredy written for MySQL. Thus, the two user-defined functions included in
bigjimnolan marked this conversation as resolved.
Show resolved Hide resolved
bigjimnolan marked this conversation as resolved.
Show resolved Hide resolved
storage.sql. MySQL doesn't kill a transaction when a duplicate is detected, but
PostgreSQL does. So, to preserve the workflow, I included the two functions
which trap this error and allow the code to continue executing. The only other
change I made was to fully translate the MySQL queries to PostgreSQL compatible ones
and tidy up some of the extant tree storage code.

storage_unsafe.sql really isn't unsafe, but I have pulled some of the safety
rails from the tables to improve performance. It also works under the notion that
bigjimnolan marked this conversation as resolved.
Show resolved Hide resolved
there will only be a single tree in a given database. An improvement on this theme
would be to add all layers below the trees table in their own separate schemas.
This would further eliminate indexs and foreign key requirements, but it should
be left for those who require enhanced performance.
Loading