v1.4.0

v1.4.0

• Recommended go version for development: 1.17
  This is the version used by the cloudbuild presubmits. Using a different version can lead to presubmits failing due to unexpected diffs.
• GCP terraform script updated. GKE 1.19 and updated CPU type to E2

Dependency updates

Many dep updates, including:

• Upgraded to etcd v3 in order to allow grpc to be upgraded (#2195)
• etcd was v0.5.0-alpha.5, now v3.5.0
• grpc upgraded from v1.29.1 to v1.40.0
• certificate-transparency-go from v1.0.21 to v1.1.2-0.20210512142713-bed466244fa6
• protobuf upgraded from v1 to v2
• MySQL driver from 1.5.0 to 1.6.0

Cleanup

• Removed signatures from LogRoot and EntryTimestamps returned by RPCs (reflecting that there should not be a trust boundary between Trillian and the personality.)
• Removed the deprecated crypto.NewSHA256Signer function.
• Finish removing the LogMetadata.GetUnsequencedCounts() method.
• Removed the following APIs:
  • TrillianLog.GetLeavesByHash
  • TrillianLog.GetLeavesByIndex
  • TrillianLog.QueueLeaves
• Removed the incomplete Postgres storage backend (#1298).
• Deprecated LogRootV1.Revision field.
• Moved rfc6962 hasher one directory up to eliminate empty leftover package.
• Removed unused log_client tool.
• Various tidyups and improvements to merke & proof generation code.
• Remove some remnants of experimental map.

Storage refactoring

• NodeReader.GetMerkleNodes does not accept revisions anymore. The implementations must use the transaction's ReadRevision
  instead.
• TreeStorage migrated to using compact.NodeID type suitable for logs.
• Removed the tree storage ReadRevision and WriteRevision methods.
  Revisions are now an implementation detail of the current storages. The change allows log implementations which don't need revisions.
• Removed Rollback methods from storage interfaces, as Close is enough to cover the use-case.
• Removed the unused IsOpen and IsClosed methods from transaction interfaces.
• Removed the ReadOnlyLogTX interface, and put its only used GetActiveLogIDs method to LogStorage.
• Inlined the LogMetadata interface to ReadOnlyLogStorage.
• Inlined the TreeStorage interfaces to LogStorage.
• Removed the need for the storage layer to return ephemeral node hashes. The application layer always requests for complete subtree nodes comprising the compact ranges corresponding to the requests.
• Removed the single-tile callback from SubtreeCache, it uses only GetSubtreesFunc now.
• Removed SetSubtreesFunc callback from SubtreeCache. The tiles should be written by the caller now, i.e. the caller must invoke the callback.

v1.3.13

Cleanup

• Removed the experimental map API.

v1.3.12

Misc improvements

• Removed unused PeekTokens method from the quota.Manager interface.
• Ensure goroutines never block in the subtree cache (#2272).
• Breaking unnecessary dependencies for Trillian clients:
  • Moved verifiers from merkle into merkle/{log,map}verifiersub-pacakges,
    reducing the amount of extra baggage inadvertently pulled in by clients.
• Concrete hashers have been moved into subpackages, separating them from their
  registration code, allowing clients to directly pull just the hasher they're
  interested in and avoid the Trillian/hasher registry+protobuf deps.
• Moved some packages intended for internal-only use into internal packages:
  • InMemoryMerkleTree (indended to only be used by Trillian tests)
• Removed wrapper for etcd client (#2288).
• Moved --quota_system and --storage_system flags to main.go so that they
  are initialised properly. It might break depending builds relying on these
  flags. Suggested fix: add the flags to main.go.
• Made signer tolerate mastership election failures [#1150].
• testdb no longer accepts the --test_mysql_uri flag, and instead honours the
  TEST_MYSQL_URI ENV var. This makes it easier to blanket configure tests to use a
  specific test DB instance.
• Removed experimental Skylog folder (#2297).
• Fixed a race condition in the operation manager that should only affect tests
  (#2302).
• Run gofumpt formatter on the whole repository (#2315).
• Refactor signer operation loop (#2294).

Upgrades

• Dockerfiles are now based on Go 1.13 image.
• The etcd is now pinned to v3.4.12.
• The golangci-lint suite is now at v1.36.0.
• CI/CD has migrated from Travis to Google Cloud Build.
• prometheus from 1.7.1 to 1.9.0 (#2239, #2270).
• go-cmp from 0.5.2 to 0.5.4 (#2262).
• apache/beam from 2.26.0+incompatible to 2.27.0+incompatible (#2273).
• lib/pq from 1.8.0 to 1.9.0 (#2264).
• go-redis from 6.15.8+incompatible to 6.15.9+incompatible (#2215).

Process

• Recognise that we do not follow strict semantic versioning practices.

v1.3.11

Documentation

Added docs which describe the Claimant Model of transparency, a useful
framework for reasoning about the design and architecture of transparent
systems.

Misc improvements

• Fixed int to string conversion warnings for golang 1.15
• Metric improvements for fetched leaf counts
• Move tools.go into its own directory to help with dependencies

Dependency updates

• go-grpc-middleware from 1.2.0 to 1.2.2 (#2219, #2229)
• stackdriver from 0.13.2 to 0.13.4 (#2220, #2223)
• Google api from 0.28.0 to 0.29.0 (#2193)

Improvements to cloudspanner

New database-integration tests, #1740 have identified and help resolve inconsistencies and missing features between the various storage implementations.

Storage TX Interfaces

• QueueLeaves has been removed from the LogTreeTX interface because
  QueueLeaves is not transactionaal. All callers use the
  QueueLeaves function in the LogStorage interface.
• AddSequencedLeaves has been removed from the LogTreeTX for the same reason.

MySQL Dequeueing Change #2159

mysql will now remove leaves from the queue inside of UpdateLeaves rather than directly inside of Dequeue.
This change brings the behavior of the mysql storage implementation into line with the spanner implementation
and makes consistent testing possible.

Update Dependencies

Selected Dependency Updates

• etcd from v3.3.18 to 3.4.7 (#2090)
• etcd-operator from v0.9.1 to v0.9.4
• upgraded protoc version to latest (#2088)
• github.com/golang/protobuf to v1.4.1 (#2111)
• google.golang.org/grpc from v1.26 to 1.29.1 (#2108)

See all changes

v1.3.8

HTTP APIs

The HTTP/JSON APIs have been removed in favor of a pure gRPC intereface.
grpcurl is the recommended way
of interacting with the gRPC API from the commandline.

v1.3.7

Server Binaries

The trillian_log_server, trillian_log_signer and trillian_map_server
binaries have moved from github.com/google/trillian/server/ to
github.com/google/trillian/cmd. A subset of the server package has also
moved and has been split into cmd/internal/serverutil, quota/etcd and
quota/mysqlqm packages.

v1.3.6

Deployments

the kubernetes configs will now provision 5 nodes for trillian's etcd cluster,
instead of 3 nodes.
this makes the etcd cluster more resilient
to nodes becoming temporarily unavailable, such as during updates (it can now
tolerate 2 nodes being unavailable, instead of just 1).

Monitoring & Metrics

A count of the total number of individual leaves the logserver attempts to
fetch via the GetEntries.* API methods has been added.

v1.3.5

Log Changes

Potential sequencer hang fixed

A potential deadlock condition in the log sequencer when the process is
attempting to exit has been addressed.

Quota

New Features

An experimental Redis-based quota.Manager implementation has been added.

Behaviour Changes

Quota used to be refunded for all failed requests. For uses of quota that were
to protect against abuse or fair utilization, this could allow infinite QPS in
situations that really should have the requests throttled. Refunds are now only
performed for tokens in Global buckets, which prevents tokens being leaked if
duplicate leaves are queued.

Tools

The licenses tool has been moved from "scripts/licenses" to a dedicated
repository.

Bazel Changes

Python support is disabled unless we hear that the community cares about this
being re-enabled. This was broken by a downstream change and without a signal
from the Trillian community to say this is needed, the pragmatic action is to
not spend time investigating this issue.