release oak functions without squash #53
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jul-sh
force-pushed
the
main
branch
28 times, most recently
from
3029665 to
7d7be69
Compare
jul-sh
force-pushed
the
main
branch
2 times, most recently
from
3dca7f2 to
4e6e8bc
Compare
1. not everything needs to be `mut` 2. we don't really need the `byteorder` crate 3. I found a couple more places where we unnecessarily format strings for logs. Change-Id: Ic66bd447bfc190718ab8a4687e5799138ba98e21
These tests make sure that if we specify any single wrong reference value for Oak Containers the attestation verification will fail. BUG: 325509208 Change-Id: I7c1e5932685ca993141cd03f3d3a40a00c973a98
For now we can use this CI to debug provenance publication more easily. Later on we may want to make it available for simple applications that use Oak. b/334900893 Change-Id: I6b6f9cd7703b36fdb17552335c87c58b33be1125
Different builds of these binaries at the same revision produce different results for (at least) the following reasons: * orchestrator: debug symbols are not always identical; this seems to be more than just an ordering issue. Stripping debug symbols addresses this. * syslogd: the RUNPATH ELF section contains paths in a non-deterministic order. Since all of the paths won't exist in the target environment, they can be removed. This is consistent with the orchestrator, which has an empty rpath. * both: when not built from the Oak Docker image, file paths end up in the binary in various places. Trimming them (via `trim-paths`) removes the system-dependent part of the paths and improves reproducibility. This isn't strictly necessary, but it's an easy fix. Bug: 332535604 Change-Id: I49dbff0549dcf1d07309aa7bd9f98e1622fc87a1
These tests make sure that if we specify any single wrong reference value for a restricted kernel application the attestation verification will fail. BUG: 325509208 Change-Id: I97fbf2da15e8978d2bb29c21b9cd057a21c4c5a9
…system_image. Change-Id: I8766e2b1d80a7995a186fe7d08351f2ec65e0b00
b/334900893 Change-Id: I7cb74027312014f6367877f786d8e0b4984405cb
Adds a simple implementation for now (literally just the display string), so other work can proceed. Will be improved later. Long term I'm a little bit skeptical creating manual descriptions for each reference values, as they risk becoming comments describing the functioning of the verification library that may become inaccurate. Eg see http://b/337920594 as an example of how behavior is unclear from reference values alone. The better approach may be to implement a slightly prettier version of the std::fmt::Display for the relevant structs (eg one that omits fields set to None) and rely on the technical expertise of the auditor to make sense of these printed values. Bug: 334900893 Change-Id: I67e9ed63b9e3501c8baa52ad743adf3655389e1a
The hope here is that this will reduce allocator pressure. One chunk may contain ~60K entries, so the allocator has to do a fair amount of work for the prost deserializer (allocate a vector for the entries, allocate memory for all the fiels inside entries, fill them, ...) only for us to copy data to the hash map and then release the memory fairly quickly. This will let us deserialize one message at a time, hopefully relying more on the heap. The downside is that we'll likely hold on to the mutex longer; but we already have discussed about splitting that mutex up as well. The new implementation is also completely backwards compatible; the `oneof` doesn't change how data is serialized. Preliminary tests suggest that it is indeed consistently about ~20% faster, but that's on my workstation with a limited data set size. I'll need to do some more large-scale testing, but the first results are promising. Bug: 337854886 Change-Id: Ie3a1e2a996ec9cbc851e144db88aacf58e376de9
…t call patchelf twice. The patchelf call is not idempotent, the binary changes with each subsequent call (gpaste/6116921312804864). Bug: 332535604 Change-Id: I314188476c7ca5a5049c5d40d52f94ea4750a734
…n library tests Bug: 335457019 Change-Id: I75ae17efeafa193c10b52ebfb6998d1cf261fab7
Bug: 337258843 Change-Id: I4c906de534de14ceee41585dfa7b3c9cb594ff9b
jul-sh
force-pushed
the
main
branch
3 times, most recently
from
887e9fd to
29b1c29
Compare
The goal here is to allow for more concurrency when inserting data by breaking up the single `Mutex` that guards the `DataBuilder` by sharding it S ways, each with its own `Mutex`. The method chosen here is intentionally not particulary clever and just distributes entries among the 16 hashmaps based on their hash. This has the benefit of making lookups fairly simple, at the cost of us not being able to fully exploit knowledge we have about our data layout (e.g. how many shards we have in input and we know the input is sorted). Downside is obviously one extra layer of indirection to go through when doing lookups. The constant of 16 has been chosen somewhat arbitrarily: I think it should correlate well with how many CPUs we have available. In any case, bumping it up to 128 made performance far worse. Testing with our 100G dataset suggests that if S = 1, it'll take 40-45 minutes to upload all data; with S = 16, it'll take 19-25 minutes. Bug: 337062283 Change-Id: I55ac133f2587df93c73b41748738252078eb0131
Bug: 338559296 Change-Id: I25ac385da673f85b294f48d41608a3a89fdf570e
Change-Id: Idb2254dff2d3affe4e846830c266052103fa486b
During the oak_proto_rust build script, the generated proto files also get copied to a known location inside of the source tree. We can automatically import these files using copybara. A CI check is added to make sure that the generated files are included in the expected location. This removes the need to manually generate/copy files when proto changes are made. Change-Id: I7351d5348b3fec22824faf26a8d99eabdbbf52ae
bug: 335457019 Change-Id: I2e2397f620adf3b5e90789173c75612c4768f3ec
…ux_boot_params crate for bare metal. BUG: 339182449 Change-Id: Id4ec7500deb0fadf4aaaab86650d103b37939d70
Also, build for bare metal during CI. BUG: 339182533,339183160,339406711 Change-Id: I1e8fd1d3dfe7847a5362db808d632fcd26e9889f
…ith Bazel. BUG: 339182429 Change-Id: Ic3c686ca1d85047f4679ca573c6b70fc198ab728
Change-Id: I17f46187683689d7c390b4fe154ad970a45ce0bf
Make `oak_proto_rust` a dependency of shell script: * Ensures that the generated files are generated when running the `sh_binary` targets. * Ensures that any changes to it get picked up. Tested: 1) bazel clean 2) bazel run `oak_proto_rust:verify_generated` 3) Observe succcess message. 4) Add a new proto to build.rs list (and deps) 5) bazel run `oak_proto_rust:verify_generated` 6) Observe message that a `copy_generated` run is needed. Change-Id: I01cacab5092383ddf7cec1f7279d519c8d3e7b9a
…sformations. This will be followed by a CL to remove the copybara transformations that create these build targets. BUG: 329220699 Change-Id: I81eaa4b1b320eccb8dd19a03a7cf8aa44525e916
jul-sh
force-pushed
the
main
branch
2 times, most recently
from
186172e to
fb9f9c7
Compare
Change-Id: Ia76292ae941358c559df4b63e5941489c567157a
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.