cli: debug commands to deserialize descriptors and job payloads #52063
Assignees
Labels
C-bug
Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior.
Comments
|
Hi @ajwerner, I've guessed the C-ategory of your issue and suitably labeled it. Please re-label if inaccurate. While you're here, please consider adding an A- label to help keep our repository tidy. 🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is otan. |
blathers-crl
bot
added
the
C-bug
|
cc @spaskob |
spaskob
pushed a commit
to spaskob/cockroach
that referenced
this issue
Aug 19, 2020
We currently export the descriptors and job payloads in debug zips with hex encoding. It is often very valuable to decode these protos into JSON for inspection. Fixes cockroachdb#52063. Release note (cli change): The new debug command `decode-proto` reads descriptor from stdin in hex or base64 format (auto-detected) and a flag --name=<fully qualified name to decode> with default value `cockroach.sql.sqlbase.Descriptor` and outputs to stdout the deserialized proto in JSON format.
spaskob
pushed a commit
to spaskob/cockroach
that referenced
this issue
Aug 24, 2020
We currently export the descriptors and job payloads in debug zips with hex encoding. It is often very valuable to decode these protos into JSON for inspection. Fixes cockroachdb#52063. Release note (cli change): The new debug command `decode-proto` reads descriptor from stdin in hex or base64 format (auto-detected) and a flag --schema=<fully qualified name to decode> with default value `cockroach.sql.sqlbase.Descriptor` and outputs to stdout the deserialized proto in JSON format. If the input is not a hex/base64 encoded proto then it is outputted verbatim.
craig bot
pushed a commit
that referenced
this issue
Aug 24, 2020
52972: cli: command to deserialize descriptors r=spaskob a=spaskob We currently export the descriptors and job payloads in debug zips with hex encoding. It is often very valuable to decode these protos into JSON for inspection. Fixes #52063. Release note (cli change): The new debug command `decode-proto` reads descriptor from stdin in hex or base64 format (auto-detected) and a flag --schema=\<fully qualified name to decode\> with default value `cockroach.sql.sqlbase.Descriptor` and outputs to stdout the deserialized proto in JSON format. 53226: colexec: make hash table more dynamic and fix hash join in some cases r=yuzefovich a=yuzefovich **colexec: make hash table more dynamic** This commit replaces all fixed allocations of constant size in the hash table in favor of allocating the slices dynamically which makes the hash table more dynamic as well. This allows us to place the logic of clearing up those slices for the next iteration in a single place which simplifies the code a bit. Additionally it moves `buckets` slice that is only used by the hash joiner out of the hash table. Release note: None **colexec: fix LEFT ANTI hash join when right eq cols are key** Release note (bug fix): Previously, CockroachDB could return incorrect results when performing LEFT ANTI hash join when right equality columns form a key when it was executed via the vectorized engine, and now this has been fixed. **colexec: fix set-op hash joins and optimize them a bit** This commit fixes several problems with set-operation hash joins: 1. similar to how LEFT ANTI hash joins are handled, INTERSECT ALL and EXCEPT ALL hash joins rely on `headID` to be populated. That step is skipped if the right side is distinct (meaning right equality columns form a key). This would result in incorrect output, and we now override `rightDistinct` flag to get the desired behavior (probably sub-optimal but correct). 2. actually, before we would get an incorrect result, we would hit an out of bounds error because `hashTable.visited` would not have been created previously. That slice is used by set-op joins to track which tuples from the right side have already been "deleted" (i.e. they were used for a match). This is now also fixed. However, currently the information whether the right equality columns form a key is not propagated for set-operation joins, so the bug would not have occurred in the non-test environment. Additionally, this commit optimizes the handling of LEFT ANTI and EXCEPT ALL joins by not allocating `same` slice because those two join types have separate `collectAnti*` methods that don't use that slice. Release note: None 53248: release: fix adding libgeos files to .tar.gz and compress .zip r=jlinder a=otan * Fix issue where only the first file was added to a .tar.gz. * Compress files better when adding to zips. Release note: None 53252: sql/kv: optimize heap allocations for inverted joins and scans with many spans r=nvanbenschoten a=nvanbenschoten This PR contains a collection of optimizations that build on #53181, #53215, and #53237 to reduce the number and size of heap allocations performed by inverted joins. Combined, these changes results in a **1.46%** speedup on the following geospatial query: ```sql SELECT Count(census.blkid), Count(subways.name) FROM nyc_census_blocks AS census JOIN nyc_subway_stations AS subways ON ST_Intersects(subways.geom, census.geom); ``` ``` name old ms new ms delta Test/postgis_geometry_tutorial/nyc 144 ±15% 142 ±16% -1.46% (p=0.000 n=972+974) ``` ---- ### kv: reserve lock spans in SpanSet ahead of time In #52610, we merged the latch and lock span declared by requests. Doing so is the long term goal in this area, but it doesn't seem like that's going to be possible for v20.2. Meanwhile, the collection of lock spans is currently less efficient than the collection of latch spans. This is because we have a heuristic to eagerly allocate lock span slices ahead of time, instead of letting them grow as they are appended to. This optimization does not exist for lock spans. We can see this by looking at an alloc_space heap profile while running a geospatial query of interest. We see that the lock spans account for **9.76%** of all allocated space while the latch spans account for only **2.65%** of all allocated space. ``` ----------------------------------------------------------+------------- 646.37MB 100% | github.com/cockroachdb/cockroach/pkg/kv/kvserver/batcheval.DefaultDeclareIsolatedKeys /go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/batcheval/declare.go:90 0 0% 100% 646.37MB 9.76% | github.com/cockroachdb/cockroach/pkg/kv/kvserver/spanset.(*SpanSet).AddNonMVCC /go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/spanset/spanset.go:127 646.37MB 100% | github.com/cockroachdb/cockroach/pkg/kv/kvserver/spanset.(*SpanSet).AddMVCC /go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/spanset/spanset.go:139 ----------------------------------------------------------+------------- 175.37MB 100% | github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).collectSpans /go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/replica_send.go:691 175.37MB 2.65% 61.51% 175.37MB 2.65% | github.com/cockroachdb/cockroach/pkg/kv/kvserver/spanset.(*SpanSet).Reserve /go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/spanset/spanset.go:120 ----------------------------------------------------------+------------- ``` To improve this, we can do the same ahead of time slice resizing for lock spans by reserving an estimate for the number and type of lock spans we expect a request to declare. This guess may be a little less accurate than our guess for latch spans, because there are a few request types that declare latch spans but not lock spans, but these are rare requests that are much less important to optimize than the requests that do declare a latch and a lock span. With this change, we shave off about **6.57%** of total allocated space in this workload. We no longer see allocations in `SpanSet.AddNonMVCC` and the amount of heap memory allocated by `SpanSet.Reserve` roughly doubles, as we would expect. ``` ----------------------------------------------------------+------------- 190.99MB 53.34% | github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).collectSpans /go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/replica_send.go:692 167.09MB 46.66% | github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).collectSpans /go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/replica_send.go:693 358.08MB 5.84% 36.53% 358.08MB 5.84% | github.com/cockroachdb/cockroach/pkg/kv/kvserver/spanset.(*SpanSet).Reserve /go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/spanset/spanset.go:120 ----------------------------------------------------------+------------- ``` ### sql/row: batch allocate RequestUnion wrapper structs in txnKVFetcher.fetch A Request's approach towards representing a union type results in double indirection. For instance, a ScanRequest is represented like: ``` Request{Value: &RequestUnion_Scan{Scan: &ScanRequest{}}} ``` The code in txnKVFetcher.fetch was batch allocating the inner struct in this structure across all scans in a batch, but was not batch allocating the outer "union" struct. The effect of this is that the outer struct's heap allocation showed up as 4.78% of the alloc_objects heap profile of an interesting geospatial query: ``` ----------------------------------------------------------+------------- 2621460 100% | github.com/cockroachdb/cockroach/pkg/sql/row.(*txnKVFetcher).fetch /go/src/github.com/cockroachdb/cockroach/pkg/sql/row/kv_batch_fetcher.go:297 0 0% 100% 2621460 4.78% | github.com/cockroachdb/cockroach/pkg/roachpb.(*RequestUnion).MustSetInner /go/src/github.com/cockroachdb/cockroach/pkg/roachpb/api.go:604 2621460 100% | github.com/cockroachdb/cockroach/pkg/roachpb.(*RequestUnion).SetInner /go/src/github.com/cockroachdb/cockroach/pkg/roachpb/batch_generated.go:354 ----------------------------------------------------------+------------- ``` This commit avoids this source of per-request allocations by batch allocating these union structs across all requests in a batch just like the code was previously batch allocating the inner structs. ### sql/rowexec: avoid allocating slice header on sort in fragmentPendingSpans batchedInvertedExprEvaluator's fragmentPendingSpans performs a `sort.Slice` to sort a list of pending spans by their end key. `sort.Slice` is convenient, but it has some trade-offs. For one, it uses reflection to mimic parameterization over all slice types. Second, it accepts the slice as an interface, which causes the slice header to escape to the heap. The effect of this is that the call was showing up as 4.35% of the alloc_objects heap profile of an interesting geospatial query: ``` ----------------------------------------------------------+------------- 2387773 100% | github.com/cockroachdb/cockroach/pkg/sql/rowexec.(*batchedInvertedExprEvaluator).init /go/src/github.com/cockroachdb/cockroach/pkg/sql/rowexec/inverted_expr_evaluator.go:406 2162754 3.94% 40.58% 2387773 4.35% | github.com/cockroachdb/cockroach/pkg/sql/rowexec.(*batchedInvertedExprEvaluator).fragmentPendingSpans /go/src/github.com/cockroachdb/cockroach/pkg/sql/rowexec/inverted_expr_evaluator.go:284 225019 9.42% | sort.Slice /usr/local/go/src/sort/slice.go:15 ----------------------------------------------------------+------------- ``` This commit avoids this heap allocation by assigning the slice to a field on the batchedInvertedExprEvaluator (which was already on the heap) and using the old `sort.Sort` function. ---- @otan an area I didn't touch but wanted to talk to you about was the memory representation of `DGeography` and `DGeometry`. Specifically, I'm wondering why we're embedding pointers instead of values inside of these datums, effectively creating a double-boxing situation. I see that most of the geo operators work off pointer and that the "canonical" representation of the objects exported by `pkg/geo` is a pointer (e.g. the package exports "New..." methods instead of "Make..." methods). I don't understand why this is though. Both `Geography` and `Geometry` are only 40 bytes large, which is well within reason to pass on the stack. I ask because this makes working with these datums more expensive than strictly necessary. We make an effort to batch allocate datum objects themselves (see `DatumAlloc`), but don't do anything special with internal heap allocations. I suspect that there would be a moderate win here if we eliminated the double-boxing on any query that scans a large number of rows containing one of these data types, as I'm seeing the heap allocations in DecodeUntaggedDatum fairly prominently in profiles. Co-authored-by: Spas Bojanov <[email protected]> Co-authored-by: Yahor Yuzefovich <[email protected]> Co-authored-by: Oliver Tan <[email protected]> Co-authored-by: Nathan VanBenschoten <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Is your feature request related to a problem? Please describe.
We currently export the descriptors and job payloads in debug zips with hex encoding. It is often very valuable to decode these protos into JSON or another human readable format for inspection.
Describe the solution you'd like
I'd like to see a debug commands which take descriptors on stdin in
hexorbase64(with some input encoding flag or auto-detected if we feel fancy). Something likecockroach debug decode-descriptor. I could see it being generalized to justdecode-protoand have a flag for the proto type. I suspect the more general command would be only marginally more work and would be better.Describe alternatives you've considered
One thing we've discussed is to just put a deserialized descriptor into the debug zip. This won't help with older versions and it doesn't help as much with getting descriptors out of running clusters.
For what it's worth, there's been discussion about adding deserialization of protos to sql in #protocolb
The text was updated successfully, but these errors were encountered: