cdc: enable fingerprint validator in randomized tests #134159
Labels
A-cdc
Change Data Capture
C-enhancement
Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)
T-cdc
Comments
rharding6373
added
C-enhancement
![blathers[crl]](https://avatars.githubusercontent.com/in/59700?s=40&u=373d9101a12edc3847d2fb6ae78e3adcfb4ca41e&v=4){kind=avatar}
|
cc @cockroachdb/cdc |
9 tasks
wenyihu6
added a commit
to wenyihu6/cockroach
that referenced
this issue
Jan 15, 2025
…ata types Previously, fingerprint validator parses json message and passes the parsed data into argument $1 for UPSERT, INSERT. This usually works, but gojson.Unmarshal does not work well with all data types. This patch changes fingerprint validator to parse the data by referring to the table info for their column types and parses it using datums. Resolves: cockroachdb#134159 Release note: none
wenyihu6
added a commit
to wenyihu6/cockroach
that referenced
this issue
Jan 15, 2025
…ata types Previously, fingerprint validator parses json message and passes the parsed data into argument $1 for UPSERT, INSERT. This usually works, but gojson.Unmarshal does not work well with all data types. This patch changes fingerprint validator to parse the data by referring to the table info for their column types and parses it using datums. Resolves: cockroachdb#134159 Release note: none
wenyihu6
added a commit
to wenyihu6/cockroach
that referenced
this issue
Jan 15, 2025
…ata types Previously, fingerprint validator parses json message and passes the parsed data into argument $1 for UPSERT, INSERT. This usually works, but gojson.Unmarshal does not work well with all data types. This patch changes fingerprint validator to parse the data by referring to the table info for their column types and parses it using datums. Resolves: cockroachdb#134159 Release note: none
wenyihu6
added a commit
to wenyihu6/cockroach
that referenced
this issue
Jan 15, 2025
…ata types Previously, fingerprint validator parses json message and passes the parsed data into argument $1 for UPSERT, INSERT. This usually works, but gojson.Unmarshal does not work well with all data types. This patch changes fingerprint validator to parse the data by referring to the table info for their column types and parses it using datums. Resolves: cockroachdb#134159 Release note: none
wenyihu6
added a commit
to wenyihu6/cockroach
that referenced
this issue
Jan 15, 2025
…ata types Previously, fingerprint validator parses json message and passes the parsed data into argument $1 for UPSERT, INSERT. This usually works, but gojson.Unmarshal does not work well with all data types. This patch changes fingerprint validator to parse the data by referring to the table info for their column types and parses it using datums. Note that this workaround doesn't support user-created types enums due to its lack of OidToType utility. Resolves: cockroachdb#134159 Release note: none
wenyihu6
added a commit
to wenyihu6/cockroach
that referenced
this issue
Jan 15, 2025
Previously, fingerprint validator parses json message and passes the parsed data into argument $1 for UPSERT, INSERT. This usually works, but gojson.Unmarshal does not work well with all data types. This patch changes fingerprint validator to parse the data by referring to the table info for their column types and parses it using datums. Note that this workaround doesn't support user-created types enums due to its lack of OidToType utility. Resolves: cockroachdb#134159 Release note: none
wenyihu6
added a commit
to wenyihu6/cockroach
that referenced
this issue
Jan 15, 2025
Previously, fingerprint validator parses json message and passes the parsed data into argument $1 for UPSERT, INSERT. This usually works, but gojson.Unmarshal does not work well with all data types. This patch changes fingerprint validator to parse the data by referring to the table info for their column types and parses it using datums. Note that this workaround doesn't support user-created types enums due to its lack of OidToType utility. Resolves: cockroachdb#134159 Release note: none
wenyihu6
added a commit
to wenyihu6/cockroach
that referenced
this issue
Jan 15, 2025
Previously, fingerprint validator parses json message and passes the parsed data into argument $1 for UPSERT, INSERT. This usually works, but gojson.Unmarshal does not work well with all data types. This patch changes fingerprint validator to parse the data by referring to the table info for their column types and parses it using datums. Note that this workaround doesn't support user-created types enums due to its lack of OidToType utility. Resolves: cockroachdb#134159 Release note: none
wenyihu6
added a commit
to wenyihu6/cockroach
that referenced
this issue
Jan 16, 2025
Previously, fingerprint validator parses json message and passes the parsed data into argument $1 for UPSERT, INSERT. This usually works, but gojson.Unmarshal does not work well with all data types. This patch changes fingerprint validator to parse the data by referring to the table info for their column types and parses it using datums. Note that this workaround doesn't support user-created types enums due to its lack of OidToType utility. Resolves: cockroachdb#134159 Release note: none
wenyihu6
added a commit
to wenyihu6/cockroach
that referenced
this issue
Jan 16, 2025
Previously, fingerprint validator parses json message and passes the parsed data into argument $1 for UPSERT, INSERT. This usually works, but gojson.Unmarshal does not work well with all data types. This patch changes fingerprint validator to parse the data by referring to the table info for their column types and parses it using datums. Note that this workaround doesn't support user-created types enums due to its lack of OidToType utility. Resolves: cockroachdb#134159 Release note: none
wenyihu6
added a commit
to wenyihu6/cockroach
that referenced
this issue
Jan 16, 2025
Previously, fingerprint validator parses json message and passes the parsed data into argument $1 for UPSERT, INSERT. This usually works, but gojson.Unmarshal does not work well with all data types. This patch changes fingerprint validator to parse the data by referring to the table info for their column types and parses it using datums. Note that this workaround doesn't support user-created types enums due to its lack of OidToType utility. Resolves: cockroachdb#134159 Release note: none
wenyihu6
added a commit
to wenyihu6/cockroach
that referenced
this issue
Jan 16, 2025
Previously, fingerprint validator parses json message and passes the parsed data into argument $1 for UPSERT, INSERT. This usually works, but gojson.Unmarshal does not work well with all data types. This patch changes fingerprint validator to parse the data by referring to the table info for their column types and parses it using datums. Note that this workaround doesn't support user-created types enums due to its lack of OidToType utility. Resolves: cockroachdb#134159 Release note: none
wenyihu6
added a commit
to wenyihu6/cockroach
that referenced
this issue
Jan 16, 2025
Previously, fingerprint validator parses json message and passes the parsed data into argument $1 for UPSERT, INSERT. This usually works, but gojson.Unmarshal does not work well with all data types. This patch changes fingerprint validator to parse the data by referring to the table info for their column types and parses it using datums. Note that this workaround doesn't support user-created types enums due to its lack of OidToType utility. Resolves: cockroachdb#134159 Release note: none
wenyihu6
added a commit
to wenyihu6/cockroach
that referenced
this issue
Jan 16, 2025
Previously, fingerprint validator parses json message and passes the parsed data into argument $1 for UPSERT, INSERT. This usually works, but gojson.Unmarshal does not work well with all data types. This patch changes fingerprint validator to parse the data by referring to the table info for their column types and parses it using datums. Note that this workaround doesn't support user-created types enums due to its lack of OidToType utility. Resolves: cockroachdb#134159 Release note: none
wenyihu6
added a commit
to wenyihu6/cockroach
that referenced
this issue
Jan 16, 2025
Previously, fingerprint validator parses json message and passes the parsed data into argument $1 for UPSERT, INSERT. This usually works, but gojson.Unmarshal does not work well with all data types. This patch changes fingerprint validator to parse the data by referring to the table info for their column types and parses it using datums. Note that this workaround doesn't support user-created types enums due to its lack of OidToType utility. Resolves: cockroachdb#134159 Release note: none
wenyihu6
added a commit
to wenyihu6/cockroach
that referenced
this issue
Jan 16, 2025
Previously, fingerprint validator parses json message and passes the parsed data into argument $1 for UPSERT, INSERT. This usually works, but gojson.Unmarshal does not work well with all data types. This patch changes fingerprint validator to parse the data by referring to the table info for their column types and parses it using datums. Note that this workaround doesn't support user-created types enums due to its lack of OidToType utility. Resolves: cockroachdb#134159 Release note: none
craig bot
pushed a commit
that referenced
this issue
Jan 17, 2025
139170: cdctest: fix fingerprint validator for random data types r=asg0451,aerfrei a=wenyihu6 Previously, fingerprint validator parses json message and passes the parsed data into argument $1 for UPSERT, INSERT. This usually works, but gojson.Unmarshal does not work well with all data types. This patch changes fingerprint validator to parse the data by referring to the table info for their column types and parses it using datums. Note that this workaround doesn't support user-created types enums due to its lack of OidToType utility. Resolves: #134159 Release note: none 139278: explain: show correct number of mutated rows in EXPLAIN ANALYZE r=yuzefovich a=yuzefovich This commit fixes an oversight in how we populate the "actual row count" in the vectorized engine for mutation statements. For context, we wrap each vectorized operator with `batchInfoCollector` which accumulates the number of tuples the wrapped operator has seen throughout its lifetime which is then shown as "actual row count". For mutation statements we have the following chain of objects involed: `batchInfoCollector -> Columnarizer -> planNodeToRowSource -> mutation planNode`. The `planNodeToRowSource` wrapper has a special "fast-path" logic where it needs to produce only the number of rows modified by its input, and it emits that row count as a single row with a single int datum. Previously, we would treat that special single row with single datum as having been produced by the mutation planNode and would show on EXPLAIN ANALYZE as 1. However, we want to actually show the value from that row instead. This commit adds some special logic to the `batchInfoCollector` for this special case. Fixes: #138971. Release note (bug fix): Previously, EXPLAIN ANALYZE of mutations statements would always get `actual row count: 1` execution statistic for the corresponding mutation node in the plan, regardless of how many rows were actually modified. The bug has been present since before 22.2 version and is now fixed. Co-authored-by: Wenyi Hu <[email protected]> Co-authored-by: Yahor Yuzefovich <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
The
FingerPrintValidatoris a useful validator for changefeeds, because it tests that a feed can be used to reconstruct a table. However, the validator is currently unable to support all types (e.g., arrays), which prevents it from being used in randomized test that can run on all database types.This issue is similar to the beforeAfterValidator: #134158
Jira issue: CRDB-43944
The text was updated successfully, but these errors were encountered: