Code

Multi stage stats#12704

Code

Merged

gortiz merged 97 commits intoapache:masterapache/pinot:masterfrom gortiz:multi-stage-statsgortiz/pinot:multi-stage-statsCopy head branch name to clipboardMay 3, 2024

+5,022-1,515

Commits

Commits on Feb 23, 2024

• Make V2 operators use their own logger

  gortiz

  committed
  c1ed6bb

  Copy full SHA for c1ed6bb

Commits on Mar 19, 2024

• Merge branch 'master' into multi-stage-stats

  gortiz

  committed
  c907f04

  Copy full SHA for c907f04

Commits on Mar 21, 2024

• Merge remote-tracking branch 'origin/master' into multi-stage-stats

  gortiz

  committed
  10e7be4

  Copy full SHA for 10e7be4

Commits on Mar 22, 2024

• Create StatMaps and use it in V2

  gortiz

  committed
  d39ef00

  Copy full SHA for d39ef00

• Merge remote-tracking branch 'origin/master' into multi-stage-stats

  gortiz

  committed
  bb72e72

  Copy full SHA for bb72e72

Commits on Mar 26, 2024

• Deep modification on multi-stage stats to:

  Show description for 5db3bb8
  gortiz

  committed
  5db3bb8

  Copy full SHA for 5db3bb8

Commits on Apr 2, 2024

• Several changes, still not tested

  gortiz

  committed
  724d337

  Copy full SHA for 724d337

Commits on Apr 3, 2024

• rename a variable/attribute

  gortiz

  committed
  12dcada

  Copy full SHA for 12dcada

• Fix checkstyle

  gortiz

  committed
  e96ac58

  Copy full SHA for e96ac58

• Modify MultiStageQueryStats to ignore errors while merging stats

  gortiz

  committed
  355732f

  Copy full SHA for 355732f

• Rollback changes in SortOperator

  gortiz

  committed
  298d6f7

  Copy full SHA for 298d6f7

• Remove TransferableBlockUtils.getEndOfStreamTransferableBlock() to always include stats with EOS.

  Show description for a09f6a3
  gortiz

  committed
  a09f6a3

  Copy full SHA for a09f6a3

• Merge branch 'master' into multi-stage-stats

  gortiz

  committed
  983bfb0

  Copy full SHA for 983bfb0

• Fix some tests

  gortiz

  committed
  3a36837

  Copy full SHA for 3a36837

• Rollback to EOS without stats

  gortiz

  committed
  bd3727c

  Copy full SHA for bd3727c

• fix metadata backward compatibility issue

  gortiz

  committed
  204398c

  Copy full SHA for 204398c

• Add and fix some tests

  gortiz

  committed
  0ab2f02

  Copy full SHA for 0ab2f02

• Keep NUM_GROUPS_LIMIT_REACHED in V1

  gortiz

  committed
  dc7dda8

  Copy full SHA for dc7dda8

Commits on Apr 4, 2024

• StatMap without boxing

  gortiz

  committed
  adf22b1

  Copy full SHA for adf22b1

• Serialize StatMaps as older stats

  gortiz

  committed
  1236b1c

  Copy full SHA for 1236b1c

• Simplify StatMap by storing data in an EnumMap

  gortiz

  committed
  84e8aae

  Copy full SHA for 84e8aae

• Create a stat enum for leaf operator

  gortiz

  committed
  ce46a3e

  Copy full SHA for ce46a3e

• Do not keep default values in StatMap

  gortiz

  committed
  d729651

  Copy full SHA for d729651

• Fix QueryLoggerTest

  gortiz

  committed
  d21f97f

  Copy full SHA for d21f97f

• Fix issue in StatMap.merge(K key, int value)

  gortiz

  committed
  ccf8294

  Copy full SHA for ccf8294

Commits on Apr 5, 2024

• Simplify MultiStageOperator.Type.deserializeStats

  gortiz

  committed
  6c979c2

  Copy full SHA for 6c979c2

• Serialize stage stats in the same way StatMap is serialized

  gortiz

  committed
  14ec9c0

  Copy full SHA for 14ec9c0

• Move custom updateEosBlock from MailboxReceiveOperator to BaseMailboxReceiveOperator

  gortiz

  committed
  85dc213

  Copy full SHA for 85dc213

• Improve error detection and report format

  gortiz

  committed
  e83c197

  Copy full SHA for e83c197

• Fix test

  gortiz

  committed
  dec1e22

  Copy full SHA for dec1e22

• Simplify stat relation between MultiStageOperator and BaseMailboxReceiveOperator

  gortiz

  committed
  14bc3af

  Copy full SHA for 14bc3af

• Fix SortedMailboxReceiveOperator not filling EOS when no row was found

  gortiz

  committed
  e172555

  Copy full SHA for e172555

• Add one stat class per multi-stage operator

  gortiz

  committed
  3e17e70

  Copy full SHA for 3e17e70

• Fix some errors when multi stage stats were converted into trees

  gortiz

  committed
  9c89a23

  Copy full SHA for 9c89a23

• Remove MultiStageOperator.BaseStatKeys

  gortiz

  committed
  8b09a7b

  Copy full SHA for 8b09a7b

Commits on Apr 8, 2024

• Merge branch 'master' into multi-stage-stats

  gortiz

  committed
  09fb937

  Copy full SHA for 09fb937

• Fix MIN_CONSUMING_FRESHNESS_TIME_MS merge function

  gortiz

  committed
  d6d344c

  Copy full SHA for d6d344c

• Create functions for generic key merges

  gortiz

  committed
  5083ec5

  Copy full SHA for 5083ec5

• Fix stats on union

  gortiz

  committed
  821237f

  Copy full SHA for 821237f

• Fix QueryLoggerTest

  gortiz

  committed
  115c9d6

  Copy full SHA for 115c9d6

• concat stats from the current stage in set operations

  gortiz

  committed
  c4b9988

  Copy full SHA for c4b9988

Commits on Apr 9, 2024

• Merge remote-tracking branch 'master' into multi-stage-stats

  gortiz

  committed
  ddb32cc

  Copy full SHA for ddb32cc

• Remove unused import

  gortiz

  committed
  f34112d

  Copy full SHA for f34112d

• Remove unused line

  gortiz

  committed
  d8ab892

  Copy full SHA for d8ab892

• Report THREAD_CPU_TIME_NS and SYSTEM_ACTIVITIES_CPU_TIME_NS in V2

  gortiz

  committed
  ec11bae

  Copy full SHA for ec11bae

Commits on Apr 10, 2024

• Add new receive, send and join stats

  gortiz

  committed
  0e38d4d

  Copy full SHA for 0e38d4d

• Make some stats visible by default in JSON format

  gortiz

  committed
  4bc476e

  Copy full SHA for 4bc476e

• Change metadata to be stored in metadata section

  gortiz

  committed
  4a2500a

  Copy full SHA for 4a2500a

• Merge remote-tracking branch 'origin/master' into multi-stage-stats

  gortiz

  committed
  69d6844

  Copy full SHA for 69d6844

• fix syntactic error.

  gortiz

  committed
  974fae2

  Copy full SHA for 974fae2

• fix a test compile issue

  gortiz

  committed
  974fccb

  Copy full SHA for 974fccb

Commits on Apr 11, 2024

• Fix issue in metadata deserialization

  gortiz

  committed
  f166ef2

  Copy full SHA for f166ef2

• FIx MetadataBlockTest.emptyMetadataBlock

  gortiz

  committed
  2a61f5c

  Copy full SHA for 2a61f5c

Commits on Apr 12, 2024

• Keep buffer in the expected position when deserializing

  gortiz

  committed
  8ed9b03

  Copy full SHA for 8ed9b03

• Add v1 deserialization tests

  gortiz

  committed
  a089b2b

  Copy full SHA for a089b2b

• add license headers

  gortiz

  committed
  bf2b554

  Copy full SHA for bf2b554

• Fix checkstyle

  gortiz

  committed
  dcb197c

  Copy full SHA for dcb197c

• Fix tests mocking new required methods

  gortiz

  committed
  4755cba

  Copy full SHA for 4755cba

• Fix concurrency issue

  gortiz

  committed
  ba3a1ce

  Copy full SHA for ba3a1ce

Commits on Apr 15, 2024

• remove unused DataBlock.toDataOnlyDataTable

  gortiz

  committed
  9c54f4d

  Copy full SHA for 9c54f4d

• Increase protocol version of ColumnarDataBlock and RowDataBlock

  gortiz

  committed
  caaaf59

  Copy full SHA for caaaf59

• Add some javadoc

  gortiz

  committed
  2b5b827

  Copy full SHA for 2b5b827

• Improve code style

  gortiz

  committed
  131b956

  Copy full SHA for 131b956

• Small changes in code style

  gortiz

  committed
  09f9a21

  Copy full SHA for 09f9a21

• Rename addStageStat to addStageStats

  gortiz

  committed
  d719a3f

  Copy full SHA for d719a3f

• Remove TODO

  gortiz

  committed
  9329a8a

  Copy full SHA for 9329a8a

• Add javadoc

  gortiz

  committed
  4349d3b

  Copy full SHA for 4349d3b

• Remove OperatorStats

  gortiz

  committed
  a582c8a

  Copy full SHA for a582c8a

• Remove generic from MultiStageOperator

  gortiz

  committed
  f91e263

  Copy full SHA for f91e263

• Rename method

  gortiz

  committed
  8cde598

  Copy full SHA for 8cde598

• Remove blank line

  gortiz

  committed
  4131f10

  Copy full SHA for 4131f10

• Remove test

  gortiz

  committed
  550329b

  Copy full SHA for 550329b

• Update testQueryTestCasesWithMetadata test

  gortiz

  committed
  da15044

  Copy full SHA for da15044

• Merge remote-tracking branch 'origin/master' into multi-stage-stats

  gortiz

  committed
  ebc24f3

  Copy full SHA for ebc24f3

Commits on Apr 23, 2024

• Add comments in recursive case

  gortiz

  committed
  8afef13

  Copy full SHA for 8afef13

• Modify recursiveCase to do not throw even if there is a mismatch in the types

  gortiz

  committed
  600a547

  Copy full SHA for 600a547

• Remove nullable annotation in TransferableBlock constructor

  gortiz

  committed
  3bdcedb

  Copy full SHA for 3bdcedb

• Remove assertion trick in MultiStageQueryStats

  gortiz

  committed
  66bd9ba

  Copy full SHA for 66bd9ba

Commits on Apr 24, 2024

• Make BrokerResponseNativeV2 do not extends BrokerResponseNative

  gortiz

  committed
  2d55d24

  Copy full SHA for 2d55d24

• Merge remote-tracking branch 'master' into multi-stage-stats

  gortiz

  committed
  47ae138

  Copy full SHA for 47ae138

• add comment

  gortiz

  committed
  8327156

  Copy full SHA for 8327156

Commits on Apr 25, 2024

• Properly serialize exceptions in BrokerResponseNativeV2

  gortiz

  committed
  a3d44d2

  Copy full SHA for a3d44d2

• Fix some json issues in BrokerResponseNative and v2

  gortiz

  committed
  bc58f63

  Copy full SHA for bc58f63

Commits on Apr 26, 2024

• Merge remote-tracking branch 'master' into multi-stage-stats

  gortiz

  committed
  bd7afc1

  Copy full SHA for bd7afc1

• Change stageStats to be a tree instead of an array

  gortiz

  committed
  b1bef34

  Copy full SHA for b1bef34

• Remove shouldCollectStats on V2 operators

  gortiz

  committed
  dc4db11

  Copy full SHA for dc4db11

• Show maxRowsInOperator in Pinot UI

  gortiz

  committed
  39ac9eb

  Copy full SHA for 39ac9eb

• Make maxRowsInOperator read only in Jackson

  gortiz

  committed
  0193a0d

  Copy full SHA for 0193a0d

Commits on Apr 29, 2024

• remove isPartialResult

  gortiz

  committed
  2d96491

  Copy full SHA for 2d96491

• Remove StatMap.Type.STRING and make MetadataKey do not implement StatMap.Key

  gortiz

  committed
  c229fbf

  Copy full SHA for c229fbf

• Add StatMap.Type.STRING again

  gortiz

  committed
  12059aa

  Copy full SHA for 12059aa

• Fix bug that reported hash stats in the inverse order

  gortiz

  committed
  746fe2d

  Copy full SHA for 746fe2d

• Fix children array in json stats being listed in inverse order

  gortiz

  committed
  c8468e1

  Copy full SHA for c8468e1

Commits on Apr 30, 2024

• Remove getMaxRowsInOperator in BrokerResponse and Pinot UI

  gortiz

  committed
  02ef0b2

  Copy full SHA for 02ef0b2

• Merge remote-tracking branch master into multi-stage-stats

  gortiz

  committed
  8bebbfc

  Copy full SHA for 8bebbfc

Commits on May 1, 2024

• Add more backward compatibility tests

  gortiz

  committed
  b8334a9

  Copy full SHA for b8334a9

Commits on May 3, 2024

• do not include metadata type in the serialization.

  gortiz

  committed
  fae4984

  Copy full SHA for fae4984