Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

*: support setting endKey for ScanRange #1700

Merged
merged 7 commits into from
Aug 22, 2019
Merged

Conversation

disksing
Copy link
Contributor

@disksing disksing commented Aug 22, 2019

What problem does this PR solve?

Fix #1699
Wait for pingcap/kvproto#447

What is changed and how it works?

  • grpc API support setting endKey
  • merge scan by limit and scan by endkey implementations

Backward compatibility is reserved.

Check List

Tests

  • Unit test

Signed-off-by: disksing <[email protected]>
Signed-off-by: disksing <[email protected]>
Signed-off-by: disksing <[email protected]>
@disksing disksing added the DNM label Aug 22, 2019
@@ -701,6 +701,7 @@ func (c *client) ScanRegions(ctx context.Context, key []byte, limit int) ([]*met
Header: c.requestHeader(),
StartKey: key,
Limit: int32(limit),
EndKey: endKey,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like line 703 is a better place.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok

Signed-off-by: disksing <[email protected]>
@disksing disksing removed the DNM label Aug 22, 2019
Signed-off-by: disksing <[email protected]>
Copy link
Contributor

@Luffbee Luffbee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@disksing disksing added the status/can-merge Indicates a PR has been approved by a committer. label Aug 22, 2019
@sre-bot
Copy link
Contributor

sre-bot commented Aug 22, 2019

/run-all-tests

@disksing disksing merged commit 864c221 into tikv:master Aug 22, 2019
@disksing disksing deleted the scan-regions branch August 22, 2019 10:43
Luffbee pushed a commit that referenced this pull request Aug 27, 2019
* *: unify get store function everywhere (#1671)

Signed-off-by: Ryan Leung <[email protected]>

*  server: use leader lease to determine tso service validity (#1676)

Signed-off-by: disksing <[email protected]>

* test: fix tests (#1696)

* test: fix region syncer test

Signed-off-by: disksing <[email protected]>

* add config-check flag for pd-server (#1695)

Signed-off-by: cwen0 <[email protected]>

* operator: rewrite move region related functions (#1667)

* *: support setting endKey for ScanRange (#1700)

Signed-off-by: disksing <[email protected]>

* *: reduce some unnecessary parameters (#1698)

Signed-off-by: Ryan Leung <[email protected]>

* schedule: Do not send an operator of a region wth a stale epoch (#1659)

* schedule: Do not send an operator of a region wth a stale epoch

Signed-off-by: Shafreeck Sea <[email protected]>

* schedule: check the version changed by the operator self

Signed-off-by: Shafreeck Sea <[email protected]>

* schedule: fix unit test

Signed-off-by: Shafreeck Sea <[email protected]>

* schedule: fix to avoid dispatching a stale opstep

Signed-off-by: Shafreeck Sea <[email protected]>

* dispatch: refactor "ConsumeConfVer() int" to "ExpectConfVerChange() bool"

Signed-off-by: Shafreeck Sea <[email protected]>

* dispatch: fix typo in comment

Signed-off-by: Shafreeck Sea <[email protected]>

* fix typo

Co-Authored-By: Ryan Leung <[email protected]>

* dispatch: fix unittest

Signed-off-by: Shafreeck Sea <[email protected]>

* dispatch: refine format

Signed-off-by: Shafreeck Sea <[email protected]>

* server: fix the dead lock in scatter region (#1706)

Signed-off-by: Ryan Leung <[email protected]>
Luffbee added a commit that referenced this pull request Sep 9, 2019
* *: unify get store function everywhere (#1671)

Signed-off-by: Ryan Leung <[email protected]>

* remove unnecessary parentheses

*  server: use leader lease to determine tso service validity (#1676)

Signed-off-by: disksing <[email protected]>

* change internal stat values to float64

* add pending operator influence

* add metrics of pending influence

* fix metrics

* fix panic

* adjust pending influence of balanceHotWrite

* change weight of pending influence

* test: fix tests (#1696)

* test: fix region syncer test

Signed-off-by: disksing <[email protected]>

* decrease region rolling window; store pending influence in scheduler

* add config-check flag for pd-server (#1695)

Signed-off-by: cwen0 <[email protected]>

* decrease possiblility transfer hot write leader

* change pending influence weight

* add unstarted op metrics

* add logs for debug

* add log for debug

* add logs for debug

* add logs for debug

* add logs for debug

* add logs for debug

* add logs for debug

* add logs for debug

* Revert "add logs for debug"

This reverts commit e74c7a9.

* add metrics for hotspot operators

* operator: rewrite move region related functions (#1667)

* add metrics for pending operators

* *: support setting endKey for ScanRange (#1700)

Signed-off-by: disksing <[email protected]>

* fix bug

* fix bug

* fix bug

* fix metrics thread-safe bug

* fix logic bug

* *: reduce some unnecessary parameters (#1698)

Signed-off-by: Ryan Leung <[email protected]>

* schedule: Do not send an operator of a region wth a stale epoch (#1659)

* schedule: Do not send an operator of a region wth a stale epoch

Signed-off-by: Shafreeck Sea <[email protected]>

* schedule: check the version changed by the operator self

Signed-off-by: Shafreeck Sea <[email protected]>

* schedule: fix unit test

Signed-off-by: Shafreeck Sea <[email protected]>

* schedule: fix to avoid dispatching a stale opstep

Signed-off-by: Shafreeck Sea <[email protected]>

* dispatch: refactor "ConsumeConfVer() int" to "ExpectConfVerChange() bool"

Signed-off-by: Shafreeck Sea <[email protected]>

* dispatch: fix typo in comment

Signed-off-by: Shafreeck Sea <[email protected]>

* fix typo

Co-Authored-By: Ryan Leung <[email protected]>

* dispatch: fix unittest

Signed-off-by: Shafreeck Sea <[email protected]>

* dispatch: refine format

Signed-off-by: Shafreeck Sea <[email protected]>

* server: fix the dead lock in scatter region (#1706)

Signed-off-by: Ryan Leung <[email protected]>

* add drop time for operator

* use IsDropped to recognize canceled ops

* try to fix trans leader burst

* try to fix trans leader burst

* add zombie influence

* change select src dst strategy; improve op_controller

* change select src strategy

* fix bug

* tools: fix set namespace in pd-ctl (#1701)

Signed-off-by: Ryan Leung <[email protected]>

* tools: fix parse url without http prefix (#1703)

Signed-off-by: Ryan Leung <[email protected]>

* tests: support deadlock detection in make test (#1704)

Signed-off-by: Ryan Leung <[email protected]>

* Makefile: fix failpoint enable (#1722)

Signed-off-by: nolouch <[email protected]>

* checker: fix the issue that a region does not merge to the sibling with smaller size (#1723)

Signed-off-by: disksing <[email protected]>

* tools: balance region simulator (#1708)

* scheduler: do not remove the operator when the step does not finish (#1715)

Signed-off-by: Shafreeck Sea <[email protected]>

* operator: fix the AddLearner config version judgment (#1732)

Signed-off-by: nolouch <[email protected]>

* tools: fix TLS in pd control (#1729)

Signed-off-by: Ryan Leung <[email protected]>

* syncer: support TLS for region syncer (#1728)

Signed-off-by: Ryan Leung <[email protected]>

* schedule: fix a thread-safe bug and improve code (#1719)
Luffbee added a commit that referenced this pull request Sep 11, 2019
* *: unify get store function everywhere (#1671)

Signed-off-by: Ryan Leung <[email protected]>

*  server: use leader lease to determine tso service validity (#1676)

Signed-off-by: disksing <[email protected]>

* test: fix tests (#1696)

* test: fix region syncer test

Signed-off-by: disksing <[email protected]>

* add config-check flag for pd-server (#1695)

Signed-off-by: cwen0 <[email protected]>

* operator: rewrite move region related functions (#1667)

* *: support setting endKey for ScanRange (#1700)

Signed-off-by: disksing <[email protected]>

* *: reduce some unnecessary parameters (#1698)

Signed-off-by: Ryan Leung <[email protected]>

* schedule: Do not send an operator of a region wth a stale epoch (#1659)

* schedule: Do not send an operator of a region wth a stale epoch

Signed-off-by: Shafreeck Sea <[email protected]>

* schedule: check the version changed by the operator self

Signed-off-by: Shafreeck Sea <[email protected]>

* schedule: fix unit test

Signed-off-by: Shafreeck Sea <[email protected]>

* schedule: fix to avoid dispatching a stale opstep

Signed-off-by: Shafreeck Sea <[email protected]>

* dispatch: refactor "ConsumeConfVer() int" to "ExpectConfVerChange() bool"

Signed-off-by: Shafreeck Sea <[email protected]>

* dispatch: fix typo in comment

Signed-off-by: Shafreeck Sea <[email protected]>

* fix typo

Co-Authored-By: Ryan Leung <[email protected]>

* dispatch: fix unittest

Signed-off-by: Shafreeck Sea <[email protected]>

* dispatch: refine format

Signed-off-by: Shafreeck Sea <[email protected]>

* server: fix the dead lock in scatter region (#1706)

Signed-off-by: Ryan Leung <[email protected]>

* tools: fix set namespace in pd-ctl (#1701)

Signed-off-by: Ryan Leung <[email protected]>

* tools: fix parse url without http prefix (#1703)

Signed-off-by: Ryan Leung <[email protected]>

* tests: support deadlock detection in make test (#1704)

Signed-off-by: Ryan Leung <[email protected]>

* Makefile: fix failpoint enable (#1722)

Signed-off-by: nolouch <[email protected]>

* checker: fix the issue that a region does not merge to the sibling with smaller size (#1723)

Signed-off-by: disksing <[email protected]>

* tools: balance region simulator (#1708)

* scheduler: do not remove the operator when the step does not finish (#1715)

Signed-off-by: Shafreeck Sea <[email protected]>

* operator: fix the AddLearner config version judgment (#1732)

Signed-off-by: nolouch <[email protected]>

* tools: fix TLS in pd control (#1729)

Signed-off-by: Ryan Leung <[email protected]>

* syncer: support TLS for region syncer (#1728)

Signed-off-by: Ryan Leung <[email protected]>

* schedule: fix a thread-safe bug and improve code (#1719)

* statistics: fix region flow calculation (#1688)

Signed-off-by: jiyingtk <[email protected]>

* makefile: improve deadlock-enable/disable (#1736)

* api: fix missing keys statistic in region information (#1741)

Signed-off-by: nolouch <[email protected]>

* *: update go version to 1.13 (#1742)

Signed-off-by: disksing <[email protected]>

* coordinator: add the operator cost time in log field (#1748)

Signed-off-by: nolouch <[email protected]>
overvenus pushed a commit to overvenus/pd that referenced this pull request Oct 17, 2019
overvenus added a commit that referenced this pull request Oct 18, 2019
* *: support setting endKey for ScanRange (#1700)

Signed-off-by: disksing <[email protected]>
Signed-off-by: Neil Shen <[email protected]>

* update kvproto

Signed-off-by: Neil Shen <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
status/can-merge Indicates a PR has been approved by a committer.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

feature request: support upperbound (endKey) for ScanRegions API
4 participants