Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

improve watch feature #3848

Closed
1 task done
yichengq opened this issue Nov 10, 2015 · 5 comments
Closed
1 task done

improve watch feature #3848

yichengq opened this issue Nov 10, 2015 · 5 comments
Assignees
@gyuho
Labels
area/performance
Milestone
v2.3.0

Comments

@yichengq
Copy link
Contributor

To-do list

problem: One watcher includes multiple watches, and their events are sent out through one channel. For the received event, user would like to know which watch it belongs to.

- [x] the way to return watched events in one txn: https://github.com//pull/4091

problem: the changes in one txn should be observed by users at the same time, or users may get half-state.

- [x] remove ongoing transaction: https://github.com//pull/3817
- [x] one watch ID for multiple events: https://github.com//pull/4122, https://github.com//pull/4120
- [ ] inefficiency of event distribution through watch

problem: when reading value and sending out event through gRPC, we now unmarshal it first, then marshal it to byte slice again to distribute each time. This is inefficient.

- [ ] add progress notification on watch

problem: user wants to know whether the watch is still working and how far it has gone.
@yichengq yichengq modified the milestone: v2.3.0-alpha1 Nov 10, 2015
@yichengq
Copy link
Contributor Author

The first points have discussed with xiang. The fifth point was mentioned before, and i also list it here. let me know if you have any thoughts or want to reorganize the list.

@xiang90
Copy link
Contributor

xiang90 commented Nov 10, 2015

One watcher includes multiple watches, and their events are sent out through one channel. For the received event, user would like to know which watch it belongs to.

The internal go chan is not the real issue. The real issue is for server-client interaction. The client uses one stream per watcher, which means one stream per multiple watches. The client needs to have a way to figure out which watch the received event belong to.

My thought is to ensure that the events in one txn are returned to users in event slice in one gRPC

I think we can try batched events in one watch response first.

Another thing we need to add is cancellation.

@jonboulle jonboulle added this to the v2.3.0 milestone Nov 12, 2015
@yichengq yichengq mentioned this issue Nov 19, 2015
Merged
@gyuho gyuho self-assigned this Dec 18, 2015
gyuho added a commit to gyuho/etcd that referenced this issue Dec 22, 2015
@gyuho
storage: add more tests for synced, unsynced for watchable store
121d425 
This adds more tests on functions that updates synced and unsynced in watchable
store. Preparatory change for etcd-io#3848.
@gyuho gyuho mentioned this issue Dec 22, 2015
Merged
gyuho added a commit to gyuho/etcd that referenced this issue Dec 22, 2015
@gyuho
storage: add more tests for synced, unsynced for watchable store
9f8af86 
This adds more tests on functions that updates synced and unsynced in watchable
store. Preparatory change for etcd-io#3848.
gyuho added a commit to gyuho/etcd that referenced this issue Dec 22, 2015
@gyuho
storage: add more tests for synced, unsynced for watchable store
d60996f 
This adds more tests on functions that updates synced and unsynced in watchable
store. Preparatory change for etcd-io#3848.
gyuho added a commit to gyuho/etcd that referenced this issue Dec 22, 2015
@gyuho
storage: add more tests for synced, unsynced for watchable store
ea55932 
This adds more tests on functions that updates synced and unsynced in watchable
store. Preparatory change for etcd-io#3848.
gyuho added a commit to gyuho/etcd that referenced this issue Dec 22, 2015
@gyuho
storage: add more tests for synced, unsynced for watchable store
51a029e 
This adds more tests on functions that updates synced and unsynced in watchable
store. Preparatory change for etcd-io#3848.
gyuho added a commit to gyuho/etcd that referenced this issue Dec 22, 2015
@gyuho
storage: add more tests for synced, unsynced for watchable store
28ac0d1 
This adds more tests on functions that updates synced and unsynced in watchable
store. Preparatory change for etcd-io#3848.
gyuho added a commit to gyuho/etcd that referenced this issue Dec 22, 2015
@gyuho
storage: add more tests for synced, unsynced for watchable store
2793ea2 
This adds more tests on functions that updates synced and unsynced in watchable
store. Preparatory change for etcd-io#3848.
gyuho added a commit to gyuho/etcd that referenced this issue Dec 22, 2015
@gyuho
storage: add more tests for synced, unsynced for watchable store
05eb17a 
This adds more tests on functions that updates synced and unsynced in watchable
store. Preparatory change for etcd-io#3848.
gyuho added a commit to gyuho/etcd that referenced this issue Dec 22, 2015
@gyuho
storage: add more tests for synced, unsynced for watchable store
28d0e47 
This adds more tests on functions that updates synced and unsynced in watchable
store. Preparatory change for etcd-io#3848.
gyuho added a commit to gyuho/etcd that referenced this issue Dec 22, 2015
@gyuho
storage: range all unsynced at once
3849241 
This is for etcd-io#3848
to batch RangeHistory for all watchings at once.
@gyuho gyuho mentioned this issue Dec 22, 2015
Merged
gyuho added a commit to gyuho/etcd that referenced this issue Dec 23, 2015
@gyuho
storage: range all unsynced at once
1945eb6 
This is for etcd-io#3848
to batch RangeHistory for all watchings at once.
gyuho added a commit to gyuho/etcd that referenced this issue Dec 23, 2015
@gyuho
storage: range all unsynced at once
d5ce3fd 
This is for etcd-io#3848
to batch RangeHistory for all watchings at once.
gyuho added a commit to gyuho/etcd that referenced this issue Dec 23, 2015
@gyuho
storage: range all unsynced at once
4be489b 
This is for etcd-io#3848
to batch RangeHistory for all watchings at once.
gyuho added a commit to gyuho/etcd that referenced this issue Dec 23, 2015
@gyuho
storage: range all unsynced at once
c2d8f74 
This is for etcd-io#3848
to batch RangeHistory for all watchings at once.
gyuho added a commit to gyuho/etcd that referenced this issue Dec 24, 2015
@gyuho
storage: range all unsynced at once
f01f143 
This is for etcd-io#3848
to batch RangeHistory for all watchings at once.
gyuho added a commit to gyuho/etcd that referenced this issue Dec 24, 2015
@gyuho
storage: range all unsynced at once
41d8b62 
This is for etcd-io#3848
to batch RangeHistory for all watchings at once.
gyuho added a commit to gyuho/etcd that referenced this issue Dec 28, 2015
@gyuho
storage: range all unsynced at once
50b23dd 
This is for etcd-io#3848
to batch RangeHistory for all watchings at once.
gyuho added a commit to gyuho/etcd that referenced this issue Dec 28, 2015
@gyuho
storage: range all unsynced at once
e98f1bf 
This is for etcd-io#3848
to batch RangeHistory for all watchings at once.
gyuho added a commit to gyuho/etcd that referenced this issue Dec 28, 2015
@gyuho
storage: range all unsynced at once
0ce6de1 
This is for etcd-io#3848
to batch RangeHistory for all watchings at once.
gyuho added a commit to gyuho/etcd that referenced this issue Dec 28, 2015
@gyuho
storage: range all unsynced at once
2f4a90a 
This is for etcd-io#3848
to batch RangeHistory for all watchings at once.
gyuho added a commit to gyuho/etcd that referenced this issue Dec 28, 2015
@gyuho
storage: range all unsynced at once
53e90ed 
This is for etcd-io#3848
to batch RangeHistory for all watchings at once.
gyuho added a commit to gyuho/etcd that referenced this issue Dec 28, 2015
@gyuho
storage: range all unsynced at once
cbdef07 
This is for etcd-io#3848
to batch RangeHistory for all watchings at once.
gyuho added a commit to gyuho/etcd that referenced this issue Dec 28, 2015
@gyuho
storage: range all unsynced at once
41b4617 
This is for etcd-io#3848
to batch RangeHistory for all watchings at once.
gyuho added a commit to gyuho/etcd that referenced this issue Dec 28, 2015
@gyuho
storage: range all unsynced at once
56961ed 
This is for etcd-io#3848.
It replaces RangeHistory method for more efficient event
sending.
gyuho added a commit to gyuho/etcd that referenced this issue Dec 28, 2015
@gyuho
storage: range all unsynced at once
78b0b8a 
This is for etcd-io#3848.
It replaces RangeHistory method for more efficient event
sending.
@gyuho gyuho mentioned this issue Dec 29, 2015
Merged
gyuho added a commit to gyuho/etcd that referenced this issue Jan 5, 2016
@gyuho
*: fill in WatchResponse.Header
135dd4e 
Related to etcd-io#3848.
gyuho added a commit to gyuho/etcd that referenced this issue Jan 5, 2016
@gyuho
*: fill in WatchResponse.Header
82366de 
Related to etcd-io#3848.
@gyuho gyuho mentioned this issue Jan 5, 2016
Merged
gyuho added a commit to gyuho/etcd that referenced this issue Jan 5, 2016
@gyuho
*: fill in WatchResponse.Header
72c2c96 
Related to etcd-io#3848.
gyuho added a commit to gyuho/etcd that referenced this issue Jan 5, 2016
@gyuho
*: fill in WatchResponse.Header
5e6fa59 
Related to etcd-io#3848.
gyuho added a commit to gyuho/etcd that referenced this issue Jan 5, 2016
@gyuho
*: fill in WatchResponse.Header
fd465e6 
Related to etcd-io#3848.
gyuho added a commit to gyuho/etcd that referenced this issue Jan 5, 2016
@gyuho
*: fill in WatchResponse.Header
8ac502a 
Related to etcd-io#3848.
gyuho referenced this issue in gyuho/etcd Jan 6, 2016
@gyuho
*: fill in WatchResponse.Header
5a8f624 
Related to coreos#3848.
gyuho referenced this issue in gyuho/etcd Jan 6, 2016
@gyuho
*: fill in WatchResponse.Header
e6be738 
Related to coreos#3848.
gyuho referenced this issue in gyuho/etcd Jan 6, 2016
@gyuho
*: fill in WatchResponse.Header
dde6c6f 
Related to coreos#3848.

remove interface type assertion

use interface, Revision in storage.WatchResponse

use interface raftTimer, and revision from WatchResponse

use EtcdServer to get raft terms, remove new header function

fix deadlock

add interface
gyuho referenced this issue in gyuho/etcd Jan 6, 2016
@gyuho
*: fill in WatchResponse.Header
bbbd858 
Related to coreos#3848.

remove interface type assertion

use interface, Revision in storage.WatchResponse

use interface raftTimer, and revision from WatchResponse

use EtcdServer to get raft terms, remove new header function

fix deadlock

add interface
gyuho referenced this issue in gyuho/etcd Jan 6, 2016
@gyuho
*: fill in WatchResponse.Header
17d8728 
Related to coreos#3848.

remove interface type assertion

use interface, Revision in storage.WatchResponse

use interface raftTimer, and revision from WatchResponse

use EtcdServer to get raft terms, remove new header function

fix deadlock

add interface

fix cluster, memberID
gyuho referenced this issue in gyuho/etcd Jan 6, 2016
@gyuho
*: fill in WatchResponse.Header
7fbf873 
Related to coreos#3848.
gyuho referenced this issue in gyuho/etcd Jan 6, 2016
@gyuho
*: fill in WatchResponse.Header
6629992 
Related to coreos#3848.
gyuho referenced this issue in gyuho/etcd Jan 6, 2016
@gyuho
*: fill in WatchResponse.Header
9654eba 
Related to coreos#3848.
gyuho referenced this issue in gyuho/etcd Jan 6, 2016
@gyuho
*: fill in WatchResponse.Header
366e7a8 
Related to coreos#3848.
@gyuho
Copy link
Contributor

gyuho commented Feb 26, 2016

@xiang90 This feels out of date. Should we close this?

@xiang90
Copy link
Contributor

xiang90 commented Feb 26, 2016

@gyuho Actually we have not finished the last two.

@gyuho
Copy link
Contributor

gyuho commented Feb 26, 2016

Ok got it. thanks.

This was referenced Feb 26, 2016
Closed
Closed
@gyuho gyuho closed this as completed Feb 26, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@gyuho gyuho

Labels
area/performance
Milestone
v2.3.0
Development

No branches or pull requests
4 participants
@jonboulle @yichengq @xiang90 @gyuho