Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use any index specified by .watches for Watcher #39541

Merged
merged 9 commits into from
Mar 5, 2019

Conversation

gwbrown
Copy link
Contributor

@gwbrown gwbrown commented Mar 1, 2019

Previously, Watcher only attached its listener to indices that started
with the prefix .watches, which causes Watcher to silently fail to
schedule newly created Watches if the .watches alias is redirected to
an index that does not start with .watches.

Watcher now attaches the listener to all indices, so that Watcher can
respond to changes in which index has the .watches alias.

All tests now randomly use a concrete index which does not start
with .watches (same for .triggered_watches), as well this behavior
being explicitly tested.

Relates to #39478

Previously, Watcher only attached its listener to indices that started
with the prefix `.watches`, which causes Watcher to silently fail to
schedule newly created Watches if the `.watches` alias is redirected to
an index that does not start with `.watches`.

Watcher now attaches the listener to all indices, so that Watcher can
respond to changes in which index has the `.watches` alias.
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-core-features

@gwbrown
Copy link
Contributor Author

gwbrown commented Mar 1, 2019

I did some basic tests with Rally and this doesn't seem to noticeably impact indexing performance, but I've only done a couple of trials.

@gwbrown gwbrown requested review from hub-cap and jakelandis March 1, 2019 23:58
@gwbrown gwbrown marked this pull request as ready for review March 1, 2019 23:58
@gwbrown
Copy link
Contributor Author

gwbrown commented Mar 2, 2019

While I've adjusted the tests to randomly use non-prefixed concrete indices for .watches and .triggered_watches, I've run :x-pack:plugin:watcher:check many times locally with this option forced on (that is, using only non-prefixed indices) to ensure that this is stable.

@gwbrown
Copy link
Contributor Author

gwbrown commented Mar 4, 2019

It's worth noting that the behavior of Watcher when changing the alias is still the same as it was before when it was restricted to prefixed index names: If the new index pointed to by .watches contains the same watches as the old index, everything will keep working just fine. If the contents are different, Watcher will need to be restarted to force it to reload watch definitions from the index.

It's best to switch the alias with Watcher stopped for this reason, but isn't technically necessary as long as the old index and the new index have the same contents.

Copy link
Contributor

@jakelandis jakelandis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@gwbrown gwbrown merged commit d79cd8c into elastic:master Mar 5, 2019
gwbrown added a commit to gwbrown/elasticsearch that referenced this pull request Mar 5, 2019
Previously, Watcher only attached its listener to indices that started
with the prefix `.watches`, which causes Watcher to silently fail to
schedule newly created Watches if the `.watches` alias is redirected to
an index that does not start with `.watches`.

Watcher now attaches the listener to all indices, so that Watcher can
respond to changes in which index has the `.watches` alias.

Also adjusts the tests to randomly use non-prefixed concrete indices for .watches and .triggered_watches.
gwbrown added a commit to gwbrown/elasticsearch that referenced this pull request Mar 5, 2019
Previously, Watcher only attached its listener to indices that started
with the prefix `.watches`, which causes Watcher to silently fail to
schedule newly created Watches if the `.watches` alias is redirected to
an index that does not start with `.watches`.

Watcher now attaches the listener to all indices, so that Watcher can
respond to changes in which index has the `.watches` alias.

Also adjusts the tests to randomly use non-prefixed concrete indices for .watches and .triggered_watches.
gwbrown added a commit to gwbrown/elasticsearch that referenced this pull request Mar 5, 2019
Previously, Watcher only attached its listener to indices that started
with the prefix `.watches`, which causes Watcher to silently fail to
schedule newly created Watches if the `.watches` alias is redirected to
an index that does not start with `.watches`.

Watcher now attaches the listener to all indices, so that Watcher can
respond to changes in which index has the `.watches` alias.

Also adjusts the tests to randomly use non-prefixed concrete indices for .watches and .triggered_watches.
gwbrown added a commit that referenced this pull request Mar 5, 2019
* Use any index specified by .watches for Watcher (#39541)

Previously, Watcher only attached its listener to indices that started
with the prefix `.watches`, which causes Watcher to silently fail to
schedule newly created Watches if the `.watches` alias is redirected to
an index that does not start with `.watches`.

Watcher now attaches the listener to all indices, so that Watcher can
respond to changes in which index has the `.watches` alias.

Also adjusts the tests to randomly use non-prefixed concrete indices 
for .watches and .triggered_watches.
gwbrown added a commit that referenced this pull request Mar 5, 2019
Previously, Watcher only attached its listener to indices that started
with the prefix `.watches`, which causes Watcher to silently fail to
schedule newly created Watches if the `.watches` alias is redirected to
an index that does not start with `.watches`.

Watcher now attaches the listener to all indices, so that Watcher can
respond to changes in which index has the `.watches` alias.

Also adjusts the tests to randomly use non-prefixed concrete indices 
for .watches and .triggered_watches.
gwbrown added a commit that referenced this pull request Mar 5, 2019
Previously, Watcher only attached its listener to indices that started
with the prefix `.watches`, which causes Watcher to silently fail to
schedule newly created Watches if the `.watches` alias is redirected to
an index that does not start with `.watches`.

Watcher now attaches the listener to all indices, so that Watcher can
respond to changes in which index has the `.watches` alias.

Also adjusts the tests to randomly use non-prefixed concrete indices 
for .watches and .triggered_watches.
jasontedor added a commit to jasontedor/elasticsearch that referenced this pull request Mar 6, 2019
* 6.7: (39 commits)
  Remove beta label from CCR (elastic#39722)
  Rename retention lease setting (elastic#39719)
  Add Docker build type (elastic#39378)
  Use any index specified by .watches for Watcher (elastic#39541) (elastic#39706)
  Add documentation on remote recovery (elastic#39483)
  fix typo in synonym graph filter docs
  Removed incorrect ML YAML tests (elastic#39400)
  Improved Terms Aggregation documentation (elastic#38892)
  Fix Fuzziness#asDistance(String) (elastic#39643)
  Revert "unmute EvilLoggerTests#testDeprecatedSettings (elastic#38743)"
  Mute TokenAuthIntegTests.testExpiredTokensDeletedAfterExpiration (elastic#39690)
  Fix security index auto-create and state recovery race (elastic#39582)
  [DOCS] Sorts security APIs
  Check for .watches that wasn't upgraded properly (elastic#39609)
  Assert recovery done in testDoNotWaitForPendingSeqNo (elastic#39595)
  [DOCS] Updates API in Watcher transform context (elastic#39540)
  Fixing the custom object serialization bug in diffable utils. (elastic#39544)
  mute test
  SQL: Don't allow inexact fields for MIN/MAX (elastic#39563)
  Update release notes for 6.7.0
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants