Merge main to 1.x (#183)

* Add integTest script to the repo (#94) Signed-off-by: Peter Zhu <[email protected]> * Removing Usages of Action Get Call and using listeners (#100) Signed-off-by: Aditya Jindal <[email protected]> * Enhance ISM template (#105) Signed-off-by: bowenlan-amzn <[email protected]> * Explain response still use old opendistro policy id (#109) * Explain response still use old opendistro policy id * Use hardcoded policyid setting in tests for explain response * Trying to fix flaky tests * Storing user information as part of the job when security plugin is installed (#113) Signed-off-by: Ravi Thaluru <[email protected]> * ISM/Notification channel support (#117) * Updates NAME of transport actions * Upgrades Kotlin version, updates dependencies on other OS plugins, adds notification plugin as a test resource and includes it in test clusters * Adds support for Channels in error notifications and notification actions * Adds support for sending notifications to channels * Adds support for publishing notifications to the legacy destinations through the Notification plugin and some cleanup * Removes notification alerting jar dependency * Adds compile only dep on commons codec for digest utils sha1 method in ism rollup * Updates Error Notification to make channel/destination nullable, and adds helper methods for publish calls * Constructs URL for legacy custom webhook Signed-off-by: Drew Baugher <[email protected]> * Fixes Feature enum and dep Signed-off-by: Drew Baugher <[email protected]> * Trying something else Signed-off-by: Drew Baugher <[email protected]> * Addresses comments Signed-off-by: Drew Baugher <[email protected]> * Storing user object in all APIs and enabling filter of response based on user (#115) Signed-off-by: Ravi Thaluru <[email protected]> * Upgrade dependencies to 1.1 and build snapshot by default. (#121) Signed-off-by: dblock <[email protected]> * Security improvements (#126) Signed-off-by: Ravi Thaluru <[email protected]> * Removes support for notification plugin (#136) Signed-off-by: Drew Baugher <[email protected]> * Updating security filtering logic (#137) Signed-off-by: Ravi Thaluru <[email protected]> * Release note for 1.1.0.0 release. (#139) * Release note for 1.1.0.0 release. Signed-off-by: bowenlan-amzn <[email protected]> * Correct copyright notices * Uses published daily snapshot dependencies (#141) Signed-off-by: Drew Baugher <[email protected]> * License header check (#142) * Provide default copyright header using IDE feature Signed-off-by: bowenlan-amzn <[email protected]> * Address #103 history write index is rolled over even if the history indices are disabled * Removed integtest.sh. (#148) Signed-off-by: dblock <[email protected]> * Adds mavenLocal back to repositories (#158) Signed-off-by: Drew Baugher <[email protected]> * Making snapshot name to scripted input in template (#77) Signed-off-by: Ravi Thaluru <[email protected]> * Fix issues with security changes in rollup runnner (#161) * Updates index management version to 1.2 (#157) * Updates index management version to 1.2 * Updates job scheduler snapshot to 1.2 in test resources Signed-off-by: Robert Downs <[email protected]> * Adds setting to search all rollup jobs on a target index (#165) * Adds cluster setting to search all rollup jobs Signed-off-by: Clay Downs <[email protected]> * Adds implementation for the delay feature in rollup jobs (#147) * Adds delay implementation for rollup jobs * Removes non-continuous delay implementation * Adds additional rollup delay tests Signed-off-by: Clay Downs <[email protected]> * Updates testCompile mockito version, adds AwaitsFix annotation to MetadataRegressionIT tests (#168) * Updates testCompile mockito version to match OpenSearch changes * AwaitsFix on the failing MetadataRegressionIT tests Signed-off-by: Robert Downs <[email protected]> * Adds cluster setting to configure index state management jitter (#153) * Adds jitter cluster setting, sets jitter to 0 for ISM tests Signed-off-by: Clay Downs <[email protected]> * Allows out of band rollovers on an index without causing ISM to fail (#180) * Allows out of band rollovers on an index without causing ISM to fail Signed-off-by: Drew Baugher <[email protected]> * Fixes detekt issue Signed-off-by: Drew Baugher <[email protected]> * Remove policy API on read only indices (#182) * In explain API not showing the total count to all users (#185) Co-authored-by: Peter Zhu <[email protected]> Co-authored-by: Aditya Jindal <[email protected]> Co-authored-by: Bowen Lan <[email protected]> Co-authored-by: Ravi <[email protected]> Co-authored-by: Drew Baugher <[email protected]> Co-authored-by: Daniel Doubrovkine (dB.) <[email protected]>
opensearch-project · Nov 5, 2021 · 19215c2 · 19215c2
1 parent 091fe9f
commit 19215c2
Show file tree

Hide file tree

Showing 167 changed files with 4,321 additions and 1,507 deletions.
diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md
@@ -3,7 +3,7 @@
 *Description of changes:*
 
 *CheckList:*
-[ ] Commits are signed per the DCO using --signoff
+- [ ] Commits are signed per the DCO using --signoff
 
 By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
 For more information on following Developer Certificate of Origin and signing off your commits, please check [here](https://github.com/opensearch-project/index-management/blob/main/CONTRIBUTING.md#developer-certificate-of-origin).
diff --git a/.github/draft-release-notes-config.yml b/.github/draft-release-notes-config.yml
@@ -14,7 +14,7 @@ replacers:
   - search: '##'
     replace: '###'
 
-# Organizing the tagged PRs into unified ODFE categories
+# Organizing the tagged PRs into unified OpenSearch categories
 categories:
   - title: 'Breaking changes'
     labels:
@@ -45,4 +45,4 @@ categories:
       - 'backwards-compatibility'
   - title: 'Refactoring'
     labels:
-      - 'Refactor'
+      - 'refactor'
diff --git a/.github/workflows/multi-node-test-workflow.yml b/.github/workflows/multi-node-test-workflow.yml
@@ -20,51 +20,11 @@ jobs:
         uses: actions/setup-java@v1
         with:
           java-version: 14
-      # dependencies: OpenSearch
-      - name: Checkout OpenSearch
-        uses: actions/checkout@v2
-        with:
-          repository: 'opensearch-project/OpenSearch'
-          path: OpenSearch
-          ref: '1.0'
-      - name: Build OpenSearch
-        working-directory: ./OpenSearch
-        run: ./gradlew publishToMavenLocal -Dbuild.snapshot=false
-      # dependencies: common-utils
-      - name: Checkout common-utils
-        uses: actions/checkout@v2
-        with:
-          repository: 'opensearch-project/common-utils'
-          path: common-utils
-          ref: '1.0'
-      - name: Build common-utils
-        working-directory: ./common-utils
-        run: ./gradlew publishToMavenLocal -Dopensearch.version=1.0.0
-      # dependencies: job-scheduler
-      - name: Checkout job-scheduler
-        uses: actions/checkout@v2
-        with:
-          repository: 'opensearch-project/job-scheduler'
-          path: job-scheduler
-          ref: '1.0'
-      - name: Build job-scheduler
-        working-directory: ./job-scheduler
-        run: ./gradlew publishToMavenLocal -Dopensearch.version=1.0.0 -Dbuild.snapshot=false
-      # dependencies: alerting-notification
-      - name: Checkout alerting
-        uses: actions/checkout@v2
-        with:
-          repository: 'opensearch-project/alerting'
-          path: alerting
-          ref: '1.0'
-      - name: Build alerting
-        working-directory: ./alerting
-        run: ./gradlew :alerting-notification:publishToMavenLocal -Dopensearch.version=1.0.0 -Dbuild.snapshot=false
       # index-management
       - name: Checkout Branch
         uses: actions/checkout@v2
       - name: Run integration tests with multi node config
-        run: ./gradlew integTest -PnumNodes=3
+        run: ./gradlew integTest -PnumNodes=3 -Dopensearch.version=1.2.0-SNAPSHOT
       - name: Upload failed logs
         uses: actions/upload-artifact@v2
         if: failure()

diff --git a/.github/workflows/test-and-build-workflow.yml b/.github/workflows/test-and-build-workflow.yml
@@ -20,51 +20,11 @@ jobs:
         uses: actions/setup-java@v1
         with:
           java-version: 14
-      # dependencies: OpenSearch
-      - name: Checkout OpenSearch
-        uses: actions/checkout@v2
-        with:
-          repository: 'opensearch-project/OpenSearch'
-          path: OpenSearch
-          ref: '1.0'
-      - name: Build OpenSearch
-        working-directory: ./OpenSearch
-        run: ./gradlew publishToMavenLocal -Dbuild.snapshot=false
-      # dependencies: common-utils
-      - name: Checkout common-utils
-        uses: actions/checkout@v2
-        with:
-          repository: 'opensearch-project/common-utils'
-          path: common-utils
-          ref: '1.0'
-      - name: Build common-utils
-        working-directory: ./common-utils
-        run: ./gradlew publishToMavenLocal -Dopensearch.version=1.0.0
-      # dependencies: job-scheduler
-      - name: Checkout job-scheduler
-        uses: actions/checkout@v2
-        with:
-          repository: 'opensearch-project/job-scheduler'
-          path: job-scheduler
-          ref: '1.0'
-      - name: Build job-scheduler
-        working-directory: ./job-scheduler
-        run: ./gradlew publishToMavenLocal -Dopensearch.version=1.0.0 -Dbuild.snapshot=false
-      # dependencies: alerting-notification
-      - name: Checkout alerting
-        uses: actions/checkout@v2
-        with:
-          repository: 'opensearch-project/alerting'
-          path: alerting
-          ref: '1.0'
-      - name: Build alerting
-        working-directory: ./alerting
-        run: ./gradlew :alerting-notification:publishToMavenLocal -Dopensearch.version=1.0.0 -Dbuild.snapshot=false
-      # index-management
+      # build index management
       - name: Checkout Branch
         uses: actions/checkout@v2
       - name: Build with Gradle
-        run: ./gradlew build
+        run: ./gradlew build -Dopensearch.version=1.2.0-SNAPSHOT
       - name: Upload failed logs
         uses: actions/upload-artifact@v2
         if: failure()

diff --git a/.gitignore b/.gitignore
@@ -1,9 +1,12 @@
 .gradle/
 build/
 out/
-.idea/
+.idea/*
+!.idea/copyright
 *.ipr
 *.iws
 .DS_Store
 *.log
-http
+http
+.project
+.settings
diff --git a/.idea/copyright/OpenSearch.xml b/.idea/copyright/OpenSearch.xml
diff --git a/.idea/copyright/profiles_settings.xml b/.idea/copyright/profiles_settings.xml
diff --git a/NOTICE b/NOTICE
@@ -1,12 +1,2 @@
-OpenSearch
-Copyright 2021 OpenSearch Contributors
-
-This product includes software developed by
-Elasticsearch (http://www.elastic.co).
-Copyright 2009-2018 Elasticsearch
-
-This product includes software developed by The Apache Software
-Foundation (http://www.apache.org/).
-
-This product includes software developed by
-Joda.org (http://www.joda.org/).
+OpenSearch (https://opensearch.org/)
+Copyright OpenSearch Contributors
diff --git a/README.md b/README.md
@@ -1,6 +1,6 @@
 [![Test and Build Workflow](https://github.com/opensearch-project/index-management/workflows/Test%20and%20Build%20Workflow/badge.svg)](https://github.com/opensearch-project/index-management/actions)
 [![codecov](https://codecov.io/gh/opensearch-project/index-management/branch/main/graph/badge.svg)](https://codecov.io/gh/opensearch-project/index-management)
-[![Documentation](https://img.shields.io/badge/api-reference-blue.svg)](https://docs-beta.opensearch.org/im-plugin/index/)
+[![Documentation](https://img.shields.io/badge/api-reference-blue.svg)](https://opensearch.org/docs/im-plugin/index/)
 [![Chat](https://img.shields.io/badge/chat-on%20forums-blue)](https://discuss.opendistrocommunity.dev/c/index-management/)
 ![PRs welcome!](https://img.shields.io/badge/PRs-welcome!-success)
 
@@ -58,7 +58,7 @@ See [developer guide](DEVELOPER_GUIDE.md) and [how to contribute to this project
 
 If you find a bug, or have a feature request, please don't hesitate to open an issue in this repository.
 
-For more information, see [project website](https://opensearch.org/) and [documentation](https://docs-beta.opensearch.org/). If you need help and are unsure where to open an issue, try [forums](https://discuss.opendistrocommunity.dev/).
+For more information, see [project website](https://opensearch.org/) and [documentation](https://opensearch.org/docs/). If you need help and are unsure where to open an issue, try [forums](https://discuss.opendistrocommunity.dev/).
 
 ## Code of Conduct
 
@@ -74,4 +74,4 @@ This project is licensed under the [Apache v2.0 License](./LICENSE)
 
 ## Copyright
 
-Copyright 2020-2021 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+Copyright OpenSearch Contributors. See [NOTICE](NOTICE) for details.
diff --git a/build-tools/coverage.gradle b/build-tools/coverage.gradle
@@ -1,3 +1,8 @@
+/*
+ * Copyright OpenSearch Contributors
+ * SPDX-License-Identifier: Apache-2.0
+ */
+
 /*
  *   Copyright 2019 Amazon.com, Inc. or its affiliates. All Rights Reserved.
  *

diff --git a/build-tools/pkgbuild.gradle b/build-tools/pkgbuild.gradle
@@ -1,3 +1,8 @@
+/*
+ * Copyright OpenSearch Contributors
+ * SPDX-License-Identifier: Apache-2.0
+ */
+
 /*
  *   Copyright 2019 Amazon.com, Inc. or its affiliates. All Rights Reserved.
  *

diff --git a/build.gradle b/build.gradle
@@ -31,14 +31,20 @@ import java.util.function.Predicate
 
 buildscript {
     ext {
-        opensearch_version = System.getProperty("opensearch.version", "1.0.0")
-        kotlin_version = System.getProperty("kotlin.version", "1.3.72")
+        opensearch_version = System.getProperty("opensearch.version", "1.2.0-SNAPSHOT")
+        // 1.1.0 -> 1.1.0.0, and 1.1.0-SNAPSHOT -> 1.1.0.0-SNAPSHOT
+        opensearch_build = opensearch_version.replaceAll(/(\.\d)([^\d]*)$/, '$1.0$2')
+        notification_version = System.getProperty("notification.version", opensearch_build)
+        common_utils_version = System.getProperty("common_utils.version", opensearch_build)
+        job_scheduler_version = System.getProperty("job_scheduler_version.version", opensearch_build)
+        kotlin_version = System.getProperty("kotlin.version", "1.4.0")
     }
 
     repositories {
         mavenLocal()
         mavenCentral()
         maven { url "https://plugins.gradle.org/m2/" }
+        maven { url "https://aws.oss.sonatype.org/content/repositories/snapshots" }
     }
 
     dependencies {
@@ -139,30 +145,35 @@ configurations.testCompile {
 
 ext {
     projectSubstitutions = [:]
-    opensearchVersion = "${version}"
     isSnapshot = "true" == System.getProperty("build.snapshot", "true")
     licenseFile = rootProject.file('LICENSE')
     noticeFile = rootProject.file('NOTICE')
 }
 
-group = "org.opensearch"
-version = "${opensearchVersion}.0"
+allprojects {
+    group = "org.opensearch"
+    version = "${opensearch_version}" - "-SNAPSHOT" + ".0"
+    if (isSnapshot) {
+        version += "-SNAPSHOT"
+    }
+}
 
 dependencies {
     compileOnly "org.opensearch:opensearch:${opensearch_version}"
-    compileOnly "org.opensearch:opensearch-job-scheduler-spi:1.0.0.0"
+    compileOnly "org.opensearch:opensearch-job-scheduler-spi:${job_scheduler_version}"
     compile "org.jetbrains.kotlin:kotlin-stdlib:${kotlin_version}"
     compile "org.jetbrains.kotlin:kotlin-stdlib-common:${kotlin_version}"
-    compile 'org.jetbrains.kotlinx:kotlinx-coroutines-core:1.3.7'
+    compile "org.jetbrains.kotlin:kotlin-stdlib-jdk8:${kotlin_version}"
+    compile 'org.jetbrains.kotlinx:kotlinx-coroutines-core:1.3.9'
     compile "org.jetbrains:annotations:13.0"
-    compile "org.opensearch:notification:1.0.0.0"
-    compile "org.opensearch:common-utils:1.0.0.0"
+    compile "org.opensearch:notification:${notification_version}"
+    compile "org.opensearch:common-utils:${common_utils_version}"
     compile "com.github.seancfoley:ipaddress:5.3.3"
 
     testCompile "org.opensearch.test:framework:${opensearch_version}"
     testCompile "org.jetbrains.kotlin:kotlin-test:${kotlin_version}"
     testImplementation "com.nhaarman.mockitokotlin2:mockito-kotlin:2.2.0"
-    testCompile "org.mockito:mockito-core:2.23.0"
+    testCompile "org.mockito:mockito-core:3.12.4"
 
     add("ktlint", "com.pinterest:ktlint:0.41.0") {
         attributes {
@@ -173,10 +184,7 @@ dependencies {
 
 repositories {
     mavenLocal()
-}
-
-if (isSnapshot) {
-    version += "-SNAPSHOT"
+    maven { url "https://aws.oss.sonatype.org/content/repositories/snapshots" }
 }
 
 plugins.withId('java') {
@@ -186,7 +194,6 @@ plugins.withId('java') {
 plugins.withId('org.jetbrains.kotlin.jvm') {
     compileKotlin.kotlinOptions.jvmTarget = compileTestKotlin.kotlinOptions.jvmTarget = "1.8"
     compileKotlin.dependsOn ktlint
-
 }
 
 javadoc.enabled = false // turn off javadoc as it barfs on Kotlin code
@@ -261,6 +268,7 @@ testClusters.integTest {
             File getAsFile() { fileTree("src/test/resources/job-scheduler").getSingleFile() }
         }
     }))
+
     if (securityEnabled) {
         plugin(provider({
             new RegularFile() {

diff --git a/docs/rfc.md b/docs/rfc.md
@@ -113,4 +113,4 @@ After building Index State Management API, we will build an administrative panel
 
 ## Providing Feedback
 
-If you have comments or feedback on our plans for Index Management, please comment on [the RFC Github issue](../../issues/1) in this project to discuss.
+If you have comments or feedback on our plans for Index Management, please comment on [the RFC Github issue](https://github.com/opendistro-for-elasticsearch/index-management/issues/1) in this project to discuss.
diff --git a/gradle.properties b/gradle.properties
diff --git a/release-notes/opensearch-index-management.release-notes-1.1.0.0.md b/release-notes/opensearch-index-management.release-notes-1.1.0.0.md
@@ -0,0 +1,23 @@
+## Version 1.1.0.0 2021-09-03
+
+Compatible with OpenSearch 1.1.0
+
+### Infrastructure
+
+* Upgrade dependencies to 1.1 and build snapshot by default. ([#121](https://github.com/opensearch-project/index-management/pull/121))
+
+### Features
+
+* Storing user information as part of the job when security plugin is installed ([#113](https://github.com/opensearch-project/index-management/pull/113))
+* Storing user object in all APIs and enabling filter of response based on user ([#115](https://github.com/opensearch-project/index-management/pull/115))
+* Security improvements ([#126](https://github.com/opensearch-project/index-management/pull/126))
+* Updating security filtering logic ([#137](https://github.com/opensearch-project/index-management/pull/137))
+
+### Enhancements
+
+* Enhance ISM template ([#105](https://github.com/opensearch-project/index-management/pull/105))
+
+### Bug Fixes
+
+* Removing Usages of Action Get Call and using listeners ([#100](https://github.com/opensearch-project/index-management/pull/100))
+* Explain response still use old opendistro policy id ([#109](https://github.com/opensearch-project/index-management/pull/109))
Original file line number	Diff line number	Diff line change
Expand Up		@@ -113,4 +113,4 @@ After building Index State Management API, we will build an administrative panel

		## Providing Feedback

		If you have comments or feedback on our plans for Index Management, please comment on [the RFC Github issue](../../issues/1) in this project to discuss.
		If you have comments or feedback on our plans for Index Management, please comment on [the RFC Github issue](https://github.com/opendistro-for-elasticsearch/index-management/issues/1) in this project to discuss.