feat(configuration): allow to specify Elasticsearch index settings #337

alexandrebouthinon · 2024-03-22T15:05:43Z

What does this PR do ?

It add missing settings fields in the plugin configuration and make sure to use it during indices creation

How to test it?

Using the kuzzlerc file, you can now set the desired index settings. For example the following configuration:

{
  "plugins": {
    "device-manager": {
      "engineCollections": {
        "config": {
          "settings": {
            "number_of_shards": 3
          }
        },
        "asset": {
          "settings": {
            "number_of_shards": 4
          }
        },
        "assetGroup": {
          "settings": {
            "number_of_shards": 1
          }
        },
        "assetHistory": {
          "settings": {
            "number_of_shards": 3
          }
        },
        "device": {
          "settings": {
            "number_of_shards": 2
          }
        },
        "measures": {
          "settings": {
            "number_of_shards": 7
          }
        }
      }
    }
  }
}

gives you the following index configuration when you create a tenant:

Boyscout

Add a complete documentation about the plugin configuration usage

… indices creation

… recent changes

… format

rolljee · 2024-06-06T12:45:03Z

Test are currently failing, but the rest looks good

… change

# [2.4.0-beta.19](v2.4.0-beta.18...v2.4.0-beta.19) (2024-07-19) ### Features * add tooltip model by asset model ([#341](#341)) ([865ae6d](865ae6d)) * **configuration:** allow to specify Elasticsearch index settings ([#337](#337)) ([6d22fa9](6d22fa9)) * **measure:** add device metadata to origin ([#356](#356)) ([1ed9d8b](1ed9d8b)) * upsert devices ([#355](#355)) ([c75abef](c75abef))

# [2.4.0-elasticsearch-8.1](v2.3.2...v2.4.0-elasticsearch-8.1) (2024-08-06) ### Bug Fixes * **asset_migration:** fixes multiple assets migration and optimize ([fd546cf](fd546cf)) * **assetsGroups:** disable checkRights on impersonate ([#325](#325)) ([eaa3edc](eaa3edc)) * **assets:** migrate tenant only if user is admin ([c24f942](c24f942)) * **assetsMigrateTenant:** add type for frontend and fix admin only ([e249f13](e249f13)) * catch errors when updating the mappings on startup ([#338](#338)) ([e09c5a3](e09c5a3)) * **ci:** correct github token in ci ([a5a6b69](a5a6b69)) * **ci:** fix npm glob not working anymore ([18e79aa](18e79aa)) * copy-version script ([72b5e49](72b5e49)) * correct api types ([#333](#333)) ([05c426f](05c426f)) * correct KuzzleRequest for pipes ([ef932d1](ef932d1)) * correct release script ([f02c285](f02c285)) * **device:** correct device HTTP API ([#354](#354)) ([d9fe9e1](d9fe9e1)) * **engine:** update conflicts ask return values ([0f817c3](0f817c3)) * **export:** correct scroll in getNamedMeasures ([05fe4fd](05fe4fd)) * improve get internal devices documents ([03ccb29](03ccb29)) * improve measure export to be usable with multiple measures in one measure ([#330](#330)) ([dab8d57](dab8d57)) * **migrateTenant:** clear groups of newly created assets ([79f6a5a](79f6a5a)) * **models:** list asset returns commons assets ([#345](#345)) ([2844437](2844437)) * **npm:** fix repository url ([514dc59](514dc59)) * update semantic-release ([380c8b3](380c8b3)) ### Features * add group descriptions to models ([#335](#335)) ([de00fb7](de00fb7)) * add log in decoders ([#339](#339)) ([fe08b7c](fe08b7c)) * add tooltip model by asset model ([#341](#341)) ([865ae6d](865ae6d)) * add UTC export by default for devices and assets ([#342](#342)) ([878ca58](878ca58)) * **asset:** add upsert method ([#323](#323)) ([5f87787](5f87787)) * **assets:** assets migrate tenant ([5f05101](5f05101)) * **assets:** remove assetId in upsert ([#326](#326)) ([2f286de](2f286de)) * **configuration:** allow to specify Elasticsearch index settings ([#337](#337)) ([6d22fa9](6d22fa9)) * digitaltwin exports ([#315](#315)) ([aab65af](aab65af)) * **digitalTwin:** allow to modify mapping from application ([#322](#322)) ([89d375a](89d375a)) * **digitalTwin:** implement generic document pipes triggers ([#321](#321)) ([60a21fc](60a21fc)) * **engine:** add mappings update conflict ask call ([#340](#340)) ([5cd6aca](5cd6aca)) * implement comprehensible metadata ([#332](#332)) ([e80ef8e](e80ef8e)) * improve export ([#316](#316)) ([51db84a](51db84a)) * **measure:** add device metadata to origin ([#356](#356)) ([1ed9d8b](1ed9d8b)) * **measuremodels:** add optional local names and unit to measure definitions ([#343](#343)) ([50038b2](50038b2)) * **measures:** add lastMeasuredAt on DigitalTwin ([#314](#314)) ([05b8a53](05b8a53)) * **metadata:** add support nested metadata properties ([#348](#348)) ([17eda8b](17eda8b)) * **metadata:** improve asset metadata ([#346](#346)) ([2eb5887](2eb5887)) * patch-in support for ES 8 ([6afbe79](6afbe79)) * upsert devices ([#355](#355)) ([c75abef](c75abef))

# [2.4.0-beta.19](v2.4.0-beta.18...v2.4.0-beta.19) (2024-07-19) ### Features * add tooltip model by asset model ([#341](#341)) ([865ae6d](865ae6d)) * **configuration:** allow to specify Elasticsearch index settings ([#337](#337)) ([6d22fa9](6d22fa9)) * **measure:** add device metadata to origin ([#356](#356)) ([1ed9d8b](1ed9d8b)) * upsert devices ([#355](#355)) ([c75abef](c75abef))

# [2.4.0-beta.19](v2.4.0-beta.18...v2.4.0-beta.19) (2024-08-12) ### Features * add tooltip model by asset model ([#341](#341)) ([865ae6d](865ae6d)) * **assetrole:** add link / unlink device action to asset admin role ([#357](#357)) ([febf3a6](febf3a6)) * **configuration:** allow to specify Elasticsearch index settings ([#337](#337)) ([6d22fa9](6d22fa9)) * **measure:** add device metadata to origin ([#356](#356)) ([1ed9d8b](1ed9d8b)) * **metadata:** add option field readOnly to prevent editing ([#358](#358)) ([e0641b4](e0641b4)) * upsert devices ([#355](#355)) ([c75abef](c75abef))

# [2.4.0-dev.1](v2.3.2...v2.4.0-dev.1) (2024-09-27) ### Bug Fixes * add specific dev branch in releaserc.js ([e11a823](e11a823)) * **asset_migration:** fixes multiple assets migration and optimize ([fd546cf](fd546cf)) * **assetsGroups:** disable checkRights on impersonate ([#325](#325)) ([eaa3edc](eaa3edc)) * **assets:** migrate tenant only if user is admin ([c24f942](c24f942)) * **assetsMigrateTenant:** add type for frontend and fix admin only ([e249f13](e249f13)) * catch errors when updating the mappings on startup ([#338](#338)) ([e09c5a3](e09c5a3)) * **ci:** correct github token in ci ([a5a6b69](a5a6b69)) * **ci:** fix npm glob not working anymore ([18e79aa](18e79aa)) * copy-version script ([72b5e49](72b5e49)) * correct api types ([#333](#333)) ([05c426f](05c426f)) * correct KuzzleRequest for pipes ([ef932d1](ef932d1)) * correct release script ([f02c285](f02c285)) * **device-metadata:** fetch device's metadata from assigned tenant when available ([#374](#374)) ([acfd8a0](acfd8a0)) * **device:** correct device HTTP API ([#354](#354)) ([d9fe9e1](d9fe9e1)) * **engine:** update conflicts ask return values ([0f817c3](0f817c3)) * **export:** correct scroll in getNamedMeasures ([05fe4fd](05fe4fd)) * improve get internal devices documents ([03ccb29](03ccb29)) * improve measure export to be usable with multiple measures in one measure ([#330](#330)) ([dab8d57](dab8d57)) * **migrateTenant:** clear groups of newly created assets ([79f6a5a](79f6a5a)) * **models:** list asset returns commons assets ([#345](#345)) ([2844437](2844437)) * **npm:** fix repository url ([514dc59](514dc59)) * only list asset models for the requested engine group ([#367](#367)) ([8a14207](8a14207)) * update semantic-release ([380c8b3](380c8b3)) ### Features * add actions to get last measures on digital twins ([#363](#363)) ([b8005a2](b8005a2)) * add dev builds when pushing on 2-dev ([#375](#375)) ([aa11885](aa11885)) * add group descriptions to models ([#335](#335)) ([de00fb7](de00fb7)) * add log in decoders ([#339](#339)) ([fe08b7c](fe08b7c)) * add search actions for the models ([#362](#362)) ([9990e57](9990e57)) * add tooltip model by asset model ([#341](#341)) ([865ae6d](865ae6d)) * add UTC export by default for devices and assets ([#342](#342)) ([878ca58](878ca58)) * **asset:** add upsert method ([#323](#323)) ([5f87787](5f87787)) * **assetrole:** add link / unlink device action to asset admin role ([#357](#357)) ([febf3a6](febf3a6)) * **assets:** assets migrate tenant ([5f05101](5f05101)) * **assets:** remove assetId in upsert ([#326](#326)) ([2f286de](2f286de)) * **configuration:** allow to specify Elasticsearch index settings ([#337](#337)) ([6d22fa9](6d22fa9)) * digitaltwin exports ([#315](#315)) ([aab65af](aab65af)) * **digitalTwin:** allow to modify mapping from application ([#322](#322)) ([89d375a](89d375a)) * **digitalTwin:** implement generic document pipes triggers ([#321](#321)) ([60a21fc](60a21fc)) * **engine:** add mappings update conflict ask call ([#340](#340)) ([5cd6aca](5cd6aca)) * implement comprehensible metadata ([#332](#332)) ([e80ef8e](e80ef8e)) * improve export ([#316](#316)) ([51db84a](51db84a)) * improve search model request types ([#364](#364)) ([5310ad1](5310ad1)) * **measure:** add device metadata to origin ([#356](#356)) ([1ed9d8b](1ed9d8b)) * **measuremodels:** add optional local names and unit to measure definitions ([#343](#343)) ([50038b2](50038b2)) * **measures:** add lastMeasuredAt on DigitalTwin ([#314](#314)) ([05b8a53](05b8a53)) * **metadata:** add option field readOnly to prevent editing ([#358](#358)) ([e0641b4](e0641b4)) * **metadata:** add optional editor hints for metadata ([#370](#370)) ([afc5129](afc5129)) * **metadata:** add support nested metadata properties ([#348](#348)) ([17eda8b](17eda8b)) * **metadata:** improve asset metadata ([#346](#346)) ([2eb5887](2eb5887)) * **roles:** add specific roles for assets and devices ([#365](#365)) ([a01982c](a01982c)) * upsert devices ([#355](#355)) ([c75abef](c75abef))

# [2.4.0](v2.3.2...v2.4.0) (2024-10-01) ### Bug Fixes * add specific dev branch in releaserc.js ([e11a823](e11a823)) * **asset_migration:** fixes multiple assets migration and optimize ([fd546cf](fd546cf)) * **assetsGroups:** disable checkRights on impersonate ([#325](#325)) ([eaa3edc](eaa3edc)) * **assets:** migrate tenant only if user is admin ([c24f942](c24f942)) * **assetsMigrateTenant:** add type for frontend and fix admin only ([e249f13](e249f13)) * catch errors when updating the mappings on startup ([#338](#338)) ([e09c5a3](e09c5a3)) * **ci:** correct github token in ci ([a5a6b69](a5a6b69)) * **ci:** fix npm glob not working anymore ([18e79aa](18e79aa)) * copy-version script ([72b5e49](72b5e49)) * correct api types ([#333](#333)) ([05c426f](05c426f)) * correct KuzzleRequest for pipes ([ef932d1](ef932d1)) * correct release script ([f02c285](f02c285)) * **device-metadata:** fetch device's metadata from assigned tenant when available ([#374](#374)) ([acfd8a0](acfd8a0)) * **device:** correct device HTTP API ([#354](#354)) ([d9fe9e1](d9fe9e1)) * **engine:** update conflicts ask return values ([0f817c3](0f817c3)) * **export:** correct scroll in getNamedMeasures ([05fe4fd](05fe4fd)) * improve get internal devices documents ([03ccb29](03ccb29)) * improve measure export to be usable with multiple measures in one measure ([#330](#330)) ([dab8d57](dab8d57)) * **migrateTenant:** clear groups of newly created assets ([79f6a5a](79f6a5a)) * **models:** list asset returns commons assets ([#345](#345)) ([2844437](2844437)) * **npm:** fix repository url ([514dc59](514dc59)) * only list asset models for the requested engine group ([#367](#367)) ([8a14207](8a14207)) * update semantic-release ([380c8b3](380c8b3)) ### Features * add actions to get last measures on digital twins ([#363](#363)) ([b8005a2](b8005a2)) * add dev builds when pushing on 2-dev ([#375](#375)) ([aa11885](aa11885)) * add group descriptions to models ([#335](#335)) ([de00fb7](de00fb7)) * add log in decoders ([#339](#339)) ([fe08b7c](fe08b7c)) * add search actions for the models ([#362](#362)) ([9990e57](9990e57)) * add tooltip model by asset model ([#341](#341)) ([865ae6d](865ae6d)) * add UTC export by default for devices and assets ([#342](#342)) ([878ca58](878ca58)) * **asset:** add upsert method ([#323](#323)) ([5f87787](5f87787)) * **assetrole:** add link / unlink device action to asset admin role ([#357](#357)) ([febf3a6](febf3a6)) * **assets:** assets migrate tenant ([5f05101](5f05101)) * **assets:** remove assetId in upsert ([#326](#326)) ([2f286de](2f286de)) * **configuration:** allow to specify Elasticsearch index settings ([#337](#337)) ([6d22fa9](6d22fa9)) * digitaltwin exports ([#315](#315)) ([aab65af](aab65af)) * **digitalTwin:** allow to modify mapping from application ([#322](#322)) ([89d375a](89d375a)) * **digitalTwin:** implement generic document pipes triggers ([#321](#321)) ([60a21fc](60a21fc)) * **engine:** add mappings update conflict ask call ([#340](#340)) ([5cd6aca](5cd6aca)) * implement comprehensible metadata ([#332](#332)) ([e80ef8e](e80ef8e)) * improve export ([#316](#316)) ([51db84a](51db84a)) * improve search model request types ([#364](#364)) ([5310ad1](5310ad1)) * **measure:** add device metadata to origin ([#356](#356)) ([1ed9d8b](1ed9d8b)) * **measuremodels:** add optional local names and unit to measure definitions ([#343](#343)) ([50038b2](50038b2)) * **measures:** add lastMeasuredAt on DigitalTwin ([#314](#314)) ([05b8a53](05b8a53)) * **metadata:** add option field readOnly to prevent editing ([#358](#358)) ([e0641b4](e0641b4)) * **metadata:** add optional editor hints for metadata ([#370](#370)) ([afc5129](afc5129)) * **metadata:** add support nested metadata properties ([#348](#348)) ([17eda8b](17eda8b)) * **metadata:** improve asset metadata ([#346](#346)) ([2eb5887](2eb5887)) * **roles:** add specific roles for assets and devices ([#365](#365)) ([a01982c](a01982c)) * upsert devices ([#355](#355)) ([c75abef](c75abef))

alexandrebouthinon force-pushed the feat/be-able-to-update-elasticsearch-indices-settings branch from 7cc0911 to eb7a7ca Compare March 22, 2024 15:06

alexandrebouthinon self-assigned this Mar 22, 2024

alexandrebouthinon requested review from etrousset and rolljee March 22, 2024 16:28

rolljee approved these changes Mar 25, 2024

View reviewed changes

Add missing settings fields in plugin configuration and use it during…

6ca4998

… indices creation

alexandrebouthinon marked this pull request as draft May 23, 2024 15:30

alexandrebouthinon force-pushed the feat/be-able-to-update-elasticsearch-indices-settings branch from eb7a7ca to 6ca4998 Compare May 29, 2024 15:03

alexandrebouthinon added 3 commits May 29, 2024 17:22

Use correct spelling and adapt collection creation arguments to match…

b04d181

… recent changes

Update Docker Compose stack to use kuzzleio/kuzzle-runner:20

f0609c0

Add documentation about the plugin configuration

a3093aa

alexandrebouthinon changed the title ~~Permit to update Elasticsearch indices settings for each index~~ feat(configuration): allow to specify Elasticsearch index settings Jun 5, 2024

alexandrebouthinon marked this pull request as ready for review June 5, 2024 13:59

alexandrebouthinon requested a review from rolljee June 5, 2024 13:59

Remove plural on assets to avoid breaking change on the configuration…

83c2ff7

… format

Remove plural on devices engine collection property to avoid breaking…

5f1b06e

… change

alexandrebouthinon force-pushed the feat/be-able-to-update-elasticsearch-indices-settings branch from e5837cf to 5f1b06e Compare June 7, 2024 09:24

rolljee approved these changes Jun 10, 2024

View reviewed changes

fmauNeko approved these changes Jul 2, 2024

View reviewed changes

sebtiz13 force-pushed the 2-dev branch from 35a3d1c to 662e642 Compare July 11, 2024 10:47

etrousset merged commit 6d22fa9 into 2-dev Jul 17, 2024
3 checks passed

sebtiz13 deleted the feat/be-able-to-update-elasticsearch-indices-settings branch December 5, 2024 15:59

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(configuration): allow to specify Elasticsearch index settings #337

feat(configuration): allow to specify Elasticsearch index settings #337

alexandrebouthinon commented Mar 22, 2024 •

edited

Loading

rolljee commented Jun 6, 2024

feat(configuration): allow to specify Elasticsearch index settings #337

feat(configuration): allow to specify Elasticsearch index settings #337

Conversation

alexandrebouthinon commented Mar 22, 2024 • edited Loading

What does this PR do ?

How to test it?

Boyscout

rolljee commented Jun 6, 2024

alexandrebouthinon commented Mar 22, 2024 •

edited

Loading