feat: embed measure slots in asset and device documents

[Kuruyia]

What does this PR do ?

This adds a new field to the documents in the assets and devices collections, measureSlots, which is populated with the name and type of the measures from the asset/device model.

This field is automatically updated when the measures of an asset/device model is updated, for each asset/device of that model provisioned in the platform.

Important

Existing assets and devices will not automatically be updated with this new field.

The following script can be used to create this field manually in all existing assets and devices of all engines using kourou sdk:execute:

Migration script

// Change me if necessary
const adminIndex = 'device-manager';

console.log('Fetching engine list...');
const { result: { engines } } = await sdk.query({
  controller: 'device-manager/engine',
  action: 'list',
});
console.log(`Got ${engines.length} engines`);

const deviceModels = [];
console.log('Fetching device models...');
let result = await sdk.document.search(
  adminIndex,
  'models',
  {
    query: {
      term: {
        type: 'device'
      }
    }
  },
  { scroll: '10s' }
);

while (result) {
  deviceModels.push(...result.hits.map(v => v._source));
  result = await result.next();
}

console.log(`Got ${deviceModels.length} device models`);

const assetModels = [];
console.log('Fetching asset models...');
result = await sdk.document.search(
  adminIndex,
  'models',
  {
    query: {
      term: {
        type: 'asset'
      }
    }
  },
  { scroll: '10s' }
);

while (result) {
  assetModels.push(...result.hits.map(v => v._source));
  result = await result.next();
}

const assetModelsPerGroup = assetModels.reduce((acc, v) => {
  return {
    ...acc,
    [v.engineGroup]: [
      ...(acc[v.engineGroup] ?? []),
      v,
    ],
  };
}, {});

console.log(`Got ${assetModels.length} asset models`);

for (const engine of engines) {
  const index = engine.index;
  console.log(`Migrating index "${index}"...`);

  result = await sdk.document.search(
    index,
    'assets',
    {
      _source: ['model'],
      query: {
        bool: {
          must_not: {
            exists: {
              field: 'measureSlots',
            },
          },
        }
      }
    },
    { scroll: '10s' }
  );

  while (result) {
    const documents = [];
    for (const hit of result.hits) {
      let assetModel = assetModelsPerGroup[engine.group].find(v => v.asset.model === hit._source.model);

      if (assetModel === undefined) {
        assetModel = assetModels.find(v => v.asset.model === hit._source.model);

        if (assetModel === undefined) {
          console.error(`In index "${index}" for asset "${hit._id}": unable to find the model (engine group: "${engine.group}"; model: "${hit._source.model}")`);
          continue;
        }

        console.warn(`In index "${index}" for asset "${hit._id}": using a model not from the same engine group (tenant engine group: "${engine.group}"; model: "${hit._source.model}"; model engine group: "${assetModel.engineGroup}")`);
      }

      documents.push({
        _id: hit._id,
        body: {
          measureSlots: assetModel.asset.measures,
        },
      });
    }

    console.log(`Updating ${documents.length} assets`);
    await sdk.document.mUpdate(
      index,
      'assets',
      documents,
      {
        retryOnConflict: 10,
        silent: true,
      },
    );

    result = await result.next();
  }

  result = await sdk.document.search(
    index,
    'devices',
    {
      _source: ['model'],
      query: {
        bool: {
          must_not: {
            exists: {
              field: 'measureSlots',
            },
          },
        }
      }
    },
    { scroll: '10s' }
  );

  while (result) {
    const documents = [];
    for (const hit of result.hits) {
      let deviceModel = deviceModels.find(v => v.device.model === hit._source.model);

      if (deviceModel === undefined) {
        console.error(`In index "${index}" for device "${hit._id}": unable to find the model (engine group: "${engine.group}"; model: "${hit._source.model}")`);
        continue;
      }

      documents.push({
        _id: hit._id,
        body: {
          measureSlots: deviceModel.device.measures,
        },
      });
    }

    console.log(`Updating ${documents.length} devices`);
    await sdk.document.mUpdate(
      index,
      'devices',
      documents,
      {
        retryOnConflict: 10,
        silent: true,
      },
    );

    result = await result.next();
  }
}

Other changes

A new ask, ask:device-manager:device:refresh-model, has been added, that updates the embedded measure slots in each provisioned device with the one found in the requested model (akin to the already-existing ask:device-manager:asset:refresh-model ask).

Boyscout

The device-manager/models:updateAsset action has been updated to update all engines and refresh all assets of the updated model if necessary.

The docs were updated to fix the controller names in the controllers/models/update-asset/ and controllers/models/write-asset/ pages.

Part of KZLPRD-438.