[Remote State] Decide directory structure and file names for remote cluster state

[soosinha]

Is your feature request related to a problem? Please describe.
We need to decide the file names and directory structure in which the cluster metadata files will be stored. Also, the file data structure should be finalized.

[soosinha]

Uploaded files

To start with, we will only persist the index metadata from the cluster state. One file will be created per index metadata. In addition, there will be a marker/tracker file which will contain the details of the uploaded index metadata files. This marker file will ensure atomicity as well.

Index metadata
Index metadata file path will contain cluster name, cluster uuid and index uuid for segregating the files. The file name will contain metadata version and timestamp. Metadata version is incremented everytime the metadata is updated. Timestamp is added to prevent any conflicts.
Example: test-cluster/cluster-state/dsgYj10Nkso7/index/ftqsCnn9TgOX/metadata_4_1690947200.dat

Marker file
Marker file path will contain cluster name and cluster uuid for grouping. The file name will contain inverted term and inverted version in order to get the highest term and version first since blob storage services returns keys in lexicographical order. The inverted timestamp will also be added to prevent conflicts.
Example: test-cluster/cluster-state/dsgYj10Nkso7/marker/marker_2147483642_2147483637_456536447
Structure of marker file:

{
  "term": 5,
  "version" 10,
  "cluster_uuid": "dsgYj10Nkso7",
  "state_uuid":"dIu34Dh2Hiq",
  "node_id":"7rsyg5FbSeSt",
  "node_version": "3000099",
  "committed": true,
  "metadata":{
    indices:[
      {
        "index_name": "test-index1",
        "index_uuid":"erRh23KlqpT",
        "uploaded_filename": "<uploaded filename with path>"
      },
      {
        "index_name": "test-index2",
        "index_uuid":"eoQui9aW3",
        "uploaded_filename": "<uploaded filename with path>"
      },
      ...
    }
  }
}

Blob Directory structure

• <cluster-name(sanitized)>
  • cluster-state
    • < cluster-uuid >
      • index
        • - metadata_< index-metadata-version >_< timestamp >.dat
      • marker
        • marker_< inverted-term >< inverted-state-version >< inverted-timestamp >