Add term frequency functions to score script query

[kolchfa-aws]

Fixes #4858

Checklist

• By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and subject to the Developers Certificate of Origin.
  For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[noCharger]

There's script context https://github.com/noCharger/OpenSearch/blob/main/server/src/main/java/org/opensearch/script/ScriptContext.java and each script has their own context instance https://github.com/noCharger/OpenSearch/blob/main/server/src/main/java/org/opensearch/script/ScoreScript.java#L287. I would say expose them in score script source.

[noCharger]

There's some setup for using tf, without them you will get errors like this:

{
  "error": {
    "root_cause": [
      {
        "type": "unsupported_operation_exception",
        "reason": "unsupported_operation_exception: requires a TFIDFSimilarity (such as ClassicSimilarity)"
      }
    ],
    "type": "search_phase_execution_exception",
    "reason": "all shards failed",
    "phase": "query",
    "grouped": true,
    "failed_shards": [
      {
        "shard": 0,
        "index": "test",
        "node": "sQkgvgQWRT2pfI4leTC8lg",
        "reason": {
          "type": "script_exception",
          "reason": "runtime error",
          "script_stack": [
            "org.opensearch.ExceptionsHelper.convertToOpenSearchException(ExceptionsHelper.java:86)",
            "org.opensearch.script.ScoreScriptUtils$TF.tf(ScoreScriptUtils.java:113)",
            "tf(params.field, params.term)",
            "                       ^---- HERE"
          ],
          "script": "tf(params.field, params.term)",
          "lang": "painless",
          "position": {
            "offset": 23,
            "start": 0,
            "end": 29
          },
          "caused_by": {
            "type": "exception",
            "reason": "java.lang.UnsupportedOperationException: requires a TFIDFSimilarity (such as ClassicSimilarity)",
            "caused_by": {
              "type": "unsupported_operation_exception",
              "reason": "unsupported_operation_exception: requires a TFIDFSimilarity (such as ClassicSimilarity)"
            }
          }
        }
      }
    ],
    "caused_by": {
      "type": "unsupported_operation_exception",
      "reason": "unsupported_operation_exception: requires a TFIDFSimilarity (such as ClassicSimilarity)"
    }
  },
  "status": 400
}

Here's some suggestions.

1. Setup the Similarity Model:
   You need to set the similarity model of the field you're working with Ex: ClassicSimilarity, TFIDFSimilarity). Make sure the model is supported under current OpenSearch Version

To do this, you can update the mappings of your index:

PUT /test/_mapping
{
  "properties": {
    "<field_name>": {
      "type": "text",
      "similarity": "classic"
    }
  }
}

2. Reindex your data for existing indices

[kolchfa-aws]

When I tried to set similarity to "classic", I got an error that suggests to use "BM25" as similarity:

{
  "error": {
    "root_cause": [
      {
        "type": "illegal_argument_exception",
        "reason": "The [classic] similarity may not be used anymore. Please use the [BM25] similarity or build a custom [scripted] similarity instead."
      }
    ],
    "type": "illegal_argument_exception",
    "reason": "The [classic] similarity may not be used anymore. Please use the [BM25] similarity or build a custom [scripted] similarity instead."
  },
  "status": 400
}

[noCharger]

Good catch, would you create an issue in OpenSearch repo or I can do it too.

[kolchfa-aws]

I can create an issue

[kolchfa-aws]

Done. The OpenSearch issue is opensearch-project/OpenSearch#9958

[noCharger]

nit picking: it's more consistent use the same doc schema for all examples in this page

[cwillum]

Looks good. Just one comment from me.

[natebower]

@kolchfa-aws Just a couple of commas. Otherwise, LGTM!

[noCharger]

let's remove the docs for tf(). opensearch-project/OpenSearch#9995

[noCharger]

Hi team, anything can I do to get this PR merged?

[kolchfa-aws]

@noCharger The PR is in the "Done but ready to merge" status. This means we'll merge it by the release date. So you can mark it done on your side.