Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Cloud security] Fix runtime fields scripts - checking field exist #156458

Merged
merged 6 commits into from
May 4, 2023

Conversation

kfirpeled
Copy link
Contributor

@kfirpeled kfirpeled commented May 2, 2023

Summary

This PR address to an issue we have with backward and forward compatibility

For a robust check of the existence of a field, and that it has a value we first need to use doc.containsKey and afterwards to check it is not empty using empty property: doc["field.path"].empty

Checklist

How to test this

I would recommend using first the Dev Tools to check the runtime fields is correct
For example, I run a query with aggs to see when things break and when not

List of edge cases I've checked:

  • What happens when field's mapping doesn't exist
  • What happens when field's mapping exist but there are documents without that field
  • Checking happy flow
  • Checking field that doesn't exist and doesn't have mappings

Here is the query

  GET /logs-cloud_security_posture.findings_latest-default/_search
  {
    "size": 10,
    "runtime_mappings": { 
      "asset_identifier": {
        "type": "keyword",
        "script": {
          "source": """
          emit(doc["rule.benchmark.posture_type"].empty ? "empty" : doc["rule.benchmark.posture_type"].value)
          """}
      }
    },
    "query": {
      "match": {
        "asset_identifier": "empty"
      }
    },
    "aggs": {
      "score_by_cluster_id": {
        "terms": {
          "field": "asset_identifier"
        }
      }
    }
  }

@kfirpeled kfirpeled changed the title [Cloud security] fix runtime fields scripts - checking field exist [Cloud security] Fix runtime fields scripts - checking field exist May 2, 2023
@kfirpeled kfirpeled marked this pull request as ready for review May 3, 2023 00:04
@kfirpeled kfirpeled requested a review from a team as a code owner May 3, 2023 00:04
@kfirpeled kfirpeled added release_note:skip Skip the PR/issue when compiling release notes v8.8.0 v8.9.0 labels May 3, 2023
@kibana-ci
Copy link
Collaborator

💚 Build Succeeded

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id before after diff
cloudSecurityPosture 166 167 +1

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
cloudSecurityPosture 193.2KB 194.0KB +815.0B
Unknown metric groups

ESLint disabled line counts

id before after diff
enterpriseSearch 19 21 +2
securitySolution 398 401 +3
total +5

Total ESLint disabled count

id before after diff
enterpriseSearch 20 22 +2
securitySolution 478 481 +3
total +5

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@kfirpeled kfirpeled requested review from opauloh and CohenIdo May 3, 2023 21:00
@CohenIdo
Copy link
Contributor

CohenIdo commented May 4, 2023

I tested it locally: LGTM

image

image

@kfirpeled kfirpeled merged commit 17528b2 into elastic:main May 4, 2023
@kfirpeled kfirpeled deleted the cloud_security/fix_runtime_fields branch May 4, 2023 14:41
kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request May 4, 2023
@kibanamachine
Copy link
Contributor

💚 All backports created successfully

Status Branch Result
8.8

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

kibanamachine added a commit that referenced this pull request May 4, 2023
…ist (#156458) (#156701)

# Backport

This will backport the following commits from `main` to `8.8`:
- [[Cloud security] Fix runtime fields scripts - checking field exist
(#156458)](#156458)

<!--- Backport version: 8.9.7 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Kfir
Peled","email":"[email protected]"},"sourceCommit":{"committedDate":"2023-05-04T14:41:27Z","message":"[Cloud
security] Fix runtime fields scripts - checking field exist
(#156458)","sha":"17528b291af9fc7601a9c408cb235f0de083acce","branchLabelMapping":{"^v8.9.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v8.8.0","v8.9.0"],"number":156458,"url":"https://github.com/elastic/kibana/pull/156458","mergeCommit":{"message":"[Cloud
security] Fix runtime fields scripts - checking field exist
(#156458)","sha":"17528b291af9fc7601a9c408cb235f0de083acce"}},"sourceBranch":"main","suggestedTargetBranches":["8.8"],"targetPullRequestStates":[{"branch":"8.8","label":"v8.8.0","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v8.9.0","labelRegex":"^v8.9.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/156458","number":156458,"mergeCommit":{"message":"[Cloud
security] Fix runtime fields scripts - checking field exist
(#156458)","sha":"17528b291af9fc7601a9c408cb235f0de083acce"}}]}]
BACKPORT-->

Co-authored-by: Kfir Peled <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release_note:skip Skip the PR/issue when compiling release notes v8.8.0 v8.9.0
Projects
None yet
4 participants