[Filebeat][Okta] Parse additonal debug data risk field for Okta module

[ynirk]

Describe the enhancement:

This request is similar to #25689

Okta enrich their system log events with a risk score ref in debugContext.debugData.risk but it's not parsed by filebeat and it would be valuable.

Exemple of values extracted from event.original:

"risk":"{reasons=Suspicious rate of requests, level=HIGH}"
"risk":"{level=HIGH}"

In addition to parse debugContext.debugData.risk field, other information might be valuable in debugData and it could be a flattened field

Describe a specific use case for the enhancement or feature:

A security analyst would need the risk score parsed in order to build a detection from it.

[elasticmachine]

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

[efd6]

@ynirk Are you able to provide a few redacted example log lines from event.original to include in tests?

Also, to follow up "other information might be valuable in debugData and it could be a flattened field", would you like to see a okta.debug_context.debug_data.flattened field?

[ynirk]

We've made additional tests on this data and it's a bit more messy than expected:
Interesting data can be put in a separate field depending on the configuration: If you don't configure rules to specifically evaluate the risk level of sign-in requests, the results of the risk and behavior evaluation are added to the DebugContext in the System Log in a separate LogOnlySecurityData field ref

This is what we've done in our tests:

• create a flattened field from okta.debug_context.debug_data
• apply the ingest json processor on logOnlySecurityData

So if possible we'd like to have

• okta.debug_context.debug_data.flattened field
• with logOnlySecurityData parsed as json in it
• a separate okta.debug_context.debug_data.risk.level field based either on debugContext.debugData.risk.level or debugContext.debugData.logOnlySecurityData.risk.level (depending which one is present)

Does it make sense? Do you need more information or context ?
Here are 2 redacted event.original (one with logOnlySecurityData and one without it)

okta_risk.txt

[efd6]

Thanks. I'll see how far I get and may get back to you with more questions.

[efd6]

@ynirk I've made a sketch of this for the okta integration first (elastic/integrations#3362), since it's easier to work there. Would you take a look to confirm that it is what you were looking for, once that's done, I'll port over to filebeat.

Note that instead of making ...risk.level, I have pulled back one level and made that ...risk_level.

[ynirk]

@efd6 I made some comments there. I did not realize Okta format is also not consistent depending on risk field location - it's a bit painfull

[efd6]

I did not realize Okta format is also not consistent depending on risk field location - it's a bit painfull

Yes, that's why I pulled it out into okta.debug_context.debug_data.risk_level as a consistent format.