[Fleet] Too many privileges for Elastic Agent data streams

[ruflin]

• Version: 7.14.0-SNAPSHOT

Looking at the default policy in 7.14 it contains the following permissions:

output_permissions:
  default:
    system-1:
      indices:
        - names:
            - logs-system.auth-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - logs-system.syslog-default
          privileges:
            - auto_configure
            - create_doc
    ....
    _elastic_agent_checks:
      cluster:
        - monitor
      indices:
        - names:
            - logs-elastic_agent.*-default
            - metrics-elastic_agent.*-default
          privileges:
            - auto_configure
            - create_doc

To not allow an API key to ingest data in any indices random data streams, no * should exists in the permissions block. But for logs-elastic_agent.*-default and metrics this is the case. Instead the specific sub processes should be listed here. It might be tricky to construct these out of the integrations used and likely even undesired, instead I think we should hardcode the current supported subprocess names and add these to the policy.

[elasticmachine]

Pinging @elastic/fleet (Team:Fleet)

[ruflin]

@michalpristas @afgomez You might be able to share more details on the above. What are the exact permissions needed for monitoring?

[afgomez]

@ruflin my understanding is that we eventually want to get these permissions from the elastic_agent package, but that the package doesn't expose that yet.

Would it make sense to hold on to any changes to this code path until the package exposes the information we need, and then we are done with it?

Aside from this, what would be the downside of leaving the permission block with the wildcard? I understand that we don't want API keys with wildcards, but realistically how big is the problem? If a random person somehow gets the API key and starts to create indices that match the logs-elastic_agent*-default pattern, what's the issue for the end user other than noise? I ask because we do have a case with APM (and any package that contains any datastream with dataset_is_prefix) where the API keys will contain wildcards on them. If it's a problem here it might be a problem there.

[michalpristas]

i'm also for enabling whatever package definition says so as alejandro described above

also bear in mind that with new change enabling us to specify namespace used in agent metrics-elastic_agent.*-default can be any metrics-elastic_agent.*-{namespace}

[ruflin]

The initial discussion on why we need this feature with more details around it can be found here: elastic/fleet-server#101 The part that must be understood is that Elastic Agent can run on untrusted machines, for example your laptop. We already give API keys away and these should have as little permissions as possible as otherwise anyone could in theory overload the cluster as one of the examples.

APM is a different scenario. It is not enabled by default like the elastic_agent integration and in general I expect apm to be run in trusted environments.

@michalpristas I don't follow your example around namespace. I assume the namespace is also set in Fleet for the Elastic Agent so Fleet can updated the permissions?

[jen-huang]

Re: namespace - Fleet already uses the namespace the user defines (not always default) to generate the permissions so there is no issue there.

@michalpristas Is there a concrete list of all possible elastic_agent data streams?

If we don't go down the route of hardcoding it, we will have to fix elastic/integrations#953 and also adjust Fleet to read from the package. I'm not sure how feasible that is right now, capacity wise.

[ruflin]

++ on fixing the package. I'll try to take a stab at it.

[michalpristas]

@jen-huang we dont have a list but assuming we have a static set of programs/processes we are able to run, I could compile one if it's needed

[ruflin]

I started elastic/integrations#1298 as a draft. @michalpristas could you share the list or check my PR to see if I miss something? I assume we need logs and metrics for each process?

[michalpristas]

@ruflin that's correct

[jen-huang]

@afgomez Can you start some work on the Fleet side with the assumption that the elastic_agent package will contain all monitoring data streams?

[ruflin]

For 7.15, we should definitively rely on the package. For 7.14 I'm thinking we should hard code it. The reason for this is that my assumption is the elastic_agent.* pattern is currently hardcoded too. So changing this hardcoded part should be a more minimal change / less risk.

This does not mean, we should still update the package for 7.14 too but I do not expect this to have any effect on the permissions but on the data stream list with the Elastic Agent icon.

[afgomez]

we dont have a list but assuming we have a static set of programs/processes we are able to run, I could compile one if it's needed

@michalpristas Could you collect that list for me? :)

[afgomez]

@michalpristas I took a list from @ruflin's work in elastic/integrations#1298. I applied the changes in #105419

Can you confirm if the values are correct?

[michalpristas]

will take a look