aws-for-fluent-bit: provide realistic runAsNonRoot security context values

[joebowbeer]

Provide realistic values for running fluent-bit as a non-root user:

I am not sure what securityContext Fluent Bit needs to run correctly.
Once this is change is in a think a separate PR could be used to determine
what a default standard policy should look like.

Originally posted by @razorsk8jz in #923 (comment)

The security context comments in values.yaml are not usable:

podSecurityContext: {}
# runAsUser: 1000
# runAsGroup: 1000
# runAsNonRoot: true
# seccompProfile:
#   type: RuntimeDefault
containerSecurityContext: {}
# allowPrivilegeEscalation: false
# capabilities: 
#   drop:
#   - ALL

Issues:

1. The user and group ids do not exist in the aws-for-fluent-bit image. AFAICT the image is based on amazonlinux/scratch and does not define any non-root users.
2. All the files in the image are owned by 0:0 (root) so runAsNonRoot probably won't suffice, at least not without some additional capabilities, such as FOWNER
3. Typical deployments will enable storage.path (e.b., /var/fluent-bit/state/flb-storage/), which appears to need a hostPath

Related:

• Unable to run fluent-bit as non root fluent/fluent-bit#872
• https://docs-bigbang.dso.mil/1.56.0/docs/understanding-bigbang/package-architecture/fluentbit/#Storage
• https://repo1.dso.mil/big-bang/product/packages/fluentbit/-/blob/main/chart/values.yaml

#719

[razorsk8jz]

Posted in fluent/helm-charts#330 as well, but i aws able to get aws-for-fluent-bit running with the following permissions

podSecurityContext:
  runAsUser: 0
  seccompProfile:
    type: RuntimeDefault
containerSecurityContext:
  allowPrivilegeEscalation: false
  readOnlyRootFilesystem: true
  privileged: false
  capabilities:
    drop:
    - ALL

I'm working to get as close to the Pod Security Standards of Restricted which you can see here https://kubernetes.io/docs/concepts/security/pod-security-standards/

but the above policy still fails with

W0329 12:40:59.452300     127 warnings.go:70] would violate PodSecurity "restricted:latest": restricted volume types (volumes "varlog", "varlibdockercontainers" use restricted volume type "hostPath"), runAsNonRoot != true (pod or container "aws-for-fluent-bit" must set securityContext.runAsNonRoot=true), runAsUser=0 (pod must not set runAsUser=0)