Elasticsearch sink refactor

[huseyinbabal]

Description

Changes proposed in this pull request:

• Elasticsearch sink is updated to accept new contract api.Message
• Introduced rawData param to send original data to sinks
• Collector is updated to enable plugins for also sinks

Testing

Build plugins

PLUGIN_DOWNLOAD_URL_BASE_PATH=http://localhost:8080/plugin-dist make gen-plugins-index

Run plugin server

npx serve --listen 8080

Run Botkube with the following config

export BOTKUBE_CONFIG_PATHS=/tmp/botkube.yml
export BOTKUBE_SETTINGS_KUBECONFIG=${USER}/.kube/config

go run cmd/botkube/main.go

## /tmp/botkube.yml
communications:
  'default-group':
    elasticsearch:
      enabled: true
      server: 'elastic_url'
      username: user
      password: pass
      indices:
        'default':
          name: botkube
          type: kubernetes
          shards: 1
          replicas: 1
          bindings:
            sources: 
              - k8s-events
              - k8s-updates


settings:
  clusterName: gke-playground
  configWatcher: true
  log:
    level: debug
  lifecycleServer:
    deployment:
      name: botkube
      namespace: botkube
    port: "2113"

configWatcher:
  enabled: false
  initialSyncTimeout: 0

sources:
  'k8s-events':
    displayName: "K8s recommendations"
    'botkube/kubernetes':
      enabled: true
      config:
        log:
          level: debug
        recommendations:
          pod:
            noLatestImageTag: true
            labelsSet: true
          ingress:
            backendServiceValid: false
            tlsSecretValid: false
        namespaces:
          include:
            - botkube
        event:
          types:
            - create
            - update

  'k8s-updates':
    displayName: "K8s ConfigMaps updates"
    'botkube/kubernetes':
      enabled: true
      config:
        log:
          level: debug
        namespaces:
          include:
            - default
        event:
          types:
            - create
            - update
            - delete
        resources:
          - type: v1/configmaps
            namespaces:
              include:
                - botkube
            event: # overrides top level `event` entry
              types:
                - update
            updateSetting:
              includeDiff: true
              fields:
                - data

plugins:
  repositories:
    botkube:
      url: http://localhost:8080/plugins-index.yaml

Create a pod to see a notification

kubectl run nginx_es --image=nginx:latest -n botkube

Related issue(s)

Fixes #998

[pkosiec]

Please see the review of #1008. Thanks!

[pkosiec]

LGTM but please apply comments before merge. Thanks!

[pkosiec]

Suggested change

	// SendEvent is no-op.
	// SendEvent sends an event to a configured server.

[pkosiec]

Is SendMessage used anywhere? I don't think so, so please remove it. Thanks!

[pkosiec]

This logic is invalid - we send events to all indexes and sources are not used. We should do

		if !sliceutil.Intersect(indexCfg.Bindings.Sources, eventSources) {
			continue
		}

instead.

Basically please use logic from SendMessage in SendEvent, and remove SendMessage. Thanks!

[huseyinbabal]

Actually, I removed SendMessage now, and each es flush index is for one source, so I removed source part