docker-compose: remove EFK (#170)

docker-compose: remove EFK Fetching EFK adds several minutes to the setup time and start-times. I think most hackers will want to just see the logs on the console, and be interested in EFK once things are more prod-like. Using EFK means the user actually has to do some initial config on their cluster, and it doesn't just work out of the box. One problem is OSIE wants to talk specifically to ElasticSearch, but I wonder if we could swap it out with syslog and a syslog listener?
tinkerbell · Jun 16, 2020 · ffd50a5 · ffd50a5
2 parents 527d14e + 6c66328
commit ffd50a5
Show file tree

Hide file tree

Showing 6 changed files with 3 additions and 161 deletions.
diff --git a/README.md b/README.md
@@ -10,8 +10,8 @@ It is comprised of following five major components:
 4.  An in-memory installation environment([osie](https://github.com/tinkerbell/osie))
 5.  A controller/handler of BMC interactions([pbnj](https://github.com/tinkerbell/pbnj))
 
-The workflow engine is comprised of a server and a CLI, which communicates over gRPC.  
- The CLI is used to create a workflow and its building blocks: templates and targeted hardware.
+The workflow engine is comprised of a server and a CLI, which communicates over gRPC.
+The CLI is used to create a workflow and its building blocks: templates and targeted hardware.
 
 ## Packet Workflow
 
@@ -34,9 +34,6 @@ A Packet Workflow is an open-source microservice that’s responsible for handli
     -   [Hegel](docs/components.md#hegel)
     -   [Database](docs/components.md#database)
     -   [Image Registry](docs/components.md#registry)
-    -   [Elasticsearch](docs/components.md#elastic)
-    -   [Fluent Bit](docs/components.md#fluent-bit)
-    -   [Kibana](docs/components.md#kibana)
 -   [Architecture](docs/architecture.md)
 -   [Say "Hello-World!" with a Workflow](docs/hello-world.md)
 -   [Concepts](docs/concepts.md)

diff --git a/deploy/docker-compose.yml b/deploy/docker-compose.yml
@@ -16,8 +16,6 @@ services:
       PGSSLMODE: disable
       PGUSER: tinkerbell
     depends_on:
-      fluentbit:
-        condition: service_started
       db:
         condition: service_healthy
     healthcheck:
@@ -27,10 +25,6 @@ services:
       retries: 30
     volumes:
       - ./state/certs:/certs/${FACILITY}
-    logging:
-      driver: fluentd
-      options:
-        tag: tink-server
     ports:
       - 42113:42113/tcp
       - 42114:42114/tcp
@@ -46,18 +40,11 @@ services:
       - ./db/tinkerbell-init.sql:/docker-entrypoint-initdb.d/tinkerbell-init.sql:ro
     ports:
       - 5432:5432
-    depends_on:
-      fluentbit:
-        condition: service_started
     healthcheck:
       test: ["CMD-SHELL", "pg_isready -U tinkerbell"]
       interval: 1s
       timeout: 1s
       retries: 30
-    logging:
-      driver: fluentd
-      options:
-        tag: db
 
   tink-cli:
     image: quay.io/tinkerbell/tink-cli:latest
@@ -94,13 +81,6 @@ services:
     volumes:
       - ./state/certs:/certs
       - ./state/registry:/var/lib/registry
-    depends_on:
-      fluentbit:
-        condition: service_started
-    logging:
-      driver: fluentd
-      options:
-        tag: registry
     network_mode: host
 
   boots:
@@ -132,14 +112,8 @@ services:
       TINKERBELL_CERT_URL: http://$TINKERBELL_HOST_IP:42114/cert
       ELASTIC_SEARCH_URL: $TINKERBELL_HOST_IP:9200
     depends_on:
-      fluentbit:
-        condition: service_started
       cacher:
         condition: service_started
-    logging:
-      driver: fluentd
-      options:
-        tag: boots
     ports:
       - $TINKERBELL_HOST_IP:80:80/tcp
       - 67:67/udp
@@ -153,41 +127,6 @@ services:
       - $TINKERBELL_NGINX_IP:80:80/tcp
     volumes:
       - ./state/webroot:/usr/share/nginx/html/
-    logging:
-      driver: fluentd
-      options:
-        tag: nginx
-
-  elasticsearch:
-    image: elasticsearch:7.3.0
-    restart: unless-stopped
-    ports:
-      - 9200:9200
-      - 9300:9300
-    environment:
-      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
-      - discovery.type=single-node
-
-  kibana:
-    image: kibana:7.3.0
-    depends_on:
-      - elasticsearch
-    restart: unless-stopped
-    environment:
-      ELASTICSEARCH_HOSTS: http://elasticsearch:9200
-    ports:
-      - 5601:5601
-
-  fluentbit:
-    image: fluent/fluent-bit:1.3
-    restart: unless-stopped
-    ports:
-      - 24224:24224
-      - 24224:24224/udp
-    depends_on:
-      - elasticsearch
-    volumes:
-      - ./fluent-bit.conf:/fluent-bit/etc/fluent-bit.conf:ro
 
   cacher:
     image: quay.io/packet/cacher:workflow
@@ -208,13 +147,7 @@ services:
       ROLLBAR_TOKEN: ${ROLLBAR_TOKEN-ignored}
     volumes:
       - ./state/certs:/certs/${FACILITY}
-    logging:
-      driver: fluentd
-      options:
-        tag: cacher
     depends_on:
-      fluentbit:
-        condition: service_started
       db:
         condition: service_healthy
     ports:
@@ -236,11 +169,5 @@ services:
       CACHER_CERT_URL: http://127.0.0.1:42112/cert
       CACHER_GRPC_AUTHORITY: 127.0.0.1:42111
     depends_on:
-      fluentbit:
-        condition: service_started
       cacher:
         condition: service_started
-    logging:
-      driver: fluentd
-      options:
-        tag: hegel
diff --git a/deploy/fluent-bit.conf b/deploy/fluent-bit.conf
diff --git a/docs/components.md b/docs/components.md
@@ -34,17 +34,3 @@ Depending on your use case, you can choose to use [Quay](https://quay.io/) or [D
 You can use the same registry to store all of the action images used for a workflow.
 
 On the other hand, if you want to keep things local, you can also setup a secure private Docker registry to hold all your images locally.
-
-### Fluent Bit
-
-[Fluent Bit](https://fluentbit.io/) is an open source and multi-platform Log Processor and Forwarder which allows you to collect data/logs from different sources, unify and send them to multiple destinations.
-The components write their logs to `stdout` and these logs are then collected by Fluent Bit and pushed to Elasticsearch.
-
-### Elasticsearch
-
-[Elasticsearch](https://www.elastic.co/) is a distributed, open source search and analytics engine for all types of data, including textual, numerical, geospatial, structured, and unstructured.
-Fluent Bit collects the logs from each component and pushes them into Elasticsearch for storage and analysis purposes.
-
-### Kibana
-
-[Kibana](https://www.elastic.co/kibana) lets you visualize your Elasticsearch data and navigate the Elastic Stack so you can do anything from tracking query load to understanding the way requests flow through your apps.
diff --git a/docs/concepts.md b/docs/concepts.md
@@ -69,9 +69,6 @@ A provisioner houses the following components:
 -   Boots
 -   Hegel
 -   Image Registry (optional)
--   Elasticsearch
--   Fluent Bit
--   Kibana
 -   NGINX
 
 It is up to you if you would like to divide these components into multiple servers.

diff --git a/setup.sh b/setup.sh
@@ -400,7 +400,6 @@ bootstrap_docker_registry() (
 	docker_login
 
 	docker_mirror_image "quay.io/tinkerbell/tink-worker:latest" "${TINKERBELL_HOST_IP}/tink-worker:latest"
-	docker_mirror_image "fluent/fluent-bit:1.3" "${TINKERBELL_HOST_IP}/fluent-bit:1.3"
 )
 
 setup_docker_registry() (
@@ -413,7 +412,7 @@ setup_docker_registry() (
 )
 
 start_components() (
-	local components=(db cacher hegel tink-server boots tink-cli nginx kibana)
+	local components=(db cacher hegel tink-server boots tink-cli nginx)
 	for comp in "${components[@]}"; do
 		docker-compose -f "$DEPLOYDIR/docker-compose.yml" up --build -d "$comp"
 		sleep 3