elk addon refactor #1312

ryane · 2016-03-31T21:02:16Z

Installs cleanly on a fresh build of most recent master branch
Upgrades cleanly from the most recent release *
Updates documentation relevant to the changes
kibana runs via mesos framework (by default)
elasticsearch framework 1.0.1 (in docker mode)
elasticsearch client node (load balancer) for improved resilience
uninstall playbook

Fixes #1208
Fixes #1207
Fixes #1205
Fixes #1201
Fixes #1198
Fixes #1197
Fixes #830

This updates the ELK addon. See `roles/elk/readme.rst' for updated documentation.

*Re-running the ELK addon playbook on a Mantl cluster running the 1.0.3 ELK addon is not supported without manual intervention. See the documentation for the details

Testing:

Review documentation
Install the addon (it can take 5+ minutes for the components to become healthy)
Verify Elasticsearch framework and Kibana UIs are available in Mantl UI
Verify that the following services are registered and healthy in consul:
- elasticsearch-client-mantl
- elasticsearch-executor-mantl
- elasticsearch-mantl
- kibana-mantl
- kibana-mantl-task
Verify that the following apps are running and healthy in Marathon (in mantl folder):
- elasticsearch
- elasticsearch-client
- kibana
Verify Kibana UI
- Verify that it does not prompt you to create an index pattern
- Verify that the default dashboard was created
- Verify that it is receiving data from all nodes
Verify Elasticsearch framework UI
- Verify that data size and # of documents are increasing
Uninstall addon (see docs)
- Verify all components above are removed

- kibana runs via a mesos framework (by default) - elasticsearch framework 1.0.1 (in docker mode) - elasticsearch client node (load balancer) for improved resilience Fixes #1208 Fixes #1207 Fixes #1205 Fixes #1201 Fixes #1198 Fixes #1197 Fixes #830

langston-barrett · 2016-04-01T19:22:24Z

langston-barrett · 2016-04-01T19:29:26Z

This isn't changed in this PR, but maybe we could add a typo fix? Current docs state

As an example, let’s say that you just wanted to stand up an ELK stack on a small cluster for evaluation purposes.

Which should probably be

As an example, let’s say that you just wanted to set up an ELK stack on a small cluster for evaluation purposes.

langston-barrett · 2016-04-01T20:14:34Z

I'm getting some errors on a fresh install on AWS, but I might just have too small instance sizes:

And ES dashboard shows no documents.

ryane · 2016-04-01T20:16:54Z

are the 3 apps healthy in marathon? how long has it been since you installed?

langston-barrett · 2016-04-01T20:17:48Z

@ryane Elasticsearch client is waiting, which makes me think about instance size. About 15min.

langston-barrett · 2016-04-01T20:18:58Z

@ryane Actually, the elasticsearch-client has been killed in mesos a few times with the following logs:

I0401 19:59:29.100385 28467 exec.cpp:134] Version: 0.25.0
I0401 19:59:29.103816 28468 exec.cpp:208] Executor registered on slave 6b90c4a5-c84f-413c-bdfa-7df215612823-S0
Warning: '-c' is deprecated, it will be replaced by '--cpu-shares' soon. See usage.
+ export ELASTICSEARCH_SERVICE=transport_port.elasticsearch-executor-mantl
+ ELASTICSEARCH_SERVICE=transport_port.elasticsearch-executor-mantl
+ export ELASTICSEARCH_CLUSTER_NAME=mantl
+ ELASTICSEARCH_CLUSTER_NAME=mantl
+ CONSUL_TEMPLATE=/usr/local/bin/consul-template
+ CONSUL_CONFIG=/consul-template/config.d
+ CONSUL_CONNECT=consul.service.consul:8500
+ CONSUL_MINWAIT=30s
+ CONSUL_MAXWAIT=60s
+ CONSUL_LOGLEVEL=warn
+ CONSUL_SSL_VERIFY=false
++ getent hosts lb0-worker-02.node.consul
++ awk '{ print $1 ; exit }'
+ export PUBLISH_HOST=10.1.2.176
+ PUBLISH_HOST=10.1.2.176
+ [[ -n consul.service.consul:8500 ]]
+ ctargs=' -consul consul.service.consul:8500'
+ [[ -n true ]]
+ ctargs=' -consul consul.service.consul:8500 -ssl'
+ [[ -n true ]]
+ ctargs=' -consul consul.service.consul:8500 -ssl -ssl-verify=false'
+ [[ -n '' ]]
+ ATTEMPT=0
+ wait_for_service
+ /usr/local/bin/consul-template -config /consul-template/config.d -log-level warn -wait 30s:60s -once -consul consul.service.consul:8500 -ssl -ssl-verify=false
2016/04/01 19:59:29 [WARN] (runner) disabling consul SSL verification
+ grep discovery.zen.ping.unicast.hosts /usr/share/elasticsearch/config/elasticsearch.yml
+ '[' 0 -eq 5 ']'
+ echo 'waiting for transport_port.elasticsearch-executor-mantl service...'
+ echo 'attempt: 0'
+ sleep 30
+ wait_for_service
+ /usr/local/bin/consul-template -config /consul-template/config.d -log-level warn -wait 30s:60s -once -consul consul.service.consul:8500 -ssl -ssl-verify=false
2016/04/01 20:00:29 [WARN] (runner) disabling consul SSL verification
+ grep discovery.zen.ping.unicast.hosts /usr/share/elasticsearch/config/elasticsearch.yml
+ '[' 1 -eq 5 ']'
+ echo 'waiting for transport_port.elasticsearch-executor-mantl service...'
+ echo 'attempt: 1'
+ sleep 30
+ wait_for_service
+ /usr/local/bin/consul-template -config /consul-template/config.d -log-level warn -wait 30s:60s -once -consul consul.service.consul:8500 -ssl -ssl-verify=false
2016/04/01 20:01:29 [WARN] (runner) disabling consul SSL verification
+ grep discovery.zen.ping.unicast.hosts /usr/share/elasticsearch/config/elasticsearch.yml
+ '[' 2 -eq 5 ']'
+ echo 'waiting for transport_port.elasticsearch-executor-mantl service...'
+ echo 'attempt: 2'
+ sleep 30
+ wait_for_service
+ /usr/local/bin/consul-template -config /consul-template/config.d -log-level warn -wait 30s:60s -once -consul consul.service.consul:8500 -ssl -ssl-verify=false
2016/04/01 20:02:29 [WARN] (runner) disabling consul SSL verification

--container="mesos-6b90c4a5-c84f-413c-bdfa-7df215612823-S0.5cf7bc13-2d7e-4cf5-92f3-759412defc15" --docker="docker" --docker_socket="/var/run/docker.sock" --help="false" --initialize_driver_logging="true" --logbufsecs="0" --logging_level="INFO" --mapped_directory="/mnt/mesos/sandbox" --quiet="false" --sandbox_directory="/var/lib/mesos/slaves/6b90c4a5-c84f-413c-bdfa-7df215612823-S0/frameworks/6b90c4a5-c84f-413c-bdfa-7df215612823-0000/executors/mantl_elasticsearch-client.3e28e4b8-f844-11e5-9100-328337229d45/runs/5cf7bc13-2d7e-4cf5-92f3-759412defc15" --stop_timeout="0ns"
--container="mesos-6b90c4a5-c84f-413c-bdfa-7df215612823-S0.5cf7bc13-2d7e-4cf5-92f3-759412defc15" --docker="docker" --docker_socket="/var/run/docker.sock" --help="false" --initialize_driver_logging="true" --logbufsecs="0" --logging_level="INFO" --mapped_directory="/mnt/mesos/sandbox" --quiet="false" --sandbox_directory="/var/lib/mesos/slaves/6b90c4a5-c84f-413c-bdfa-7df215612823-S0/frameworks/6b90c4a5-c84f-413c-bdfa-7df215612823-0000/executors/mantl_elasticsearch-client.3e28e4b8-f844-11e5-9100-328337229d45/runs/5cf7bc13-2d7e-4cf5-92f3-759412defc15" --stop_timeout="0ns"
Registered docker executor on lb0-worker-02.node.consul
Starting task mantl_elasticsearch-client.3e28e4b8-f844-11e5-9100-328337229d45
waiting for transport_port.elasticsearch-executor-mantl service...
attempt: 0
waiting for transport_port.elasticsearch-executor-mantl service...
attempt: 1
waiting for transport_port.elasticsearch-executor-mantl service...
attempt: 2
Killing docker task
Shutting down

langston-barrett · 2016-04-01T20:20:41Z

I might not have enough CPUs in Mesos:

        CPUs    Mem
Total   2   5.0 GB
Used    1.9 2.0 GB
Offered 0   0 B
Idle    0.1 3.0 GB

ryane · 2016-04-01T20:20:46Z

ah, yeah, we might have to bump up the sample config defaults. I deployed on aws with 4 worker nodes with m3.large. what do you have running?

langston-barrett · 2016-04-01T20:21:12Z

@ryane m3.medium :/

https://github.com/CiscoCloud/mantl/blob/ca4798db2f0374b02af8f40708b6f63205eccd9f/terraform/aws/instance/main.tf#L3

https://github.com/CiscoCloud/mantl/blob/ca4798db2f0374b02af8f40708b6f63205eccd9f/terraform/aws.sample.tf#L27-L29

I'll test again with m3.large.

ryane · 2016-04-01T20:30:30Z

cool, the current aws.sample.tf has

variable "worker_count" { default = 4 }
variable "worker_type" { default = "m3.large" }

which should be enough.

langston-barrett · 2016-04-01T21:45:55Z

I'm getting prompted to configure an index pattern in the Kibana UI, about 15min after installing. I'll update if this changes later.

ryane · 2016-04-01T22:19:35Z

Yeah, that's too long - not sure why that is happening. Running a test on AWS to see if I can repro...

langston-barrett · 2016-04-01T22:35:06Z

@ryane I'll add your keys to my cluster ASAP.

ryane · 2016-04-01T23:35:28Z

ok, let me know. I could not repro on a new AWS cluster. the sandbox logs for the kibana-mantl.task task might be helpful. Also, you could try an uninstall / reinstall to see if the problem is consistent.

langston-barrett · 2016-04-02T00:22:09Z

@ryane You should now have SSH access to the nodes as well.

Logs stderr:

I0401 21:35:59.531289 22398 exec.cpp:134] Version: 0.25.0
I0401 21:35:59.535097 22402 exec.cpp:208] Executor registered on slave 1f357e60-4f29-45dc-8dcb-b8c9b5b09b6a-S0
Warning: '-c' is deprecated, it will be replaced by '--cpu-shares' soon. See usage.
+ exec java -Xms32m -Xmx128m -jar /tmp/mesosframework.jar --spring.application.name=kibana-mantl --mesos.framework.name=kibana-mantl --mesos.master=zk://zookeeper.service.consul:2181/mesos --mesos.zookeeper.server=zookeeper.service.consul:2181 --mesos.principal=mantl-api --mesos.secret=t2+XWg1sM4Uq2CGUi7uTFF1t5yWUlKI/iXgM9OtHVF8y --mesos.resources.cpus=0.5 --mesos.resources.mem=512 --mesos.resources.count=1 --mesos.resources.ports.UI_5601.host=ANY --mesos.resources.ports.UI_5601.container=5601 --mesos.docker.image=ciscocloud/mantl-kibana:4.3.2 --mesos.docker.network=BRIDGE '--mesos.command=export ELASTICSEARCH_SERVICE=elasticsearch-client-mantl; export KIBANA_SERVICE=kibana-mantl-task; export CONSUL_SSL=true; export CONSUL_SSL_VERIFY=false; tini -s -- /launch.sh' --logging.level.com.containersolutions.mesos=WARN --elasticsearch.http=http://elasticsearch-executor.service.consul:4000 --server.port=31100
2016-04-01 21:36:10,580:1(0x7fab94b18700):ZOO_INFO@log_env@712: Client environment:zookeeper.version=zookeeper C client 3.4.5
2016-04-01 21:36:10,580:1(0x7fab94b18700):ZOO_INFO@log_env@716: Client environment:host.name=ip-10-1-2-38.us-west-2.compute.internal
2016-04-01 21:36:10,580:1(0x7fab94b18700):ZOO_INFO@log_env@723: Client environment:os.name=Linux
2016-04-01 21:36:10,580:1(0x7fab94b18700):ZOO_INFO@log_env@724: Client environment:os.arch=3.10.0-327.13.1.el7.x86_64
2016-04-01 21:36:10,580:1(0x7fab94b18700):ZOO_INFO@log_env@725: Client environment:os.version=#1 SMP Thu Mar 31 16:04:38 UTC 2016
2016-04-01 21:36:10,580:1(0x7fab94b18700):ZOO_INFO@log_env@733: Client environment:user.name=(null)
2016-04-01 21:36:10,581:1(0x7fab94b18700):ZOO_INFO@log_env@741: Client environment:user.home=/root
2016-04-01 21:36:10,581:1(0x7fab94b18700):ZOO_INFO@log_env@753: Client environment:user.dir=/
2016-04-01 21:36:10,581:1(0x7fab94b18700):ZOO_INFO@zookeeper_init@786: Initiating client connection, host=zookeeper.service.consul:2181 sessionTimeout=1000 watcher=0x7fab9f4ad600 sessionId=0 sessionPasswd=<null> context=0x7fab980034c0 flags=0
2016-04-01 21:36:10,593:1(0x7fab90b10700):ZOO_INFO@check_events@1703: initiated connection to server [10.1.1.57:2181]
2016-04-01 21:36:10,600:1(0x7fab90b10700):ZOO_INFO@check_events@1750: session establishment complete on server [10.1.1.57:2181], sessionId=0x153d3bd6c9a000c, negotiated timeout=4000
2016-04-01 21:36:13,273:1(0x7fab90b10700):ZOO_WARN@zookeeper_interest@1557: Exceeded deadline by 16ms
I0401 21:36:14.993381    16 sched.cpp:164] Version: 0.25.0
2016-04-01 21:36:14,996:1(0x7fab93b16700):ZOO_INFO@log_env@712: Client environment:zookeeper.version=zookeeper C client 3.4.5
2016-04-01 21:36:14,996:1(0x7fab93b16700):ZOO_INFO@log_env@716: Client environment:host.name=ip-10-1-2-38.us-west-2.compute.internal
2016-04-01 21:36:14,996:1(0x7fab93b16700):ZOO_INFO@log_env@723: Client environment:os.name=Linux
2016-04-01 21:36:14,996:1(0x7fab93b16700):ZOO_INFO@log_env@724: Client environment:os.arch=3.10.0-327.13.1.el7.x86_64
2016-04-01 21:36:14,996:1(0x7fab93b16700):ZOO_INFO@log_env@725: Client environment:os.version=#1 SMP Thu Mar 31 16:04:38 UTC 2016
2016-04-01 21:36:14,997:1(0x7fab93b16700):ZOO_INFO@log_env@733: Client environment:user.name=(null)
2016-04-01 21:36:14,997:1(0x7fab93b16700):ZOO_INFO@log_env@741: Client environment:user.home=/root
2016-04-01 21:36:14,997:1(0x7fab93b16700):ZOO_INFO@log_env@753: Client environment:user.dir=/
2016-04-01 21:36:14,997:1(0x7fab93b16700):ZOO_INFO@zookeeper_init@786: Initiating client connection, host=zookeeper.service.consul:2181 sessionTimeout=10000 watcher=0x7fab9f4ad600 sessionId=0 sessionPasswd=<null> context=0x7fabb8000f10 flags=0
2016-04-01 21:36:15,017:1(0x7fab8f108700):ZOO_INFO@check_events@1703: initiated connection to server [10.1.3.103:2181]
2016-04-01 21:36:15,025:1(0x7fab8f108700):ZOO_INFO@check_events@1750: session establishment complete on server [10.1.3.103:2181], sessionId=0x353d3bd6c94000e, negotiated timeout=10000
I0401 21:36:15.026108    26 group.cpp:331] Group process (group(1)@10.1.2.38:54212) connected to ZooKeeper
I0401 21:36:15.026140    26 group.cpp:805] Syncing group operations: queue size (joins, cancels, datas) = (0, 0, 0)
I0401 21:36:15.026151    26 group.cpp:403] Trying to create path '/mesos' in ZooKeeper
I0401 21:36:15.035145    26 detector.cpp:156] Detected a new leader: (id='4')
I0401 21:36:15.035243    26 group.cpp:674] Trying to get '/mesos/json.info_0000000004' in ZooKeeper
I0401 21:36:15.039320    26 detector.cpp:481] A new leading master ([email protected]:15050) is detected
I0401 21:36:15.039394    26 sched.cpp:262] New master detected at [email protected]:15050
I0401 21:36:15.039559    26 sched.cpp:318] Authenticating with master [email protected]:15050
I0401 21:36:15.039580    26 sched.cpp:325] Using default CRAM-MD5 authenticatee
I0401 21:36:15.039824    26 authenticatee.cpp:91] Initializing client SASL
I0401 21:36:15.040922    26 authenticatee.cpp:115] Creating new client SASL connection
I0401 21:36:15.042431    26 authenticatee.cpp:206] Received SASL authentication mechanisms: CRAM-MD5
I0401 21:36:15.042460    26 authenticatee.cpp:232] Attempting to authenticate with mechanism 'CRAM-MD5'
I0401 21:36:15.043131    26 authenticatee.cpp:252] Received SASL authentication step
I0401 21:36:15.043766    26 authenticatee.cpp:292] Authentication success
I0401 21:36:15.043833    26 sched.cpp:407] Successfully authenticated with master [email protected]:15050
I0401 21:36:15.045148    26 sched.cpp:641] Framework registered with 1f357e60-4f29-45dc-8dcb-b8c9b5b09b6a-0001
2016-04-01 21:36:58,430:1(0x7fab8f108700):ZOO_WARN@zookeeper_interest@1557: Exceeded deadline by 13ms

Stdout:

--container="mesos-1f357e60-4f29-45dc-8dcb-b8c9b5b09b6a-S0.aab5ab33-1c8b-4222-b71f-16c5e3046d0d" --docker="docker" --docker_socket="/var/run/docker.sock" --help="false" --initialize_driver_logging="true" --logbufsecs="0" --logging_level="INFO" --mapped_directory="/mnt/mesos/sandbox" --quiet="false" --sandbox_directory="/var/lib/mesos/slaves/1f357e60-4f29-45dc-8dcb-b8c9b5b09b6a-S0/frameworks/687233fa-73e8-4e17-9e89-542469ba5d3c-0000/executors/mantl_kibana.a17a3bf1-f851-11e5-aee4-32bca1230bf7/runs/aab5ab33-1c8b-4222-b71f-16c5e3046d0d" --stop_timeout="0ns"
--container="mesos-1f357e60-4f29-45dc-8dcb-b8c9b5b09b6a-S0.aab5ab33-1c8b-4222-b71f-16c5e3046d0d" --docker="docker" --docker_socket="/var/run/docker.sock" --help="false" --initialize_driver_logging="true" --logbufsecs="0" --logging_level="INFO" --mapped_directory="/mnt/mesos/sandbox" --quiet="false" --sandbox_directory="/var/lib/mesos/slaves/1f357e60-4f29-45dc-8dcb-b8c9b5b09b6a-S0/frameworks/687233fa-73e8-4e17-9e89-542469ba5d3c-0000/executors/mantl_kibana.a17a3bf1-f851-11e5-aee4-32bca1230bf7/runs/aab5ab33-1c8b-4222-b71f-16c5e3046d0d" --stop_timeout="0ns"
Registered docker executor on lb0-worker-02.node.consul
Starting task mantl_kibana.a17a3bf1-f851-11e5-aee4-32bca1230bf7

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::        (v1.3.3.RELEASE)

2016-04-01 21:36:02.519  INFO 1 --- [           main] ationConfigEmbeddedWebApplicationContext : Refreshing org.springframework.boot.context.embedded.AnnotationConfigEmbeddedWebApplicationContext@2617070e: startup date [Fri Apr 01 21:36:02 UTC 2016]; root of context hierarchy
2016-04-01 21:36:05.667  INFO 1 --- [           main] o.s.b.f.s.DefaultListableBeanFactory     : Overriding bean definition for bean 'beanNameViewResolver' with a different definition: replacing [Root bean: class [null]; scope=; abstract=false; lazyInit=false; autowireMode=3; dependencyCheck=0; autowireCandidate=true; primary=false; factoryBeanName=org.springframework.boot.autoconfigure.web.ErrorMvcAutoConfiguration$WhitelabelErrorViewConfiguration; factoryMethodName=beanNameViewResolver; initMethodName=null; destroyMethodName=(inferred); defined in class path resource [org/springframework/boot/autoconfigure/web/ErrorMvcAutoConfiguration$WhitelabelErrorViewConfiguration.class]] with [Root bean: class [null]; scope=; abstract=false; lazyInit=false; autowireMode=3; dependencyCheck=0; autowireCandidate=true; primary=false; factoryBeanName=org.springframework.boot.autoconfigure.web.WebMvcAutoConfiguration$WebMvcAutoConfigurationAdapter; factoryMethodName=beanNameViewResolver; initMethodName=null; destroyMethodName=(inferred); defined in class path resource [org/springframework/boot/autoconfigure/web/WebMvcAutoConfiguration$WebMvcAutoConfigurationAdapter.class]]
2016-04-01 21:36:08.007  INFO 1 --- [           main] s.b.c.e.t.TomcatEmbeddedServletContainer : Tomcat initialized with port(s): 31100 (http)
2016-04-01 21:36:08.102  INFO 1 --- [           main] o.apache.catalina.core.StandardService   : Starting service Tomcat
2016-04-01 21:36:08.119  INFO 1 --- [           main] org.apache.catalina.core.StandardEngine  : Starting Servlet Engine: Apache Tomcat/8.0.32
2016-04-01 21:36:08.308  INFO 1 --- [ost-startStop-1] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring embedded WebApplicationContext
2016-04-01 21:36:08.308  INFO 1 --- [ost-startStop-1] o.s.web.context.ContextLoader            : Root WebApplicationContext: initialization completed in 5801 ms
2016-04-01 21:36:09.681  INFO 1 --- [ost-startStop-1] o.s.b.c.e.ServletRegistrationBean        : Mapping servlet: 'dispatcherServlet' to [/]
2016-04-01 21:36:09.689  INFO 1 --- [ost-startStop-1] o.s.b.c.embedded.FilterRegistrationBean  : Mapping filter: 'metricFilter' to: [/*]
2016-04-01 21:36:09.689  INFO 1 --- [ost-startStop-1] o.s.b.c.embedded.FilterRegistrationBean  : Mapping filter: 'characterEncodingFilter' to: [/*]
2016-04-01 21:36:09.690  INFO 1 --- [ost-startStop-1] o.s.b.c.embedded.FilterRegistrationBean  : Mapping filter: 'hiddenHttpMethodFilter' to: [/*]
2016-04-01 21:36:09.690  INFO 1 --- [ost-startStop-1] o.s.b.c.embedded.FilterRegistrationBean  : Mapping filter: 'httpPutFormContentFilter' to: [/*]
2016-04-01 21:36:09.690  INFO 1 --- [ost-startStop-1] o.s.b.c.embedded.FilterRegistrationBean  : Mapping filter: 'requestContextFilter' to: [/*]
2016-04-01 21:36:09.690  INFO 1 --- [ost-startStop-1] o.s.b.c.embedded.FilterRegistrationBean  : Mapping filter: 'webRequestLoggingFilter' to: [/*]
2016-04-01 21:36:09.690  INFO 1 --- [ost-startStop-1] o.s.b.c.embedded.FilterRegistrationBean  : Mapping filter: 'applicationContextIdFilter' to: [/*]
2016-04-01 21:36:10.301  INFO 1 --- [ost-startStop-1] o.a.c.util.SessionIdGeneratorBase        : Creation of SecureRandom instance for session ID generation using [SHA1PRNG] took [575] milliseconds.
2016-04-01 21:36:11.850  INFO 1 --- [           main] s.w.s.m.m.a.RequestMappingHandlerAdapter : Looking for @ControllerAdvice: org.springframework.boot.context.embedded.AnnotationConfigEmbeddedWebApplicationContext@2617070e: startup date [Fri Apr 01 21:36:02 UTC 2016]; root of context hierarchy
2016-04-01 21:36:12.151  INFO 1 --- [           main] s.w.s.m.m.a.RequestMappingHandlerMapping : Mapped "{[/mesos/resources/count],methods=[POST],consumes=[text/plain]}" onto public org.springframework.http.ResponseEntity<java.lang.Void> com.containersolutions.mesos.controllers.CountController.countEndpointPost(java.lang.String)
2016-04-01 21:36:12.166  INFO 1 --- [           main] s.w.s.m.m.a.RequestMappingHandlerMapping : Mapped "{[/mesos/resources/count],methods=[GET],produces=[text/plain]}" onto public org.springframework.http.ResponseEntity<java.lang.String> com.containersolutions.mesos.controllers.CountController.countEndpointGet()
2016-04-01 21:36:12.171  INFO 1 --- [           main] s.w.s.m.m.a.RequestMappingHandlerMapping : Mapped "{[/error],produces=[text/html]}" onto public org.springframework.web.servlet.ModelAndView org.springframework.boot.autoconfigure.web.BasicErrorController.errorHtml(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse)
2016-04-01 21:36:12.171  INFO 1 --- [           main] s.w.s.m.m.a.RequestMappingHandlerMapping : Mapped "{[/error]}" onto public org.springframework.http.ResponseEntity<java.util.Map<java.lang.String, java.lang.Object>> org.springframework.boot.autoconfigure.web.BasicErrorController.error(javax.servlet.http.HttpServletRequest)
2016-04-01 21:36:12.258  INFO 1 --- [           main] o.s.w.s.handler.SimpleUrlHandlerMapping  : Mapped URL path [/webjars/**] onto handler of type [class org.springframework.web.servlet.resource.ResourceHttpRequestHandler]
2016-04-01 21:36:12.258  INFO 1 --- [           main] o.s.w.s.handler.SimpleUrlHandlerMapping  : Mapped URL path [/**] onto handler of type [class org.springframework.web.servlet.resource.ResourceHttpRequestHandler]
2016-04-01 21:36:12.414  INFO 1 --- [           main] o.s.w.s.handler.SimpleUrlHandlerMapping  : Mapped URL path [/**/favicon.ico] onto handler of type [class org.springframework.web.servlet.resource.ResourceHttpRequestHandler]
2016-04-01 21:36:13.901  INFO 1 --- [           main] o.s.b.a.e.mvc.EndpointHandlerMapping     : Mapped "{[/configprops || /configprops.json],methods=[GET],produces=[application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.mvc.EndpointMvcAdapter.invoke()
2016-04-01 21:36:13.906  INFO 1 --- [           main] o.s.b.a.e.mvc.EndpointHandlerMapping     : Mapped "{[/env/{name:.*}],methods=[GET],produces=[application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.mvc.EnvironmentMvcEndpoint.value(java.lang.String)
2016-04-01 21:36:13.906  INFO 1 --- [           main] o.s.b.a.e.mvc.EndpointHandlerMapping     : Mapped "{[/env || /env.json],methods=[GET],produces=[application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.mvc.EndpointMvcAdapter.invoke()
2016-04-01 21:36:13.915  INFO 1 --- [           main] o.s.b.a.e.mvc.EndpointHandlerMapping     : Mapped "{[/metrics/{name:.*}],methods=[GET],produces=[application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.mvc.MetricsMvcEndpoint.value(java.lang.String)
2016-04-01 21:36:13.915  INFO 1 --- [           main] o.s.b.a.e.mvc.EndpointHandlerMapping     : Mapped "{[/metrics || /metrics.json],methods=[GET],produces=[application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.mvc.EndpointMvcAdapter.invoke()
2016-04-01 21:36:13.916  INFO 1 --- [           main] o.s.b.a.e.mvc.EndpointHandlerMapping     : Mapped "{[/trace || /trace.json],methods=[GET],produces=[application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.mvc.EndpointMvcAdapter.invoke()
2016-04-01 21:36:13.916  INFO 1 --- [           main] o.s.b.a.e.mvc.EndpointHandlerMapping     : Mapped "{[/dump || /dump.json],methods=[GET],produces=[application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.mvc.EndpointMvcAdapter.invoke()
2016-04-01 21:36:13.918  INFO 1 --- [           main] o.s.b.a.e.mvc.EndpointHandlerMapping     : Mapped "{[/mappings || /mappings.json],methods=[GET],produces=[application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.mvc.EndpointMvcAdapter.invoke()
2016-04-01 21:36:13.919  INFO 1 --- [           main] o.s.b.a.e.mvc.EndpointHandlerMapping     : Mapped "{[/info || /info.json],methods=[GET],produces=[application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.mvc.EndpointMvcAdapter.invoke()
2016-04-01 21:36:13.920  INFO 1 --- [           main] o.s.b.a.e.mvc.EndpointHandlerMapping     : Mapped "{[/health || /health.json],produces=[application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.mvc.HealthMvcEndpoint.invoke(java.security.Principal)
2016-04-01 21:36:13.920  INFO 1 --- [           main] o.s.b.a.e.mvc.EndpointHandlerMapping     : Mapped "{[/beans || /beans.json],methods=[GET],produces=[application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.mvc.EndpointMvcAdapter.invoke()
2016-04-01 21:36:13.921  INFO 1 --- [           main] o.s.b.a.e.mvc.EndpointHandlerMapping     : Mapped "{[/autoconfig || /autoconfig.json],methods=[GET],produces=[application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.mvc.EndpointMvcAdapter.invoke()
2016-04-01 21:36:14.316  INFO 1 --- [           main] o.s.j.e.a.AnnotationMBeanExporter        : Registering beans for JMX exposure on startup
2016-04-01 21:36:14.342  INFO 1 --- [           main] o.s.j.e.a.AnnotationMBeanExporter        : Bean with name 'instanceCount' has been autodetected for JMX exposure
2016-04-01 21:36:14.371  INFO 1 --- [           main] o.s.j.e.a.AnnotationMBeanExporter        : Located managed bean 'instanceCount': registering with JMX server as MBean [com.containersolutions.mesos.scheduler:name=instanceCount,type=InstanceCount]
2016-04-01 21:36:14.418  INFO 1 --- [           main] o.s.c.support.DefaultLifecycleProcessor  : Starting beans in phase 0
2016-04-01 21:36:14.859  INFO 1 --- [           main] s.b.c.e.t.TomcatEmbeddedServletContainer : Tomcat started on port(s): 31100 (http)
2016-04-01 21:36:15.155  INFO 1 --- [io-31100-exec-1] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring FrameworkServlet 'dispatcherServlet'
2016-04-01 21:36:15.155  INFO 1 --- [io-31100-exec-1] o.s.web.servlet.DispatcherServlet        : FrameworkServlet 'dispatcherServlet': initialization started
2016-04-01 21:36:15.224  INFO 1 --- [io-31100-exec-1] o.s.web.servlet.DispatcherServlet        : FrameworkServlet 'dispatcherServlet': initialization completed in 69 ms

langston-barrett · 2016-04-02T00:34:35Z

addons/elk-uninstall.yml

+
+  tasks:
+
+    - include_vars: ../roles/elk/defaults/main.yml


Should we also check if the user has a customized vars file and include that?

hm, I'm not sure how we'd know if they used one though. but you should just be able to pass one in with -e

langston-barrett · 2016-04-02T00:53:49Z

Reinstall doesn't work for the Kibana dashboard:
stderr:

I0402 00:46:37.879225 27417 exec.cpp:134] Version: 0.25.0
I0402 00:46:37.882338 27422 exec.cpp:208] Executor registered on slave 1f357e60-4f29-45dc-8dcb-b8c9b5b09b6a-S1
Warning: '-c' is deprecated, it will be replaced by '--cpu-shares' soon. See usage.
+ exec java -Xms32m -Xmx128m -jar /tmp/mesosframework.jar --spring.application.name=kibana-mantl --mesos.framework.name=kibana-mantl --mesos.master=zk://zookeeper.service.consul:2181/mesos --mesos.zookeeper.server=zookeeper.service.consul:2181 --mesos.principal=mantl-api --mesos.secret=t2+XWg1sM4Uq2CGUi7uTFF1t5yWUlKI/iXgM9OtHVF8y --mesos.resources.cpus=0.5 --mesos.resources.mem=512 --mesos.resources.count=1 --mesos.resources.ports.UI_5601.host=ANY --mesos.resources.ports.UI_5601.container=5601 --mesos.docker.image=ciscocloud/mantl-kibana:4.3.2 --mesos.docker.network=BRIDGE '--mesos.command=export ELASTICSEARCH_SERVICE=elasticsearch-client-mantl; export KIBANA_SERVICE=kibana-mantl-task; export CONSUL_SSL=true; export CONSUL_SSL_VERIFY=false; tini -s -- /launch.sh' --logging.level.com.containersolutions.mesos=WARN --elasticsearch.http=http://elasticsearch-executor.service.consul:4000 --server.port=31100
2016-04-02 00:46:44,804:1(0x7fba0e7fc700):ZOO_INFO@log_env@712: Client environment:zookeeper.version=zookeeper C client 3.4.5
2016-04-02 00:46:44,804:1(0x7fba0e7fc700):ZOO_INFO@log_env@716: Client environment:host.name=ip-10-1-1-70.us-west-2.compute.internal
2016-04-02 00:46:44,804:1(0x7fba0e7fc700):ZOO_INFO@log_env@723: Client environment:os.name=Linux
2016-04-02 00:46:44,804:1(0x7fba0e7fc700):ZOO_INFO@log_env@724: Client environment:os.arch=3.10.0-327.13.1.el7.x86_64
2016-04-02 00:46:44,804:1(0x7fba0e7fc700):ZOO_INFO@log_env@725: Client environment:os.version=#1 SMP Thu Mar 31 16:04:38 UTC 2016
2016-04-02 00:46:44,805:1(0x7fba0e7fc700):ZOO_INFO@log_env@733: Client environment:user.name=(null)
2016-04-02 00:46:44,805:1(0x7fba0e7fc700):ZOO_INFO@log_env@741: Client environment:user.home=/root
2016-04-02 00:46:44,805:1(0x7fba0e7fc700):ZOO_INFO@log_env@753: Client environment:user.dir=/
2016-04-02 00:46:44,805:1(0x7fba0e7fc700):ZOO_INFO@zookeeper_init@786: Initiating client connection, host=zookeeper.service.consul:2181 sessionTimeout=1000 watcher=0x7fba1e398600 sessionId=0 sessionPasswd=<null> context=0x7fba100034c0 flags=0
2016-04-02 00:46:44,842:1(0x7fba0b7f6700):ZOO_INFO@check_events@1703: initiated connection to server [10.1.2.41:2181]
2016-04-02 00:46:44,859:1(0x7fba0b7f6700):ZOO_INFO@check_events@1750: session establishment complete on server [10.1.2.41:2181], sessionId=0x253d3bd6c930017, negotiated timeout=4000
I0402 00:46:47.166525    16 sched.cpp:164] Version: 0.25.0
2016-04-02 00:46:47,171:1(0x7fba0ffff700):ZOO_INFO@log_env@712: Client environment:zookeeper.version=zookeeper C client 3.4.5
2016-04-02 00:46:47,171:1(0x7fba0ffff700):ZOO_INFO@log_env@716: Client environment:host.name=ip-10-1-1-70.us-west-2.compute.internal
2016-04-02 00:46:47,171:1(0x7fba0ffff700):ZOO_INFO@log_env@723: Client environment:os.name=Linux
2016-04-02 00:46:47,171:1(0x7fba0ffff700):ZOO_INFO@log_env@724: Client environment:os.arch=3.10.0-327.13.1.el7.x86_64
2016-04-02 00:46:47,171:1(0x7fba0ffff700):ZOO_INFO@log_env@725: Client environment:os.version=#1 SMP Thu Mar 31 16:04:38 UTC 2016
2016-04-02 00:46:47,171:1(0x7fba0ffff700):ZOO_INFO@log_env@733: Client environment:user.name=(null)
2016-04-02 00:46:47,171:1(0x7fba0ffff700):ZOO_INFO@log_env@741: Client environment:user.home=/root
2016-04-02 00:46:47,171:1(0x7fba0ffff700):ZOO_INFO@log_env@753: Client environment:user.dir=/
2016-04-02 00:46:47,171:1(0x7fba0ffff700):ZOO_INFO@zookeeper_init@786: Initiating client connection, host=zookeeper.service.consul:2181 sessionTimeout=10000 watcher=0x7fba1e398600 sessionId=0 sessionPasswd=<null> context=0x7fba34000f30 flags=0
2016-04-02 00:46:47,185:1(0x7fba09ff0700):ZOO_INFO@check_events@1703: initiated connection to server [10.1.1.57:2181]
2016-04-02 00:46:47,193:1(0x7fba09ff0700):ZOO_INFO@check_events@1750: session establishment complete on server [10.1.1.57:2181], sessionId=0x153d3bd6c9a0012, negotiated timeout=10000
I0402 00:46:47.193224    23 group.cpp:331] Group process (group(1)@10.1.1.70:40560) connected to ZooKeeper
I0402 00:46:47.193259    23 group.cpp:805] Syncing group operations: queue size (joins, cancels, datas) = (0, 0, 0)
I0402 00:46:47.193274    23 group.cpp:403] Trying to create path '/mesos' in ZooKeeper
I0402 00:46:47.196337    23 detector.cpp:156] Detected a new leader: (id='4')
I0402 00:46:47.196456    23 group.cpp:674] Trying to get '/mesos/json.info_0000000004' in ZooKeeper
I0402 00:46:47.198940    23 detector.cpp:481] A new leading master ([email protected]:15050) is detected
I0402 00:46:47.199041    23 sched.cpp:262] New master detected at [email protected]:15050
I0402 00:46:47.199239    23 sched.cpp:318] Authenticating with master [email protected]:15050
I0402 00:46:47.199268    23 sched.cpp:325] Using default CRAM-MD5 authenticatee
I0402 00:46:47.199550    23 authenticatee.cpp:91] Initializing client SASL
I0402 00:46:47.200953    23 authenticatee.cpp:115] Creating new client SASL connection
I0402 00:46:47.204387    22 authenticatee.cpp:206] Received SASL authentication mechanisms: CRAM-MD5
I0402 00:46:47.204423    22 authenticatee.cpp:232] Attempting to authenticate with mechanism 'CRAM-MD5'
I0402 00:46:47.206125    22 authenticatee.cpp:252] Received SASL authentication step
I0402 00:46:47.208884    22 authenticatee.cpp:292] Authentication success
I0402 00:46:47.208976    22 sched.cpp:407] Successfully authenticated with master [email protected]:15050
I0402 00:46:47.212453    22 sched.cpp:641] Framework registered with 1f357e60-4f29-45dc-8dcb-b8c9b5b09b6a-0003

stdout:

--container="mesos-1f357e60-4f29-45dc-8dcb-b8c9b5b09b6a-S1.e64d9a4f-8b0d-493d-aea1-9b7ff4157200" --docker="docker" --docker_socket="/var/run/docker.sock" --help="false" --initialize_driver_logging="true" --logbufsecs="0" --logging_level="INFO" --mapped_directory="/mnt/mesos/sandbox" --quiet="false" --sandbox_directory="/var/lib/mesos/slaves/1f357e60-4f29-45dc-8dcb-b8c9b5b09b6a-S1/frameworks/687233fa-73e8-4e17-9e89-542469ba5d3c-0000/executors/mantl_kibana.5b53e7f8-f86c-11e5-aee4-32bca1230bf7/runs/e64d9a4f-8b0d-493d-aea1-9b7ff4157200" --stop_timeout="0ns"
--container="mesos-1f357e60-4f29-45dc-8dcb-b8c9b5b09b6a-S1.e64d9a4f-8b0d-493d-aea1-9b7ff4157200" --docker="docker" --docker_socket="/var/run/docker.sock" --help="false" --initialize_driver_logging="true" --logbufsecs="0" --logging_level="INFO" --mapped_directory="/mnt/mesos/sandbox" --quiet="false" --sandbox_directory="/var/lib/mesos/slaves/1f357e60-4f29-45dc-8dcb-b8c9b5b09b6a-S1/frameworks/687233fa-73e8-4e17-9e89-542469ba5d3c-0000/executors/mantl_kibana.5b53e7f8-f86c-11e5-aee4-32bca1230bf7/runs/e64d9a4f-8b0d-493d-aea1-9b7ff4157200" --stop_timeout="0ns"
Registered docker executor on lb0-worker-01.node.consul
Starting task mantl_kibana.5b53e7f8-f86c-11e5-aee4-32bca1230bf7

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::        (v1.3.3.RELEASE)

2016-04-02 00:46:39.556  INFO 1 --- [           main] ationConfigEmbeddedWebApplicationContext : Refreshing org.springframework.boot.context.embedded.AnnotationConfigEmbeddedWebApplicationContext@f1a37ba: startup date [Sat Apr 02 00:46:39 UTC 2016]; root of context hierarchy
2016-04-02 00:46:41.308  INFO 1 --- [           main] o.s.b.f.s.DefaultListableBeanFactory     : Overriding bean definition for bean 'beanNameViewResolver' with a different definition: replacing [Root bean: class [null]; scope=; abstract=false; lazyInit=false; autowireMode=3; dependencyCheck=0; autowireCandidate=true; primary=false; factoryBeanName=org.springframework.boot.autoconfigure.web.ErrorMvcAutoConfiguration$WhitelabelErrorViewConfiguration; factoryMethodName=beanNameViewResolver; initMethodName=null; destroyMethodName=(inferred); defined in class path resource [org/springframework/boot/autoconfigure/web/ErrorMvcAutoConfiguration$WhitelabelErrorViewConfiguration.class]] with [Root bean: class [null]; scope=; abstract=false; lazyInit=false; autowireMode=3; dependencyCheck=0; autowireCandidate=true; primary=false; factoryBeanName=org.springframework.boot.autoconfigure.web.WebMvcAutoConfiguration$WebMvcAutoConfigurationAdapter; factoryMethodName=beanNameViewResolver; initMethodName=null; destroyMethodName=(inferred); defined in class path resource [org/springframework/boot/autoconfigure/web/WebMvcAutoConfiguration$WebMvcAutoConfigurationAdapter.class]]
2016-04-02 00:46:42.606  INFO 1 --- [           main] s.b.c.e.t.TomcatEmbeddedServletContainer : Tomcat initialized with port(s): 31100 (http)
2016-04-02 00:46:42.627  INFO 1 --- [           main] o.apache.catalina.core.StandardService   : Starting service Tomcat
2016-04-02 00:46:42.636  INFO 1 --- [           main] org.apache.catalina.core.StandardEngine  : Starting Servlet Engine: Apache Tomcat/8.0.32
2016-04-02 00:46:42.787  INFO 1 --- [ost-startStop-1] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring embedded WebApplicationContext
2016-04-02 00:46:42.787  INFO 1 --- [ost-startStop-1] o.s.web.context.ContextLoader            : Root WebApplicationContext: initialization completed in 3235 ms
2016-04-02 00:46:43.541  INFO 1 --- [ost-startStop-1] o.s.b.c.e.ServletRegistrationBean        : Mapping servlet: 'dispatcherServlet' to [/]
2016-04-02 00:46:43.549  INFO 1 --- [ost-startStop-1] o.s.b.c.embedded.FilterRegistrationBean  : Mapping filter: 'metricFilter' to: [/*]
2016-04-02 00:46:43.550  INFO 1 --- [ost-startStop-1] o.s.b.c.embedded.FilterRegistrationBean  : Mapping filter: 'characterEncodingFilter' to: [/*]
2016-04-02 00:46:43.550  INFO 1 --- [ost-startStop-1] o.s.b.c.embedded.FilterRegistrationBean  : Mapping filter: 'hiddenHttpMethodFilter' to: [/*]
2016-04-02 00:46:43.550  INFO 1 --- [ost-startStop-1] o.s.b.c.embedded.FilterRegistrationBean  : Mapping filter: 'httpPutFormContentFilter' to: [/*]
2016-04-02 00:46:43.550  INFO 1 --- [ost-startStop-1] o.s.b.c.embedded.FilterRegistrationBean  : Mapping filter: 'requestContextFilter' to: [/*]
2016-04-02 00:46:43.551  INFO 1 --- [ost-startStop-1] o.s.b.c.embedded.FilterRegistrationBean  : Mapping filter: 'webRequestLoggingFilter' to: [/*]
2016-04-02 00:46:43.551  INFO 1 --- [ost-startStop-1] o.s.b.c.embedded.FilterRegistrationBean  : Mapping filter: 'applicationContextIdFilter' to: [/*]
2016-04-02 00:46:44.703  INFO 1 --- [ost-startStop-1] o.a.c.util.SessionIdGeneratorBase        : Creation of SecureRandom instance for session ID generation using [SHA1PRNG] took [1,118] milliseconds.
2016-04-02 00:46:45.421  INFO 1 --- [           main] s.w.s.m.m.a.RequestMappingHandlerAdapter : Looking for @ControllerAdvice: org.springframework.boot.context.embedded.AnnotationConfigEmbeddedWebApplicationContext@f1a37ba: startup date [Sat Apr 02 00:46:39 UTC 2016]; root of context hierarchy
2016-04-02 00:46:45.532  INFO 1 --- [           main] s.w.s.m.m.a.RequestMappingHandlerMapping : Mapped "{[/mesos/resources/count],methods=[POST],consumes=[text/plain]}" onto public org.springframework.http.ResponseEntity<java.lang.Void> com.containersolutions.mesos.controllers.CountController.countEndpointPost(java.lang.String)
2016-04-02 00:46:45.533  INFO 1 --- [           main] s.w.s.m.m.a.RequestMappingHandlerMapping : Mapped "{[/mesos/resources/count],methods=[GET],produces=[text/plain]}" onto public org.springframework.http.ResponseEntity<java.lang.String> com.containersolutions.mesos.controllers.CountController.countEndpointGet()
2016-04-02 00:46:45.548  INFO 1 --- [           main] s.w.s.m.m.a.RequestMappingHandlerMapping : Mapped "{[/error],produces=[text/html]}" onto public org.springframework.web.servlet.ModelAndView org.springframework.boot.autoconfigure.web.BasicErrorController.errorHtml(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse)
2016-04-02 00:46:45.548  INFO 1 --- [           main] s.w.s.m.m.a.RequestMappingHandlerMapping : Mapped "{[/error]}" onto public org.springframework.http.ResponseEntity<java.util.Map<java.lang.String, java.lang.Object>> org.springframework.boot.autoconfigure.web.BasicErrorController.error(javax.servlet.http.HttpServletRequest)
2016-04-02 00:46:45.630  INFO 1 --- [           main] o.s.w.s.handler.SimpleUrlHandlerMapping  : Mapped URL path [/webjars/**] onto handler of type [class org.springframework.web.servlet.resource.ResourceHttpRequestHandler]
2016-04-02 00:46:45.631  INFO 1 --- [           main] o.s.w.s.handler.SimpleUrlHandlerMapping  : Mapped URL path [/**] onto handler of type [class org.springframework.web.servlet.resource.ResourceHttpRequestHandler]
2016-04-02 00:46:45.690  INFO 1 --- [           main] o.s.w.s.handler.SimpleUrlHandlerMapping  : Mapped URL path [/**/favicon.ico] onto handler of type [class org.springframework.web.servlet.resource.ResourceHttpRequestHandler]
2016-04-02 00:46:46.340  INFO 1 --- [           main] o.s.b.a.e.mvc.EndpointHandlerMapping     : Mapped "{[/autoconfig || /autoconfig.json],methods=[GET],produces=[application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.mvc.EndpointMvcAdapter.invoke()
2016-04-02 00:46:46.341  INFO 1 --- [           main] o.s.b.a.e.mvc.EndpointHandlerMapping     : Mapped "{[/health || /health.json],produces=[application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.mvc.HealthMvcEndpoint.invoke(java.security.Principal)
2016-04-02 00:46:46.344  INFO 1 --- [           main] o.s.b.a.e.mvc.EndpointHandlerMapping     : Mapped "{[/info || /info.json],methods=[GET],produces=[application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.mvc.EndpointMvcAdapter.invoke()
2016-04-02 00:46:46.358  INFO 1 --- [           main] o.s.b.a.e.mvc.EndpointHandlerMapping     : Mapped "{[/trace || /trace.json],methods=[GET],produces=[application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.mvc.EndpointMvcAdapter.invoke()
2016-04-02 00:46:46.364  INFO 1 --- [           main] o.s.b.a.e.mvc.EndpointHandlerMapping     : Mapped "{[/metrics/{name:.*}],methods=[GET],produces=[application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.mvc.MetricsMvcEndpoint.value(java.lang.String)
2016-04-02 00:46:46.364  INFO 1 --- [           main] o.s.b.a.e.mvc.EndpointHandlerMapping     : Mapped "{[/metrics || /metrics.json],methods=[GET],produces=[application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.mvc.EndpointMvcAdapter.invoke()
2016-04-02 00:46:46.365  INFO 1 --- [           main] o.s.b.a.e.mvc.EndpointHandlerMapping     : Mapped "{[/beans || /beans.json],methods=[GET],produces=[application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.mvc.EndpointMvcAdapter.invoke()
2016-04-02 00:46:46.366  INFO 1 --- [           main] o.s.b.a.e.mvc.EndpointHandlerMapping     : Mapped "{[/env/{name:.*}],methods=[GET],produces=[application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.mvc.EnvironmentMvcEndpoint.value(java.lang.String)
2016-04-02 00:46:46.366  INFO 1 --- [           main] o.s.b.a.e.mvc.EndpointHandlerMapping     : Mapped "{[/env || /env.json],methods=[GET],produces=[application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.mvc.EndpointMvcAdapter.invoke()
2016-04-02 00:46:46.373  INFO 1 --- [           main] o.s.b.a.e.mvc.EndpointHandlerMapping     : Mapped "{[/configprops || /configprops.json],methods=[GET],produces=[application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.mvc.EndpointMvcAdapter.invoke()
2016-04-02 00:46:46.374  INFO 1 --- [           main] o.s.b.a.e.mvc.EndpointHandlerMapping     : Mapped "{[/mappings || /mappings.json],methods=[GET],produces=[application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.mvc.EndpointMvcAdapter.invoke()
2016-04-02 00:46:46.375  INFO 1 --- [           main] o.s.b.a.e.mvc.EndpointHandlerMapping     : Mapped "{[/dump || /dump.json],methods=[GET],produces=[application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.mvc.EndpointMvcAdapter.invoke()
2016-04-02 00:46:46.568  INFO 1 --- [           main] o.s.j.e.a.AnnotationMBeanExporter        : Registering beans for JMX exposure on startup
2016-04-02 00:46:46.598  INFO 1 --- [           main] o.s.j.e.a.AnnotationMBeanExporter        : Bean with name 'instanceCount' has been autodetected for JMX exposure
2016-04-02 00:46:46.610  INFO 1 --- [           main] o.s.j.e.a.AnnotationMBeanExporter        : Located managed bean 'instanceCount': registering with JMX server as MBean [com.containersolutions.mesos.scheduler:name=instanceCount,type=InstanceCount]
2016-04-02 00:46:46.654  INFO 1 --- [           main] o.s.c.support.DefaultLifecycleProcessor  : Starting beans in phase 0
2016-04-02 00:46:46.905  INFO 1 --- [           main] s.b.c.e.t.TomcatEmbeddedServletContainer : Tomcat started on port(s): 31100 (http)
2016-04-02 00:46:50.386  INFO 1 --- [io-31100-exec-1] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring FrameworkServlet 'dispatcherServlet'
2016-04-02 00:46:50.386  INFO 1 --- [io-31100-exec-1] o.s.web.servlet.DispatcherServlet        : FrameworkServlet 'dispatcherServlet': initialization started
2016-04-02 00:46:50.418  INFO 1 --- [io-31100-exec-1] o.s.web.servlet.DispatcherServlet        : FrameworkServlet 'dispatcherServlet': initialization completed in 32 ms
Killing docker task
Shutting down

ryane · 2016-04-02T01:34:06Z

It looks like you only have two worker nodes. You need at least 4 with the default elk config...

langston-barrett · 2016-04-03T23:38:45Z

Works with uninstall/reinstall.

elk addon refactor

3cf3864

- kibana runs via a mesos framework (by default) - elasticsearch framework 1.0.1 (in docker mode) - elasticsearch client node (load balancer) for improved resilience Fixes #1208 Fixes #1207 Fixes #1205 Fixes #1201 Fixes #1198 Fixes #1197 Fixes #830

ryane added the addons/elk label Mar 31, 2016

ryane added this to the 1.1 milestone Mar 31, 2016

ryane added has pull request and removed has pull request labels Mar 31, 2016

elk: fix incorrect line breaks in readme

310e6f8

langston-barrett reviewed Apr 2, 2016
View reviewed changes

elk: add check-requirements to uninstall playbook

9b264b6

langston-barrett merged commit 15b67bf into master Apr 3, 2016

langston-barrett deleted the feature/elk-refactor branch April 3, 2016 23:39

ryane mentioned this pull request Apr 6, 2016

Update kibana-create-dashboard.sh.tmpl #1280

Closed

3 tasks

ryane mentioned this pull request Apr 6, 2016

Accessing ELK stack from external client #1318

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

elk addon refactor #1312

elk addon refactor #1312

ryane commented Mar 31, 2016

langston-barrett commented Apr 1, 2016

langston-barrett commented Apr 1, 2016

langston-barrett commented Apr 1, 2016

ryane commented Apr 1, 2016

langston-barrett commented Apr 1, 2016

langston-barrett commented Apr 1, 2016

langston-barrett commented Apr 1, 2016

ryane commented Apr 1, 2016

langston-barrett commented Apr 1, 2016

ryane commented Apr 1, 2016

langston-barrett commented Apr 1, 2016

ryane commented Apr 1, 2016

langston-barrett commented Apr 1, 2016

ryane commented Apr 1, 2016

langston-barrett commented Apr 2, 2016

langston-barrett Apr 2, 2016

ryane Apr 2, 2016

langston-barrett commented Apr 2, 2016

ryane commented Apr 2, 2016

langston-barrett commented Apr 3, 2016

elk addon refactor #1312

elk addon refactor #1312

Conversation

ryane commented Mar 31, 2016

langston-barrett commented Apr 1, 2016

langston-barrett commented Apr 1, 2016

langston-barrett commented Apr 1, 2016

ryane commented Apr 1, 2016

langston-barrett commented Apr 1, 2016

langston-barrett commented Apr 1, 2016

langston-barrett commented Apr 1, 2016

ryane commented Apr 1, 2016

langston-barrett commented Apr 1, 2016

ryane commented Apr 1, 2016

langston-barrett commented Apr 1, 2016

ryane commented Apr 1, 2016

langston-barrett commented Apr 1, 2016

ryane commented Apr 1, 2016

langston-barrett commented Apr 2, 2016

langston-barrett Apr 2, 2016

Choose a reason for hiding this comment

ryane Apr 2, 2016

Choose a reason for hiding this comment

langston-barrett commented Apr 2, 2016

ryane commented Apr 2, 2016

langston-barrett commented Apr 3, 2016