Skip to content
This repository has been archived by the owner on Jul 27, 2023. It is now read-only.

Examples/spark fix #1380

Merged
merged 4 commits into from
Apr 27, 2016
Merged

Examples/spark fix #1380

merged 4 commits into from
Apr 27, 2016

Conversation

mcapuccini
Copy link
Contributor

I have now experimented for a while running Spark jobs in MANTL. This fixes the previous example, where for a higher N passed as argument to SparkPi, a network exception occurred. I am not sure why this was happening, but it was probably due to a problem with port mappings in the executor docker containers. In this new example docker is used only to submit the job to the Spark dispatcher, and then the executor are deployed using spark.executor.uri. Furthermore, I figured out a couple of tricks that reduce the number of prerequisites to run the example. The source code of my custom spark-mantl docker image is at: https://github.com/mcapuccini/spark-container

@langston-barrett
Copy link
Contributor

@mcapuccini Could we link to your container source in the README?

@mcapuccini
Copy link
Contributor Author

@siddharthist sure, I just linked it

@langston-barrett
Copy link
Contributor

langston-barrett commented Apr 26, 2016

I got this error:

$ ./schedule.sh
Please insert your MANTL control node IP address (or domain name)
xxxxxxxxxx
Please instert your MANTL admin password:
HTTP/1.1 204 No Content
Server: nginx/1.6.2
Date: Tue, 26 Apr 2016 18:28:02 GMT
Content-Type: application/json
Connection: keep-alive

but it looks like it worked:
screenshot

There's also a typo in schedule.sh: "instert"

@mcapuccini
Copy link
Contributor Author

mcapuccini commented Apr 27, 2016

@siddharthist 204 means OK, with empty response in the HTTP protocol. It is not an error. Unfortunately Chronos does not return any entity-body with useful messages.

I'll fix the typo.

@langston-barrett
Copy link
Contributor

@mcapuccini Thanks!

@langston-barrett langston-barrett merged commit ad81092 into mantl:master Apr 27, 2016
@ryane ryane modified the milestone: 1.1 May 10, 2016
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants