Log JobGroup #49
Log JobGroup #49
Conversation
|
build stuck? |
|
Looks like it, but it wouldn't surprise me if we can't get Jenkins to test Spark successfully right now. The 1.2 has some testing issues that are much better on 1.3, but even testing locally it can take me multiple attempts to get through all of the tests piecemeal. |
|
build hasn't failed though. where is it stuck? |
|
I'm not sure where Jenkins is getting hung up. The current attempt (#90) looks to be stuck at the same point where I killed off the prior attempt (#89), but it also appears that not all of the test console output ends up in the Jenkins console output, so I can't really see what is currently running. We may need to change our Jenkins to use something more like what the AMPlab jenkins uses (https://github.com/apache/spark/blob/master/dev/run-tests-jenkins) instead of running tests via maven. The Apache run-tests scripts are by no means trouble-free, but I think they are more stable or at least better exercised than running the tests through maven. |
|
have you tried running the tests locally like jenkins does? the jenkins runs will also be present locally on the slave where the tests are running if you want to confirm if there is a discrepancy between console output and jenkins log output. should be possible to resolve where jenkins is getting stuck. |
|
I'm going to merge this since it is just a (very useful) logging message change, the rest of spark-1.2.2-csd-1 has been working well and is passing local tests as well as do Apache Spark 1.2.2 and 1.2.1-csd-4. I'll deal with Jenkins later. |
…it jars (alteryx#30) * Revamp ports and service setup for the driver. - Expose the driver-submission service on NodePort and contact that as opposed to going through the API server proxy - Restrict the ports that are exposed on the service to only the driver submission service when uploading content and then only the Spark UI after the job has started * Move service creation down and more thorough error handling * Fix missed merge conflict * Add braces * Fix bad merge * Address comments and refactor run() more. Method nesting was getting confusing so pulled out the inner class and removed the extra method indirection from createDriverPod() * Remove unused method * Support SSL configuration for the driver application submission (alteryx#49) * Support SSL when setting up the driver. The user can provide a keyStore to load onto the driver pod and the driver pod will use that keyStore to set up SSL on its server. * Clean up SSL secrets after finishing submission. We don't need to persist these after the pod has them mounted and is running already. * Fix compilation error * Revert image change * Address comments * Programmatically generate certificates for integration tests. * Address comments * Resolve merge conflicts * Fix bad merge * Remove unnecessary braces * Fix compiler error
REL-627