[SPARK-6196][build]Add MAPR 4.0.2 support to the build #7047
Conversation
|
Can one of the admins verify this patch? |
|
So, I have this standing mild objection to the vendor-specific builds in general mostly on the grounds they don't quite work (e.g. CDH4) and are de facto not supported by the upstream Spark project, and certainly not supported by vendors. I don't think this accomplishes therefore the apparent goal of making sure Spark can get to users' hands no matter what vendors do. But maybe not the place to rehash that now. This is better in that it pulls the logic out of the pom. Are two maintenance releases different enough to need different builds entirely @trystanleftwich ? As it's at least a little improvement over the current state, seems OK to me. |
|
@trystanleftwich we are actually recommending that MapR users use the "hadoop provided" builds that became available in Spark 1.4. You just add the MapR hadoop bindings to the class at runtime. Is there any reason you can't do that? I think MapR's own Spark distribution is using those as well. |
|
@pwendell are you suggesting we remove the mapr builds here (and add in I'm OK with this change as an incremental improvement on the current state, but still wonder whether two different maintenance releases need mention? doc 4.0.2? |
|
@trystanleftwich are you able to follow up on @pwendell's question? Also note this is removing the mapr 3 build and I have no idea what effect that has. I remain OK with this change if it doesn't have two mapr 4.0.x profiles -- seems superfluous -- or else needs an explanation why both should be special-cased by Spark. |
|
Do you mind closing this PR? I will propose an alternative |
Follow up to #7047 pwendell mentioned that MapR should use `hadoop-provided` now, and indeed the new build script does not produce `mapr3`/`mapr4` artifacts anymore. Hence the action seems to be to remove the profiles, which are now not used. CC trystanleftwich Author: Sean Owen <[email protected]> Closes #8338 from srowen/SPARK-6196. (cherry picked from commit 57b960b) Signed-off-by: Sean Owen <[email protected]>
Follow up to #7047 pwendell mentioned that MapR should use `hadoop-provided` now, and indeed the new build script does not produce `mapr3`/`mapr4` artifacts anymore. Hence the action seems to be to remove the profiles, which are now not used. CC trystanleftwich Author: Sean Owen <[email protected]> Closes #8338 from srowen/SPARK-6196.
|
I think it's safe to close this PR at this point; we can always re-open later if we choose to revive this discussion. |
Follow up to apache/spark#7047 pwendell mentioned that MapR should use `hadoop-provided` now, and indeed the new build script does not produce `mapr3`/`mapr4` artifacts anymore. Hence the action seems to be to remove the profiles, which are now not used. CC trystanleftwich Author: Sean Owen <[email protected]> Closes #8338 from srowen/SPARK-6196.
No description provided.