Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

YETUS-816. Improve hadoop personality to support ozone/hdds projects #46

Merged
merged 3 commits into from
Mar 7, 2019

Conversation

elek
Copy link
Member

@elek elek commented Mar 6, 2019

Ozone/Hdds projects are not handled very well in the hadoop personality as in case of an ozone change all the hdds/ozone projects should be built and tested

@apache-yetus
Copy link

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://builds.apache.org/job/yetus-github-multibranch/job/PR-46/1/console in case of problems.

@apache-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
0 reexec 120 Docker mode activated.
_ Prechecks _
+1 dupname 0 No case conflicting files found.
+1 @author 0 The patch does not contain any @author tags.
_ master Compile Tests _
_ Patch Compile Tests _
-1 shellcheck 1 The patch generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0)
+1 shelldocs 0 There were no new shelldocs issues.
+1 whitespace 0 The patch has no whitespace issues.
_ Other Tests _
+1 asflicense 9 The patch does not generate ASF License warnings.
146
Subsystem Report/Notes
Docker Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/yetus-github-multibranch/job/PR-46/1/artifact/out/Dockerfile
GITHUB PR #46
Optional Tests dupname asflicense shellcheck shelldocs
uname Linux 9b6be6411cdc 4.4.0-138-generic #164~14.04.1-Ubuntu SMP Fri Oct 5 08:56:16 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality precommit/src/main/shell/personality/yetus.sh
git revision master / a8db93b
maven version: Apache Maven 3.3.9
shellcheck v0.4.6
shellcheck https://builds.apache.org/job/yetus-github-multibranch/job/PR-46/1/artifact/out/diff-patch-shellcheck.txt
Max. process+thread count 41 (vs. ulimit of 2000)
modules C: precommit U: precommit
Console output https://builds.apache.org/job/yetus-github-multibranch/job/PR-46/1/console
Powered by Apache Yetus 0.10.0-SNAPSHOT http://yetus.apache.org

This message was automatically generated.

personality_enqueue_module ${module} ${extra}
OZONE_CHANGED=false
CORE_HADOOP_CHANGED=false
for module in $CHANGED_MODULES
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CHANGED_MODULES is an array, not a string. You want "${CHANGED_MODULES[@]}" here. (Also, looks like a bug in shellcheck: it should have flagged that.)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot, fixed.

(BTW, It was not flagged, I don't know why:

precommit/src/main/shell/personality/hadoop.sh:426:46: note: Double quote to prevent globbing and word splitting. [SC2086]
precommit/src/main/shell/personality/hadoop.sh:427:47: note: Double quote to prevent globbing and word splitting. [SC2086]

I tested with a simple shell script and similar structure was flagged there. I think the type of the CHANGED_MODULES is unknown here...)

if [ "$CORE_HADOOP_CHANGED" = false ] && [ "$OZONE_CHANGED" = true ]; then
if [ "$testtype" != "mvnsite" ] && [ "$testtype" != "shadedclient" ]; then
personality_enqueue_module hadoop-hdds ${extra}
personality_enqueue_module hadoop-ozone ${extra}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same.


if [ "$CORE_HADOOP_CHANGED" = false ] && [ "$OZONE_CHANGED" = true ]; then
if [ "$testtype" != "mvnsite" ] && [ "$testtype" != "shadedclient" ]; then
personality_enqueue_module hadoop-hdds ${extra}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shellcheck disable the message here. extra should have been an array but it wasn't when we first wrote yetus.

@elek elek changed the title Improve hadoop personality to support ozone/hdds projects YETUS-816. Improve hadoop personality to support ozone/hdds projects Mar 7, 2019
@apache-yetus
Copy link

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://builds.apache.org/job/yetus-github-multibranch/job/PR-46/2/console in case of problems.

@apache-yetus
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
0 reexec 122 Docker mode activated.
_ Prechecks _
+1 dupname 0 No case conflicting files found.
+1 @author 0 The patch does not contain any @author tags.
_ master Compile Tests _
_ Patch Compile Tests _
+1 shellcheck 1 There were no new shellcheck issues.
+1 shelldocs 1 There were no new shelldocs issues.
+1 whitespace 0 The patch has no whitespace issues.
_ Other Tests _
+1 asflicense 9 The patch does not generate ASF License warnings.
151
Subsystem Report/Notes
Docker Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/yetus-github-multibranch/job/PR-46/2/artifact/out/Dockerfile
GITHUB PR #46
Optional Tests dupname asflicense shellcheck shelldocs
uname Linux c61495147952 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality precommit/src/main/shell/personality/yetus.sh
git revision master / a8db93b
maven version: Apache Maven 3.3.9
shellcheck v0.4.6
Max. process+thread count 51 (vs. ulimit of 2000)
modules C: precommit U: precommit
Console output https://builds.apache.org/job/yetus-github-multibranch/job/PR-46/2/console
Powered by Apache Yetus 0.10.0-SNAPSHOT http://yetus.apache.org

This message was automatically generated.

@aw-was-here aw-was-here merged commit c401f8a into apache:master Mar 7, 2019
@elek
Copy link
Member Author

elek commented Mar 13, 2019

Thank you @aw-was-here the review and merge. I created a PR for hadoop to use the latest yetus which includes this change. apache/hadoop#599

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants