-
Notifications
You must be signed in to change notification settings - Fork 282
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Bug] CentOS7 docker runner needs to setup locale or certain builds will fail #635
Comments
Related to #627. |
Seems like |
When using the same docker image on an identical AL2 host, it is working without any errors, thinking it is the same issues happens to JAVA_HOME, where Jenkins Pipeline did not retrieve the right locale from the container settings. |
Hi @dblock @peternied I need some help with this one as well. If this is still not working tomorrow I am offer to revert the changes from centos7 docker PR #627. Thanks. |
Issue found:
Working with @ylwu-amzn and found out that several problematic files are using utf-8 while all other files are using ASCII. We also tested the IDE default format is ASCII on intellij, and other plugins like alerting and k-NN all use ASCII. These are the files that cause the issues:
Thanks. |
As a fix later we can try https://gist.github.com/rponte/d660919434d094bbd35a1aabf7ef1bf0 to enable utf-8 encoding. |
Resolved. |
Describe the bug
[Bug] CentOS7 docker runner needs to setup locale or certain builds will fail
Example in AD:
2021-09-28 16:37:03 INFO Building anomaly-detection
2021-09-28 16:37:03 INFO Executing "git init" in /tmp/tmpow4z38bd/anomaly-detection
2021-09-28 16:37:03 INFO Executing "git remote add origin https://github.com/opensearch-project/anomaly-detection.git" in /tmp/tmpow4z38bd/anomaly-detection
2021-09-28 16:37:03 INFO Executing "git fetch --depth 1 origin 1.1" in /tmp/tmpow4z38bd/anomaly-detection
2021-09-28 16:37:04 INFO Executing "git checkout FETCH_HEAD" in /tmp/tmpow4z38bd/anomaly-detection
2021-09-28 16:37:04 INFO Executing "git rev-parse HEAD" in /tmp/tmpow4z38bd/anomaly-detection
2021-09-28 16:37:04 INFO Checked out https://github.com/opensearch-project/[email protected] into /tmp/tmpow4z38bd/anomaly-detection at de58aec48e2fa00e9efef7f18dc80efb9e7c0b4f
2021-09-28 16:37:04 INFO Executing "/var/jenkins/workspace/bundle-build/bundle-workflow/scripts/default/opensearch/build.sh -v 1.1.0 -a x64 -s false -o artifacts" in /tmp/tmpow4z38bd/anomaly-detection
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.codehaus.groovy.reflection.CachedClass (file:/usr/share/opensearch/.gradle/wrapper/dists/gradle-6.6.1-all/ejrtlte9hlw8v6ii20a9584rs/gradle-6.6.1/lib/groovy-all-1.3-2.5.12.jar) to method java.lang.Object.finalize()
WARNING: Please consider reporting this to the maintainers of org.codehaus.groovy.reflection.CachedClass
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
=======================================
OpenSearch Build Hamster says Hello!
Gradle Version : 6.6.1
OS Info : Linux 4.14.243-185.433.amzn2.x86_64 (amd64)
JDK Version : 14 (AdoptOpenJDK)
JAVA_HOME : /usr/lib/jvm/adoptopenjdk-14-hotspot
Random Testing Seed : 76A3B2B45A00CB15
In FIPS 140 mode : false
=======================================
^
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/CacheBuffer.java:54: error: unmappable character (0x80) for encoding US-ASCII
^
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/CacheBuffer.java:54: error: unmappable character (0x99) for encoding US-ASCII
^
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/CacheBuffer.java:56: error: unmappable character (0xE2) for encoding US-ASCII
^
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/CacheBuffer.java:56: error: unmappable character (0x80) for encoding US-ASCII
^
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/CacheBuffer.java:56: error: unmappable character (0x99) for encoding US-ASCII
^
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/CacheBuffer.java:57: error: unmappable character (0xE2) for encoding US-ASCII
^
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/CacheBuffer.java:57: error: unmappable character (0x80) for encoding US-ASCII
^
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/CacheBuffer.java:57: error: unmappable character (0x99) for encoding US-ASCII
^
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/CacheBuffer.java:62: error: unmappable character (0xE2) for encoding US-ASCII
^
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/CacheBuffer.java:62: error: unmappable character (0x80) for encoding US-ASCII
^
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/CacheBuffer.java:62: error: unmappable character (0x99) for encoding US-ASCII
^
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/PriorityCache.java:191: error: unmappable character (0xE2) for encoding US-ASCII
// We don???t want to admit the latest entity for correctness by throwing out a
^
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/PriorityCache.java:191: error: unmappable character (0x80) for encoding US-ASCII
// We don???t want to admit the latest entity for correctness by throwing out a
^
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/PriorityCache.java:191: error: unmappable character (0x99) for encoding US-ASCII
// We don???t want to admit the latest entity for correctness by throwing out a
^
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/PriorityTracker.java:54: error: unmappable character (0xC2) for encoding US-ASCII
^
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/PriorityTracker.java:54: error: unmappable character (0xA0) for encoding US-ASCII
^
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/PriorityTracker.java:147: error: unmappable character (0xC2) for encoding US-ASCII
// when detector is created. ??Can be reset. ??Unit: seconds
^
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/PriorityTracker.java:147: error: unmappable character (0xA0) for encoding US-ASCII
// when detector is created. ??Can be reset. ??Unit: seconds
^
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/PriorityTracker.java:147: error: unmappable character (0xC2) for encoding US-ASCII
// when detector is created. ??Can be reset. ??Unit: seconds
^
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/PriorityTracker.java:147: error: unmappable character (0xA0) for encoding US-ASCII
// when detector is created. ??Can be reset. ??Unit: seconds
^
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/PriorityTracker.java:153: error: unmappable character (0xC2) for encoding US-ASCII
// length of seconds in one interval. ??Used to compute elapsed periods
^
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/PriorityTracker.java:153: error: unmappable character (0xA0) for encoding US-ASCII
// length of seconds in one interval. ??Used to compute elapsed periods
^
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/PriorityTracker.java:296: error: unmappable character (0xE2) for encoding US-ASCII
^
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/PriorityTracker.java:296: error: unmappable character (0x88) for encoding US-ASCII
^
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/PriorityTracker.java:296: error: unmappable character (0x92) for encoding US-ASCII
^
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/PriorityTracker.java:303: error: unmappable character (0xE2) for encoding US-ASCII
^
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/PriorityTracker.java:303: error: unmappable character (0x88) for encoding US-ASCII
^
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/PriorityTracker.java:303: error: unmappable character (0x92) for encoding US-ASCII
^
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/PriorityTracker.java:304: error: unmappable character (0xE2) for encoding US-ASCII
^
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/PriorityTracker.java:304: error: unmappable character (0x88) for encoding US-ASCII
^
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/PriorityTracker.java:304: error: unmappable character (0x92) for encoding US-ASCII
^
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/PriorityTracker.java:329: error: unmappable character (0xE2) for encoding US-ASCII
^
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/PriorityTracker.java:329: error: unmappable character (0x88) for encoding US-ASCII
^
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/PriorityTracker.java:329: error: unmappable character (0x92) for encoding US-ASCII
^
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/PriorityTracker.java:329: error: unmappable character (0xE2) for encoding US-ASCII
^
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/PriorityTracker.java:329: error: unmappable character (0x88) for encoding US-ASCII
^
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/PriorityTracker.java:329: error: unmappable character (0x92) for encoding US-ASCII
^
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/model/ADTaskState.java:41: error: unmappable character (0xE2) for encoding US-ASCII
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/model/ADTaskState.java:41: error: unmappable character (0x80) for encoding US-ASCII
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/model/ADTaskState.java:41: error: unmappable character (0x99) for encoding US-ASCII
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/model/ADTaskState.java:59: error: unmappable character (0xE2) for encoding US-ASCII
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/model/ADTaskState.java:59: error: unmappable character (0x80) for encoding US-ASCII
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/model/ADTaskState.java:59: error: unmappable character (0x99) for encoding US-ASCII
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/model/Entity.java:250: error: unmappable character (0xE2) for encoding US-ASCII
* OpenSearch???s document Id can be at most 512 bytes. Categorical values are
^
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/model/Entity.java:250: error: unmappable character (0x80) for encoding US-ASCII
* OpenSearch???s document Id can be at most 512 bytes. Categorical values are
^
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/model/Entity.java:250: error: unmappable character (0x99) for encoding US-ASCII
* OpenSearch???s document Id can be at most 512 bytes. Categorical values are
^
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/model/InitProgressProfile.java:57: error: unmappable character (0xE2) for encoding US-ASCII
^
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/model/InitProgressProfile.java:57: error: unmappable character (0x80) for encoding US-ASCII
^
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/model/InitProgressProfile.java:57: error: unmappable character (0x99) for encoding US-ASCII
^
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/ratelimit/ColdEntityWorker.java:41: error: unmappable character (0xC2) for encoding US-ASCII
^
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/ratelimit/ColdEntityWorker.java:41: error: unmappable character (0xA0) for encoding US-ASCII
^
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/ratelimit/ColdEntityWorker.java:47: error: unmappable character (0xC2) for encoding US-ASCII
^
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/ratelimit/ColdEntityWorker.java:47: error: unmappable character (0xA0) for encoding US-ASCII
^
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/ratelimit/RateLimitedRequestWorker.java:50: error: unmappable character (0xE2) for encoding US-ASCII
^
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/ratelimit/RateLimitedRequestWorker.java:50: error: unmappable character (0x80) for encoding US-ASCII
^
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/ratelimit/RateLimitedRequestWorker.java:50: error: unmappable character (0x9C) for encoding US-ASCII
^
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/ratelimit/RateLimitedRequestWorker.java:50: error: unmappable character (0xE2) for encoding US-ASCII
^
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/ratelimit/RateLimitedRequestWorker.java:50: error: unmappable character (0x80) for encoding US-ASCII
^
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/ratelimit/RateLimitedRequestWorker.java:50: error: unmappable character (0x9D) for encoding US-ASCII
^
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/ratelimit/RateLimitedRequestWorker.java:144: error: unmappable character (0xE2) for encoding US-ASCII
* Enforcing the expiry time places an upper bound on each request???s
^
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/ratelimit/RateLimitedRequestWorker.java:144: error: unmappable character (0x80) for encoding US-ASCII
* Enforcing the expiry time places an upper bound on each request???s
^
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/ratelimit/RateLimitedRequestWorker.java:144: error: unmappable character (0x99) for encoding US-ASCII
* Enforcing the expiry time places an upper bound on each request???s
^
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/ratelimit/RateLimitedRequestWorker.java:246: error: unmappable character (0xE2) for encoding US-ASCII
^
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/ratelimit/RateLimitedRequestWorker.java:246: error: unmappable character (0x80) for encoding US-ASCII
^
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/ratelimit/RateLimitedRequestWorker.java:246: error: unmappable character (0x99) for encoding US-ASCII
^
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/ratelimit/RateLimitedRequestWorker.java:246: error: unmappable character (0xE2) for encoding US-ASCII
^
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/ratelimit/RateLimitedRequestWorker.java:246: error: unmappable character (0x80) for encoding US-ASCII
^
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/ratelimit/RateLimitedRequestWorker.java:246: error: unmappable character (0x99) for encoding US-ASCII
^
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/ratelimit/RateLimitedRequestWorker.java:248: error: unmappable character (0xE2) for encoding US-ASCII
^
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/ratelimit/RateLimitedRequestWorker.java:248: error: unmappable character (0x80) for encoding US-ASCII
^
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/ratelimit/RateLimitedRequestWorker.java:248: error: unmappable character (0x99) for encoding US-ASCII
^
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/ratelimit/RateLimitedRequestWorker.java:249: error: unmappable character (0xE2) for encoding US-ASCII
^
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/ratelimit/RateLimitedRequestWorker.java:249: error: unmappable character (0x80) for encoding US-ASCII
^
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/ratelimit/RateLimitedRequestWorker.java:249: error: unmappable character (0x99) for encoding US-ASCII
^
75 errors
FAILURE: Build failed with an exception.
Execution failed for task ':javadoc'.
Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.6.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 37s
To reproduce
docker pull centos:7
Expected behavior
This can be fixed by setting up locale properly:
Screenshots
If applicable, add screenshots to help explain your problem.
Host / Environment
No response
Additional context
No response
Relevant log output
No response
The text was updated successfully, but these errors were encountered: