Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] CentOS7 docker runner needs to setup locale or certain builds will fail #635

Closed
Tracked by #564
peterzhuamazon opened this issue Sep 28, 2021 · 8 comments
Closed
Tracked by #564
Assignees
Labels
bug Something isn't working

Comments

@peterzhuamazon
Copy link
Member

peterzhuamazon commented Sep 28, 2021

Describe the bug

[Bug] CentOS7 docker runner needs to setup locale or certain builds will fail

Example in AD:

2021-09-28 16:37:03 INFO Building anomaly-detection
2021-09-28 16:37:03 INFO Executing "git init" in /tmp/tmpow4z38bd/anomaly-detection
2021-09-28 16:37:03 INFO Executing "git remote add origin https://github.com/opensearch-project/anomaly-detection.git" in /tmp/tmpow4z38bd/anomaly-detection
2021-09-28 16:37:03 INFO Executing "git fetch --depth 1 origin 1.1" in /tmp/tmpow4z38bd/anomaly-detection
2021-09-28 16:37:04 INFO Executing "git checkout FETCH_HEAD" in /tmp/tmpow4z38bd/anomaly-detection
2021-09-28 16:37:04 INFO Executing "git rev-parse HEAD" in /tmp/tmpow4z38bd/anomaly-detection
2021-09-28 16:37:04 INFO Checked out https://github.com/opensearch-project/[email protected] into /tmp/tmpow4z38bd/anomaly-detection at de58aec48e2fa00e9efef7f18dc80efb9e7c0b4f
2021-09-28 16:37:04 INFO Executing "/var/jenkins/workspace/bundle-build/bundle-workflow/scripts/default/opensearch/build.sh -v 1.1.0 -a x64 -s false -o artifacts" in /tmp/tmpow4z38bd/anomaly-detection

  • getopts :h✌️s⭕a: arg
  • case $arg in
  • VERSION=1.1.0
  • getopts :h✌️s⭕a: arg
  • case $arg in
  • ARCHITECTURE=x64
  • getopts :h✌️s⭕a: arg
  • case $arg in
  • SNAPSHOT=false
  • getopts :h✌️s⭕a: arg
  • case $arg in
  • OUTPUT=artifacts
  • getopts :h✌️s⭕a: arg
  • '[' -z 1.1.0 ']'
  • [[ false == \t\r\u\e ]]
  • '[' -z artifacts ']'
  • mkdir -p artifacts
  • ./gradlew assemble --no-daemon --refresh-dependencies -DskipTests=true -Dopensearch.version=1.1.0 -Dbuild.snapshot=false
    WARNING: An illegal reflective access operation has occurred
    WARNING: Illegal reflective access by org.codehaus.groovy.reflection.CachedClass (file:/usr/share/opensearch/.gradle/wrapper/dists/gradle-6.6.1-all/ejrtlte9hlw8v6ii20a9584rs/gradle-6.6.1/lib/groovy-all-1.3-2.5.12.jar) to method java.lang.Object.finalize()
    WARNING: Please consider reporting this to the maintainers of org.codehaus.groovy.reflection.CachedClass
    WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
    WARNING: All illegal access operations will be denied in a future release
    =======================================
    OpenSearch Build Hamster says Hello!
    Gradle Version : 6.6.1
    OS Info : Linux 4.14.243-185.433.amzn2.x86_64 (amd64)
    JDK Version : 14 (AdoptOpenJDK)
    JAVA_HOME : /usr/lib/jvm/adoptopenjdk-14-hotspot
    Random Testing Seed : 76A3B2B45A00CB15
    In FIPS 140 mode : false
    =======================================

Task :compileJava
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.

Task :processResources
Task :classes
Task :jar
Task :copyPluginPropertiesTemplate
Task :pluginProperties
Task :bundlePlugin
Task :generatePomFileForNebulaPublication
Task :generatePom

Task :javadoc FAILED
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/CacheBuffer.java:54: error: unmappable character (0xE2) for encoding US-ASCII

  • We use a layered cache to manage active entities??? states. We have a two-level
    ^
    /tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/CacheBuffer.java:54: error: unmappable character (0x80) for encoding US-ASCII
  • We use a layered cache to manage active entities??? states. We have a two-level
    ^
    /tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/CacheBuffer.java:54: error: unmappable character (0x99) for encoding US-ASCII
  • We use a layered cache to manage active entities??? states. We have a two-level
    ^
    /tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/CacheBuffer.java:56: error: unmappable character (0xE2) for encoding US-ASCII
  • dedicated cache that stores ten (dynamically adjustable) entities??? states per
    ^
    /tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/CacheBuffer.java:56: error: unmappable character (0x80) for encoding US-ASCII
  • dedicated cache that stores ten (dynamically adjustable) entities??? states per
    ^
    /tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/CacheBuffer.java:56: error: unmappable character (0x99) for encoding US-ASCII
  • dedicated cache that stores ten (dynamically adjustable) entities??? states per
    ^
    /tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/CacheBuffer.java:57: error: unmappable character (0xE2) for encoding US-ASCII
  • node. A detector???s hottest entities load their states in the dedicated cache.
    ^
    /tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/CacheBuffer.java:57: error: unmappable character (0x80) for encoding US-ASCII
  • node. A detector???s hottest entities load their states in the dedicated cache.
    ^
    /tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/CacheBuffer.java:57: error: unmappable character (0x99) for encoding US-ASCII
  • node. A detector???s hottest entities load their states in the dedicated cache.
    ^
    /tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/CacheBuffer.java:62: error: unmappable character (0xE2) for encoding US-ASCII
  • detectors. The shared cache???s size shrinks as the dedicated cache is filled
    ^
    /tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/CacheBuffer.java:62: error: unmappable character (0x80) for encoding US-ASCII
  • detectors. The shared cache???s size shrinks as the dedicated cache is filled
    ^
    /tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/CacheBuffer.java:62: error: unmappable character (0x99) for encoding US-ASCII
  • detectors. The shared cache???s size shrinks as the dedicated cache is filled
    ^
    /tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/PriorityCache.java:191: error: unmappable character (0xE2) for encoding US-ASCII
    // We don???t want to admit the latest entity for correctness by throwing out a
    ^
    /tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/PriorityCache.java:191: error: unmappable character (0x80) for encoding US-ASCII
    // We don???t want to admit the latest entity for correctness by throwing out a
    ^
    /tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/PriorityCache.java:191: error: unmappable character (0x99) for encoding US-ASCII
    // We don???t want to admit the latest entity for correctness by throwing out a
    ^
    /tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/PriorityTracker.java:54: error: unmappable character (0xC2) for encoding US-ASCII
  • This class contains the heavy-hitter tracking logic. ??When an entity is hit,
    ^
    /tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/PriorityTracker.java:54: error: unmappable character (0xA0) for encoding US-ASCII
  • This class contains the heavy-hitter tracking logic. ??When an entity is hit,
    ^
    /tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/PriorityTracker.java:147: error: unmappable character (0xC2) for encoding US-ASCII
    // when detector is created. ??Can be reset. ??Unit: seconds
    ^
    /tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/PriorityTracker.java:147: error: unmappable character (0xA0) for encoding US-ASCII
    // when detector is created. ??Can be reset. ??Unit: seconds
    ^
    /tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/PriorityTracker.java:147: error: unmappable character (0xC2) for encoding US-ASCII
    // when detector is created. ??Can be reset. ??Unit: seconds
    ^
    /tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/PriorityTracker.java:147: error: unmappable character (0xA0) for encoding US-ASCII
    // when detector is created. ??Can be reset. ??Unit: seconds
    ^
    /tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/PriorityTracker.java:153: error: unmappable character (0xC2) for encoding US-ASCII
    // length of seconds in one interval. ??Used to compute elapsed periods
    ^
    /tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/PriorityTracker.java:153: error: unmappable character (0xA0) for encoding US-ASCII
    // length of seconds in one interval. ??Used to compute elapsed periods
    ^
    /tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/PriorityTracker.java:296: error: unmappable character (0xE2) for encoding US-ASCII
    • Each detector maintains an ordered map, filled by entities's accumulated sum of g(i???L),
      ^
      /tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/PriorityTracker.java:296: error: unmappable character (0x88) for encoding US-ASCII
    • Each detector maintains an ordered map, filled by entities's accumulated sum of g(i???L),
      ^
      /tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/PriorityTracker.java:296: error: unmappable character (0x92) for encoding US-ASCII
    • Each detector maintains an ordered map, filled by entities's accumulated sum of g(i???L),
      ^
      /tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/PriorityTracker.java:303: error: unmappable character (0xE2) for encoding US-ASCII
    • Since g(i???L) is changing and they are the same for all entities of the same detector,
      ^
      /tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/PriorityTracker.java:303: error: unmappable character (0x88) for encoding US-ASCII
    • Since g(i???L) is changing and they are the same for all entities of the same detector,
      ^
      /tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/PriorityTracker.java:303: error: unmappable character (0x92) for encoding US-ASCII
    • Since g(i???L) is changing and they are the same for all entities of the same detector,
      ^
      /tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/PriorityTracker.java:304: error: unmappable character (0xE2) for encoding US-ASCII
    • we can compare entities' priorities by considering the accumulated sum of g(i???L).
      ^
      /tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/PriorityTracker.java:304: error: unmappable character (0x88) for encoding US-ASCII
    • we can compare entities' priorities by considering the accumulated sum of g(i???L).
      ^
      /tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/PriorityTracker.java:304: error: unmappable character (0x92) for encoding US-ASCII
    • we can compare entities' priorities by considering the accumulated sum of g(i???L).
      ^
      /tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/PriorityTracker.java:329: error: unmappable character (0xE2) for encoding US-ASCII
    • compute w(i,p) for each minimum entity by scaling the sum by g(p???L). Notice g(p???L)
      ^
      /tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/PriorityTracker.java:329: error: unmappable character (0x88) for encoding US-ASCII
    • compute w(i,p) for each minimum entity by scaling the sum by g(p???L). Notice g(p???L)
      ^
      /tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/PriorityTracker.java:329: error: unmappable character (0x92) for encoding US-ASCII
    • compute w(i,p) for each minimum entity by scaling the sum by g(p???L). Notice g(p???L)
      ^
      /tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/PriorityTracker.java:329: error: unmappable character (0xE2) for encoding US-ASCII
    • compute w(i,p) for each minimum entity by scaling the sum by g(p???L). Notice g(p???L)
      ^
      /tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/PriorityTracker.java:329: error: unmappable character (0x88) for encoding US-ASCII
    • compute w(i,p) for each minimum entity by scaling the sum by g(p???L). Notice g(p???L)
      ^
      /tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/caching/PriorityTracker.java:329: error: unmappable character (0x92) for encoding US-ASCII
    • compute w(i,p) for each minimum entity by scaling the sum by g(p???L). Notice g(p???L)
      ^
      /tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/model/ADTaskState.java:41: error: unmappable character (0xE2) for encoding US-ASCII
  • After task created, coordinate node will gather all eligible node???s state and dispatch
                                                                     ^
    

/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/model/ADTaskState.java:41: error: unmappable character (0x80) for encoding US-ASCII

  • After task created, coordinate node will gather all eligible node???s state and dispatch
                                                                      ^
    

/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/model/ADTaskState.java:41: error: unmappable character (0x99) for encoding US-ASCII

  • After task created, coordinate node will gather all eligible node???s state and dispatch
                                                                       ^
    

/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/model/ADTaskState.java:59: error: unmappable character (0xE2) for encoding US-ASCII

  • feature and reran and don???t want current task run any more. When a historical detector
                             ^
    

/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/model/ADTaskState.java:59: error: unmappable character (0x80) for encoding US-ASCII

  • feature and reran and don???t want current task run any more. When a historical detector
                              ^
    

/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/model/ADTaskState.java:59: error: unmappable character (0x99) for encoding US-ASCII

  • feature and reran and don???t want current task run any more. When a historical detector
                               ^
    

/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/model/Entity.java:250: error: unmappable character (0xE2) for encoding US-ASCII
* OpenSearch???s document Id can be at most 512 bytes. Categorical values are
^
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/model/Entity.java:250: error: unmappable character (0x80) for encoding US-ASCII
* OpenSearch???s document Id can be at most 512 bytes. Categorical values are
^
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/model/Entity.java:250: error: unmappable character (0x99) for encoding US-ASCII
* OpenSearch???s document Id can be at most 512 bytes. Categorical values are
^
/tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/model/InitProgressProfile.java:57: error: unmappable character (0xE2) for encoding US-ASCII

  • there hasn???t been enough continuous data in the index. We need to use live data to initialize.
    ^
    /tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/model/InitProgressProfile.java:57: error: unmappable character (0x80) for encoding US-ASCII
  • there hasn???t been enough continuous data in the index. We need to use live data to initialize.
    ^
    /tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/model/InitProgressProfile.java:57: error: unmappable character (0x99) for encoding US-ASCII
  • there hasn???t been enough continuous data in the index. We need to use live data to initialize.
    ^
    /tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/ratelimit/ColdEntityWorker.java:41: error: unmappable character (0xC2) for encoding US-ASCII
  • the incoming feature is normal, update models, and save the detection results to disks.??
    ^
    /tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/ratelimit/ColdEntityWorker.java:41: error: unmappable character (0xA0) for encoding US-ASCII
  • the incoming feature is normal, update models, and save the detection results to disks.??
    ^
    /tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/ratelimit/ColdEntityWorker.java:47: error: unmappable character (0xC2) for encoding US-ASCII
  • entity requests.??
    ^
    /tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/ratelimit/ColdEntityWorker.java:47: error: unmappable character (0xA0) for encoding US-ASCII
  • entity requests.??
    ^
    /tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/ratelimit/RateLimitedRequestWorker.java:50: error: unmappable character (0xE2) for encoding US-ASCII
  • HCAD can bombard Opensearch with ???thundering herd??? traffic, in which many entities
    ^
    /tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/ratelimit/RateLimitedRequestWorker.java:50: error: unmappable character (0x80) for encoding US-ASCII
  • HCAD can bombard Opensearch with ???thundering herd??? traffic, in which many entities
    ^
    /tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/ratelimit/RateLimitedRequestWorker.java:50: error: unmappable character (0x9C) for encoding US-ASCII
  • HCAD can bombard Opensearch with ???thundering herd??? traffic, in which many entities
    ^
    /tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/ratelimit/RateLimitedRequestWorker.java:50: error: unmappable character (0xE2) for encoding US-ASCII
  • HCAD can bombard Opensearch with ???thundering herd??? traffic, in which many entities
    ^
    /tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/ratelimit/RateLimitedRequestWorker.java:50: error: unmappable character (0x80) for encoding US-ASCII
  • HCAD can bombard Opensearch with ???thundering herd??? traffic, in which many entities
    ^
    /tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/ratelimit/RateLimitedRequestWorker.java:50: error: unmappable character (0x9D) for encoding US-ASCII
  • HCAD can bombard Opensearch with ???thundering herd??? traffic, in which many entities
    ^
    /tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/ratelimit/RateLimitedRequestWorker.java:144: error: unmappable character (0xE2) for encoding US-ASCII
    * Enforcing the expiry time places an upper bound on each request???s
    ^
    /tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/ratelimit/RateLimitedRequestWorker.java:144: error: unmappable character (0x80) for encoding US-ASCII
    * Enforcing the expiry time places an upper bound on each request???s
    ^
    /tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/ratelimit/RateLimitedRequestWorker.java:144: error: unmappable character (0x99) for encoding US-ASCII
    * Enforcing the expiry time places an upper bound on each request???s
    ^
    /tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/ratelimit/RateLimitedRequestWorker.java:246: error: unmappable character (0xE2) for encoding US-ASCII
    • detector???s workload times out, and other detectors??? workloads continue
      ^
      /tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/ratelimit/RateLimitedRequestWorker.java:246: error: unmappable character (0x80) for encoding US-ASCII
    • detector???s workload times out, and other detectors??? workloads continue
      ^
      /tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/ratelimit/RateLimitedRequestWorker.java:246: error: unmappable character (0x99) for encoding US-ASCII
    • detector???s workload times out, and other detectors??? workloads continue
      ^
      /tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/ratelimit/RateLimitedRequestWorker.java:246: error: unmappable character (0xE2) for encoding US-ASCII
    • detector???s workload times out, and other detectors??? workloads continue
      ^
      /tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/ratelimit/RateLimitedRequestWorker.java:246: error: unmappable character (0x80) for encoding US-ASCII
    • detector???s workload times out, and other detectors??? workloads continue
      ^
      /tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/ratelimit/RateLimitedRequestWorker.java:246: error: unmappable character (0x99) for encoding US-ASCII
    • detector???s workload times out, and other detectors??? workloads continue
      ^
      /tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/ratelimit/RateLimitedRequestWorker.java:248: error: unmappable character (0xE2) for encoding US-ASCII
    • HCAD pulls off 10 requests from one detector??? queues, issues a mget request
      ^
      /tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/ratelimit/RateLimitedRequestWorker.java:248: error: unmappable character (0x80) for encoding US-ASCII
    • HCAD pulls off 10 requests from one detector??? queues, issues a mget request
      ^
      /tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/ratelimit/RateLimitedRequestWorker.java:248: error: unmappable character (0x99) for encoding US-ASCII
    • HCAD pulls off 10 requests from one detector??? queues, issues a mget request
      ^
      /tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/ratelimit/RateLimitedRequestWorker.java:249: error: unmappable character (0xE2) for encoding US-ASCII
    • to ES, wait for it to finish, and then does it again for other detectors???
      ^
      /tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/ratelimit/RateLimitedRequestWorker.java:249: error: unmappable character (0x80) for encoding US-ASCII
    • to ES, wait for it to finish, and then does it again for other detectors???
      ^
      /tmp/tmpow4z38bd/anomaly-detection/src/main/java/org/opensearch/ad/ratelimit/RateLimitedRequestWorker.java:249: error: unmappable character (0x99) for encoding US-ASCII
    • to ES, wait for it to finish, and then does it again for other detectors???
      ^
      75 errors

FAILURE: Build failed with an exception.

  • What went wrong:
    Execution failed for task ':javadoc'.

Javadoc generation failed. Generated Javadoc options file (useful for troubleshooting): '/tmp/tmpow4z38bd/anomaly-detection/build/tmp/javadoc/javadoc.options'

  • Try:
    Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

  • Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.6.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 37s

To reproduce

docker pull centos:7

Expected behavior

This can be fixed by setting up locale properly:

$ localedef -v -c -i en_US -f UTF-8 en_US.UTF-8

Screenshots

If applicable, add screenshots to help explain your problem.

Host / Environment

No response

Additional context

No response

Relevant log output

No response

@peterzhuamazon peterzhuamazon added bug Something isn't working untriaged Issues that have not yet been triaged labels Sep 28, 2021
@peterzhuamazon peterzhuamazon changed the title [Bug] CentOS7 runner needs to setup locale or certain builds will fail [Bug] CentOS7 docker runner needs to setup locale or certain builds will fail Sep 28, 2021
@peterzhuamazon peterzhuamazon self-assigned this Sep 28, 2021
@peterzhuamazon
Copy link
Member Author

Related to #627.

@peterzhuamazon peterzhuamazon removed the untriaged Issues that have not yet been triaged label Sep 28, 2021
@peterzhuamazon
Copy link
Member Author

Seems like $ localedef -v -c -i en_US -f UTF-8 en_US.UTF-8 does not fix it, need more debugging.

@peterzhuamazon
Copy link
Member Author

When using the same docker image on an identical AL2 host, it is working without any errors, thinking it is the same issues happens to JAVA_HOME, where Jenkins Pipeline did not retrieve the right locale from the container settings.

@peterzhuamazon
Copy link
Member Author

Hi @dblock @peternied I need some help with this one as well.
When running in Jenkins the error shows with AD.
However when running locally with docker image docker pull opensearchstaging/ci-runner:centos7-x64-arm64-jdk14-node10.24.1-20210924 it runs the build process to completion without issues.

If this is still not working tomorrow I am offer to revert the changes from centos7 docker PR #627.

Thanks.

@peterzhuamazon
Copy link
Member Author

peterzhuamazon commented Sep 28, 2021

Issue found:

src/main/java/org/opensearch/ad/caching/CacheBuffer.java: UTF-8 Unicode text
src/main/java/org/opensearch/ad/caching/CacheBuffer.java: UTF-8 Unicode text
src/main/java/org/opensearch/ad/caching/CacheBuffer.java: UTF-8 Unicode text
src/main/java/org/opensearch/ad/caching/CacheBuffer.java: UTF-8 Unicode text
src/main/java/org/opensearch/ad/caching/CacheBuffer.java: UTF-8 Unicode text
src/main/java/org/opensearch/ad/caching/CacheBuffer.java: UTF-8 Unicode text
src/main/java/org/opensearch/ad/caching/CacheBuffer.java: UTF-8 Unicode text
src/main/java/org/opensearch/ad/caching/CacheBuffer.java: UTF-8 Unicode text
src/main/java/org/opensearch/ad/caching/CacheBuffer.java: UTF-8 Unicode text
src/main/java/org/opensearch/ad/caching/CacheBuffer.java: UTF-8 Unicode text
src/main/java/org/opensearch/ad/caching/CacheProvider.java: ASCII text
src/main/java/org/opensearch/ad/caching/DoorKeeper.java: ASCII text
src/main/java/org/opensearch/ad/caching/EntityCache.java: ASCII text
src/main/java/org/opensearch/ad/caching/EntityCache.java: ASCII text
src/main/java/org/opensearch/ad/caching/EntityCache.java: ASCII text
src/main/java/org/opensearch/ad/caching/EntityCache.java: ASCII text
src/main/java/org/opensearch/ad/caching/EntityCache.java: ASCII text
src/main/java/org/opensearch/ad/caching/EntityCache.java: ASCII text
src/main/java/org/opensearch/ad/caching/EntityCache.java: ASCII text
src/main/java/org/opensearch/ad/caching/EntityCache.java: ASCII text
src/main/java/org/opensearch/ad/caching/EntityCache.java: ASCII text
src/main/java/org/opensearch/ad/caching/EntityCache.java: ASCII text
src/main/java/org/opensearch/ad/caching/EntityCache.java: ASCII text

Working with @ylwu-amzn and found out that several problematic files are using utf-8 while all other files are using ASCII. We also tested the IDE default format is ASCII on intellij, and other plugins like alerting and k-NN all use ASCII.

These are the files that cause the issues:

src/main/java/org/opensearch/ad/caching/CacheBuffer.java: UTF-8 Unicode text
src/main/java/org/opensearch/ad/caching/PriorityCache.java: UTF-8 Unicode text
src/main/java/org/opensearch/ad/caching/PriorityTracker.java: UTF-8 Unicode text
src/main/java/org/opensearch/ad/model/Entity.java: UTF-8 Unicode text
src/main/java/org/opensearch/ad/ratelimit/ColdEntityWorker.java: UTF-8 Unicode text
src/main/java/org/opensearch/ad/ratelimit/RateLimitedRequestWorker.java: UTF-8 Unicode text
src/main/java/org/opensearch/ad/model/ADTaskState.java: Java source, UTF-8 Unicode text
src/main/java/org/opensearch/ad/model/InitProgressProfile.java: Java source, UTF-8 Unicode text

Thanks.

@peterzhuamazon
Copy link
Member Author

As a fix later we can try https://gist.github.com/rponte/d660919434d094bbd35a1aabf7ef1bf0 to enable utf-8 encoding.

@peterzhuamazon
Copy link
Member Author

Resolved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant