Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Several files are using utf-8 encoding, should use ACSII #244

Closed
peterzhuamazon opened this issue Sep 28, 2021 · 4 comments · Fixed by #245
Closed

[BUG] Several files are using utf-8 encoding, should use ACSII #244

peterzhuamazon opened this issue Sep 28, 2021 · 4 comments · Fixed by #245
Assignees
Labels
bug Something isn't working

Comments

@peterzhuamazon
Copy link
Member

peterzhuamazon commented Sep 28, 2021

Describe the bug
opensearch-project/opensearch-build#635 (comment)

These files cause build failure due to encoding not in ASCII:

src/main/java/org/opensearch/ad/caching/CacheBuffer.java: UTF-8 Unicode text
src/main/java/org/opensearch/ad/caching/PriorityCache.java: UTF-8 Unicode text
src/main/java/org/opensearch/ad/caching/PriorityTracker.java: UTF-8 Unicode text
src/main/java/org/opensearch/ad/model/Entity.java: UTF-8 Unicode text
src/main/java/org/opensearch/ad/ratelimit/ColdEntityWorker.java: UTF-8 Unicode text
src/main/java/org/opensearch/ad/ratelimit/RateLimitedRequestWorker.java: UTF-8 Unicode text
src/main/java/org/opensearch/ad/model/ADTaskState.java: Java source, UTF-8 Unicode text
src/main/java/org/opensearch/ad/model/InitProgressProfile.java: Java source, UTF-8 Unicode text
/tmp/tmpbcap9s69/anomaly-detection/src/main/java/org/opensearch/ad/caching/CacheBuffer.java:54: error: unmappable character (0x80) for encoding US-ASCII
 * We use a layered cache to manage active entities??? states.  We have a two-level
                                                    ^
/tmp/tmpbcap9s69/anomaly-detection/src/main/java/org/opensearch/ad/caching/CacheBuffer.java:54: error: unmappable character (0x99) for encoding US-ASCII
 * We use a layered cache to manage active entities??? states.  We have a two-level
                                                     ^
/tmp/tmpbcap9s69/anomaly-detection/src/main/java/org/opensearch/ad/caching/CacheBuffer.java:56: error: unmappable character (0xE2) for encoding US-ASCII
 * dedicated cache that stores ten (dynamically adjustable) entities??? states per
 * ```

**Other plugins installed**
Please mention if you are using this plugin along side any other plugin. Security for example.

**To Reproduce**
Run the build on Jenkins with `./gradlew assemble --no-daemon --refresh-dependencies -DskipTests=true -Dopensearch.version=1.1.0 -Dbuild.snapshot=false` on CentOS7.
Not producible on local only on Jenkins.

**Expected behavior**
Javadoc build correctly without this unmapping issues.

**Screenshots**
If applicable, add screenshots to help explain your problem.

**Desktop (please complete the following information):**
 - OS: [e.g. iOS]
 - Browser [e.g. chrome, safari]
 - Version [e.g. 22]

**Additional context**
Add any other context about the problem here.
@peterzhuamazon peterzhuamazon added bug Something isn't working Beta untriaged and removed Beta untriaged labels Sep 28, 2021
@peterzhuamazon peterzhuamazon changed the title [BUG] Several files are using utf-8 format, should use ACSII [BUG] Several files are using utf-8 encoding, should use ACSII Sep 28, 2021
@peterzhuamazon peterzhuamazon linked a pull request Sep 29, 2021 that will close this issue
1 task
@peterzhuamazon peterzhuamazon self-assigned this Sep 29, 2021
@peterzhuamazon
Copy link
Member Author

#245
#246
#247

@peterzhuamazon
Copy link
Member Author

As a fix later we can try https://gist.github.com/rponte/d660919434d094bbd35a1aabf7ef1bf0 to enable utf-8 encoding.

@peterzhuamazon
Copy link
Member Author

peterzhuamazon commented Sep 29, 2021

Adding these to fully resolve the issues:

tasks.withType(JavaCompile) {
        options.encoding = "UTF-8"
}
tasks.withType(Test) {
        systemProperty "file.encoding", "UTF-8"
}
tasks.withType(Javadoc) {
    options.encoding = 'UTF-8'
}

Logs:

2021-09-29 20:55:54 INFO     Building anomaly-detection
2021-09-29 20:55:54 INFO     Executing "git init" in /tmp/tmpcfupmjmj/anomaly-detection
2021-09-29 20:55:54 INFO     Executing "git remote add origin https://github.com/opensearch-project/anomaly-detection.git" in /tmp/tmpcfupmjmj/anomaly-detection
2021-09-29 20:55:54 INFO     Executing "git fetch --depth 1 origin opensearch-ascii-utf8" in /tmp/tmpcfupmjmj/anomaly-detection

2021-09-29 20:55:55 INFO     Executing "git checkout FETCH_HEAD" in /tmp/tmpcfupmjmj/anomaly-detection
2021-09-29 20:55:55 INFO     Executing "git rev-parse HEAD" in /tmp/tmpcfupmjmj/anomaly-detection
2021-09-29 20:55:55 INFO     Checked out https://github.com/opensearch-project/anomaly-detection.git@opensearch-ascii-utf8 into /tmp/tmpcfupmjmj/anomaly-detection at f780eed22a044c29a0906c8da44c9f25316779e6
2021-09-29 20:55:55 INFO     Executing "/var/jenkins/workspace/bundle_pipelines/bundle-build_1.0.1@4/bundle-workflow/scripts/default/opensearch/build.sh -v 1.1.0 -a x64 -s false -o artifacts" in /tmp/tmpcfupmjmj/anomaly-detection
+ getopts :h:v:s:o:a: arg
+ case $arg in
+ VERSION=1.1.0
+ getopts :h:v:s:o:a: arg
+ case $arg in
+ ARCHITECTURE=x64
+ getopts :h:v:s:o:a: arg
+ case $arg in
+ SNAPSHOT=false
+ getopts :h:v:s:o:a: arg
+ case $arg in
+ OUTPUT=artifacts
+ getopts :h:v:s:o:a: arg
+ '[' -z 1.1.0 ']'
+ [[ false == \t\r\u\e ]]
+ '[' -z artifacts ']'
+ mkdir -p artifacts
+ ./gradlew assemble --no-daemon --refresh-dependencies -DskipTests=true -Dopensearch.version=1.1.0 -Dbuild.snapshot=false

WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.codehaus.groovy.reflection.CachedClass (file:/usr/share/opensearch/.gradle/wrapper/dists/gradle-6.6.1-all/ejrtlte9hlw8v6ii20a9584rs/gradle-6.6.1/lib/groovy-all-1.3-2.5.12.jar) to method java.lang.Object.finalize()
WARNING: Please consider reporting this to the maintainers of org.codehaus.groovy.reflection.CachedClass
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release

=======================================
OpenSearch Build Hamster says Hello!
  Gradle Version        : 6.6.1
  OS Info               : Linux 4.14.243-185.433.amzn2.x86_64 (amd64)
  JDK Version           : 14 (AdoptOpenJDK)
  JAVA_HOME             : /usr/lib/jvm/adoptopenjdk-14-hotspot
  Random Testing Seed   : A8C414858B0C50D1
  In FIPS 140 mode      : false
=======================================

> Task :compileJava
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.

> Task :processResources
> Task :classes
> Task :jar
> Task :copyPluginPropertiesTemplate
> Task :pluginProperties
> Task :bundlePlugin
> Task :generatePomFileForNebulaPublication
> Task :generatePom

> Task :javadoc
> Task :javadocJar
> Task :sourcesJar
> Task :assemble

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.6.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD SUCCESSFUL in 52s

@peterzhuamazon
Copy link
Member Author

#249
#250
#251

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants