Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

JavaCompileIntegrationTest.test_java_compile_with_corrupt_remote is flaky #8171

Closed
stuhood opened this issue Aug 13, 2019 · 12 comments
Closed

JavaCompileIntegrationTest.test_java_compile_with_corrupt_remote is flaky #8171

stuhood opened this issue Aug 13, 2019 · 12 comments
Labels
flaky-test

Comments

@stuhood
Copy link
Member

stuhood commented Aug 13, 2019 •
edited
Loading

I had seen this a few times, but finally got around to reporting it.

 _ JavaCompileIntegrationTest.test_java_compile_with_corrupt_remote _
 
 self = <pants_test.backend.jvm.tasks.jvm_compile.java.test_java_compile_integration.JavaCompileIntegrationTest testMethod=test_java_compile_with_corrupt_remote>
 
     def test_java_compile_with_corrupt_remote(self):
       """Tests that a corrupt artifact in the remote cache still results in a successful compile."""
       with self.temporary_workdir() as workdir, temporary_dir() as cachedir:
         with cache_server(cache_root=cachedir) as server:
           target = 'testprojects/tests/java/org/pantsbuild/testproject/matcher'
           config = {
               'cache.compile.rsc': {
                 'write_to': [server.url],
                 'read_from': [server.url],
               },
           }
     
           # Compile to populate the cache, and actually run the tests to help verify runtime.
           first_run = self.run_pants_with_workdir(['test', target], workdir, config)
           self.assert_success(first_run)
           self.assertTrue("Compiling" in first_run.stdout_data)
     
           # Build again to hit the cache.
           second_run = self.run_pants_with_workdir(['clean-all', 'test', target], workdir, config)
           self.assert_success(second_run)
 >         self.assertFalse("Compiling" in second_run.stdout_data)
 E         AssertionError: True is not false
 
 .pants.d/pyprep/sources/c4f8df83f07678107803ea5e6ce47a913857e1d2/pants_test/backend/jvm/tasks/jvm_compile/java/test_java_compile_integration.py:226: AssertionError
 -------------- Captured stderr call --------------
 127.0.0.1 - - [12/Aug/2019 23:34:49] "HEAD / HTTP/1.1" 307 -
 127.0.0.1 - - [12/Aug/2019 23:34:49] "HEAD / HTTP/1.1" 307 -
 127.0.0.1 - - [12/Aug/2019 23:34:49] "GET /2c5549f0ca6436d3d76ccaeb35104466e3c2ae23/testprojects.tests.java.org.pantsbuild.testproject.matcher.matcher/dc076e13c995ebb45f7747cf70957f164de38cbb-ResolvedJarAwareFingerprintStrategy.1ced22b2ae0d_0c0b09a8b3c3.tgz HTTP/1.1" 307 -
 127.0.0.1 - - [12/Aug/2019 23:34:49] code 404, message File not found
 127.0.0.1 - - [12/Aug/2019 23:34:49] "GET /2c5549f0ca6436d3d76ccaeb35104466e3c2ae23/testprojects.tests.java.org.pantsbuild.testproject.matcher.matcher/dc076e13c995ebb45f7747cf70957f164de38cbb-ResolvedJarAwareFingerprintStrategy.1ced22b2ae0d_0c0b09a8b3c3.tgz/__redir__ HTTP/1.1" 404 -
 127.0.0.1 - - [12/Aug/2019 23:34:49] "GET /2c5549f0ca6436d3d76ccaeb35104466e3c2ae23/testprojects.tests.java.org.pantsbuild.testproject.matcher.matcher/dc076e13c995ebb45f7747cf70957f164de38cbb-ResolvedJarAwareFingerprintStrategy.1ced22b2ae0d_0c0b09a8b3c3.tgz HTTP/1.1" 307 -
 127.0.0.1 - - [12/Aug/2019 23:34:49] code 404, message File not found
 127.0.0.1 - - [12/Aug/2019 23:34:49] "GET /2c5549f0ca6436d3d76ccaeb35104466e3c2ae23/testprojects.tests.java.org.pantsbuild.testproject.matcher.matcher/dc076e13c995ebb45f7747cf70957f164de38cbb-ResolvedJarAwareFingerprintStrategy.1ced22b2ae0d_0c0b09a8b3c3.tgz/__redir__ HTTP/1.1" 404 -
 127.0.0.1 - - [12/Aug/2019 23:36:16] "HEAD /2c5549f0ca6436d3d76ccaeb35104466e3c2ae23/testprojects.tests.java.org.pantsbuild.testproject.matcher.matcher/dc076e13c995ebb45f7747cf70957f164de38cbb-ResolvedJarAwareFingerprintStrategy.1ced22b2ae0d_0c0b09a8b3c3.tgz HTTP/1.1" 307 -
 127.0.0.1 - - [12/Aug/2019 23:36:16] code 404, message File not found
 127.0.0.1 - - [12/Aug/2019 23:36:16] "HEAD /2c5549f0ca6436d3d76ccaeb35104466e3c2ae23/testprojects.tests.java.org.pantsbuild.testproject.matcher.matcher/dc076e13c995ebb45f7747cf70957f164de38cbb-ResolvedJarAwareFingerprintStrategy.1ced22b2ae0d_0c0b09a8b3c3.tgz/__redir__ HTTP/1.1" 404 -
 127.0.0.1 - - [12/Aug/2019 23:36:17] "PUT /2c5549f0ca6436d3d76ccaeb35104466e3c2ae23/testprojects.tests.java.org.pantsbuild.testproject.matcher.matcher/dc076e13c995ebb45f7747cf70957f164de38cbb-ResolvedJarAwareFingerprintStrategy.1ced22b2ae0d_0c0b09a8b3c3.tgz HTTP/1.1" 307 -
 127.0.0.1 - - [12/Aug/2019 23:36:55] "HEAD / HTTP/1.1" 307 -
 127.0.0.1 - - [12/Aug/2019 23:36:55] "HEAD / HTTP/1.1" 307 -
 127.0.0.1 - - [12/Aug/2019 23:36:55] "GET /2c5549f0ca6436d3d76ccaeb35104466e3c2ae23/testprojects.tests.java.org.pantsbuild.testproject.matcher.matcher/dc076e13c995ebb45f7747cf70957f164de38cbb-ResolvedJarAwareFingerprintStrategy.1ced22b2ae0d_0c0b09a8b3c3.tgz HTTP/1.1" 307 -
 127.0.0.1 - - [12/Aug/2019 23:36:55] code 404, message File not found
 127.0.0.1 - - [12/Aug/2019 23:36:55] "GET /2c5549f0ca6436d3d76ccaeb35104466e3c2ae23/testprojects.tests.java.org.pantsbuild.testproject.matcher.matcher/dc076e13c995ebb45f7747cf70957f164de38cbb-ResolvedJarAwareFingerprintStrategy.1ced22b2ae0d_0c0b09a8b3c3.tgz/__redir__ HTTP/1.1" 404 -
 127.0.0.1 - - [12/Aug/2019 23:36:55] "GET /2c5549f0ca6436d3d76ccaeb35104466e3c2ae23/testprojects.tests.java.org.pantsbuild.testproject.matcher.matcher/dc076e13c995ebb45f7747cf70957f164de38cbb-ResolvedJarAwareFingerprintStrategy.1ced22b2ae0d_0c0b09a8b3c3.tgz HTTP/1.1" 307 -
 127.0.0.1 - - [12/Aug/2019 23:36:55] code 404, message File not found
 127.0.0.1 - - [12/Aug/2019 23:36:55] "GET /2c5549f0ca6436d3d76ccaeb35104466e3c2ae23/testprojects.tests.java.org.pantsbuild.testproject.matcher.matcher/dc076e13c995ebb45f7747cf70957f164de38cbb-ResolvedJarAwareFingerprintStrategy.1ced22b2ae0d_0c0b09a8b3c3.tgz/__redir__ HTTP/1.1" 404 -
 127.0.0.1 - - [12/Aug/2019 23:37:01] "HEAD /2c5549f0ca6436d3d76ccaeb35104466e3c2ae23/testprojects.tests.java.org.pantsbuild.testproject.matcher.matcher/dc076e13c995ebb45f7747cf70957f164de38cbb-ResolvedJarAwareFingerprintStrategy.1ced22b2ae0d_0c0b09a8b3c3.tgz HTTP/1.1" 307 -
 127.0.0.1 - - [12/Aug/2019 23:37:01] code 404, message File not found
 127.0.0.1 - - [12/Aug/2019 23:37:01] "HEAD /2c5549f0ca6436d3d76ccaeb35104466e3c2ae23/testprojects.tests.java.org.pantsbuild.testproject.matcher.matcher/dc076e13c995ebb45f7747cf70957f164de38cbb-ResolvedJarAwareFingerprintStrategy.1ced22b2ae0d_0c0b09a8b3c3.tgz/__redir__ HTTP/1.1" 404 -
 127.0.0.1 - - [12/Aug/2019 23:37:02] "PUT /2c5549f0ca6436d3d76ccaeb35104466e3c2ae23/testprojects.tests.java.org.pantsbuild.testproject.matcher.matcher/dc076e13c995ebb45f7747cf70957f164de38cbb-ResolvedJarAwareFingerprintStrategy.1ced22b2ae0d_0c0b09a8b3c3.tgz HTTP/1.1" 307 -
 127.0.0.1 - - [12/Aug/2019 23:37:02] "PUT /2c5549f0ca6436d3d76ccaeb35104466e3c2ae23/testprojects.tests.java.org.pantsbuild.testproject.matcher.matcher/dc076e13c995ebb45f7747cf70957f164de38cbb-ResolvedJarAwareFingerprintStrategy.1ced22b2ae0d_0c0b09a8b3c3.tgz/__redir__ HTTP/1.1" 200 -
 - generated xml file: /home/travis/build/pantsbuild/pants/.pants.d/test/pytest/tests.python.pants_test.backend.jvm.tasks.jvm_compile.java.java_compile_integration/junitxml/TEST-tests.python.pants_test.backend.jvm.tasks.jvm_compile.java.java_compile_integration.xml -
 ============ short test summary info =============
 FAIL tests/python/pants_test/backend/jvm/tasks/jvm_compile/java/test_java_compile_integration.py::JavaCompileIntegrationTest::test_java_compile_with_corrupt_remote
 1 failed in 145.31 seconds
@stuhood stuhood mentioned this issue Aug 13, 2019
Merged
@stuhood
Copy link
Member Author

stuhood commented Aug 21, 2019

Seen again in master.

1 similar comment
@stuhood
Copy link
Member Author

stuhood commented Aug 22, 2019

Seen again in master.

@stuhood
Copy link
Member Author

stuhood commented Aug 29, 2019

Seen again in #8221.

@stuhood
Copy link
Member Author

stuhood commented Aug 30, 2019

Seen again in #8225.

@stuhood
Copy link
Member Author

stuhood commented Sep 4, 2019

Seen again in #8240.

@itsmesatwik
Copy link

Seen again in #8101

@Eric-Arellano
Copy link
Contributor

Seen twice in a row in #8210.

@stuhood
Copy link
Member Author

stuhood commented Sep 17, 2019

Seen again in master.

@jsirois
Copy link
Contributor

jsirois commented Sep 23, 2019 •
edited
Loading

And again in master.

@stuhood
Copy link
Member Author

stuhood commented Sep 26, 2019

Seen again in #8282.

jsirois added a commit to jsirois/pants that referenced this issue Dec 10, 2019
@jsirois
This test has been flaky far too long.
aaac880 
First step towards killing the test.

pantsbuild#8171
@jsirois jsirois mentioned this issue Dec 10, 2019
Merged
@jsirois
Copy link
Contributor

jsirois commented Dec 10, 2019

Seen again on master ^ ... dead man walking.

jsirois added a commit that referenced this issue Dec 10, 2019
@jsirois
This test has been flaky far too long. (#8788)
2d6001a 
First step towards killing the test.

#8171
@jsirois jsirois self-assigned this Mar 24, 2020
jsirois added a commit to jsirois/pants that referenced this issue Mar 25, 2020
@jsirois
Skip all flaky tests with GitHub issues.
ec40520 
This brings our skip count up from 7 python and 2 rust to 25 python and
4 rust, hopefully in the name of stability as outlined in pantsbuild#9386.

Also updates the How to Contribute docs with a pointer to pantsbuild#9386 for
the latest procedure.

Includes flaky tests from:
 pantsbuild#9313 pantsbuild#9312 pantsbuild#8679 pantsbuild#8678 pantsbuild#8520 pantsbuild#8520 pantsbuild#8520 pantsbuild#8405 pantsbuild#8193 pantsbuild#8171 pantsbuild#7836 pantsbuild#7622
 pantsbuild#7573 pantsbuild#7572 pantsbuild#7554 pantsbuild#7547 pantsbuild#7457 pantsbuild#7168 pantsbuild#7149 pantsbuild#6838 pantsbuild#6829 pantsbuild#6787 pantsbuild#6114

[ci skip-jvm-tests]  # No JVM changes made.
@jsirois jsirois mentioned this issue Mar 25, 2020
Merged
jsirois added a commit that referenced this issue Mar 25, 2020
@jsirois
Skip all flaky tests with GitHub issues. (#9391)
f393afd 
This brings our skip count up from 7 python and 2 rust to 25 python and
4 rust, hopefully in the name of stability as outlined in #9386.

Also updates the How to Contribute docs with a pointer to #9386 for
the latest procedure.

Includes flaky tests from:
 #9313 #9312 #8679 #8678 #8520 #8520 #8520 #8405 #8193 #8171 #7836 #7622
 #7573 #7572 #7554 #7547 #7457 #7168 #7149 #6838 #6829 #6787 #6114
@jsirois jsirois removed their assignment Mar 25, 2020
@Eric-Arellano
Copy link
Contributor

Test was deleted in Pants 2.0.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
flaky-test
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@stuhood @jsirois @Eric-Arellano @itsmesatwik