Fix bad EthPM test

[gnidan]

The tests for EthPM (https://github.com/trufflesuite/truffle-core/blob/develop/test/ethpm.js) are very finicky, most notably the test around dependencies. This causes CI to fail erroneously all the time.

This can likely be solved via mocks and/or test flags that disable this test in CI contexts

[gitcoinbot]

This issue now has a funding of 0.3 ETH (369.6 USD) attached to it.

• If you would like to work on this issue you can claim it here.
• If you've completed this issue and want to claim the bounty you can do so here
• Questions? Get help on the Gitcoin Slack
• $17919.44 more Funded OSS Work Available at: https://gitcoin.co/explorer

[vs77bb]

@nlaz Interested in this issue?

[nlaz]

Yeah I'll claim it. Thanks @vs77bb

[gnidan]

Thanks @nlaz!

Ideal solution would be to set up proper mocks for EthPM; let me know what approach you're thinking of before diving in. Appreciate it!

[vs77bb]

@nlaz @gnidan Sounds good, issue is in your hands Niko. @owocki Niko and I are working to get the issue formally claimed on Gitcoin... high gas prices causing a hold up. Told him it's his to run with, though!

[nlaz]

Hey @gnidan thanks for the support. Can you give me an example of an instance where the CI failed because of these tests? This would be helpful to know how to repro the issue. I agree with you that mocking EthPM is the better approach. Disabling tests specifically for CI seems troublesome. After looking at the code, my approach would likely involve giving lib/package.js a small refactor to accept an optional EthPM object in order to mock it in the test. I would then test whether installDependency or install was called correctly for EthPM. Does this sound reasonable to you?

[gnidan]

@nlaz it doesn't look like there are currently any failed CI jobs because of that test - your best bet might be to run that test repeatedly (locally). It's an intermittent failure, but I believe it occurs often enough that you should be able to catch it.

If you can't catch it, remove the retries logic in the test(s).

As far as that approach: it sounds good. I would be interested to see the ramifications outside lib/package.js, e.g. how the EthPM object is initialized and passed in, in the two environments (test / not test). Let me know once you start to make headway and/or if you have any questions!

Thanks!

[nlaz]

@gnidan sadly I am going to back out of this ticket. It's going to take someone more familiar with the codebase than I to avoid accidentally introducing side effects into the code following the above approach. I would recommend either passing in an optional mocked EthPM object into Package.install or stubbing out a global EthPM object in the tests. Hopefully someone else has more luck than I tackling this ticket.

[owocki]

thanks for letting us know @nlaz :)

[ghost]

@gnidan , can you please provide:

Reproduction

The steps (ideally command-line commands) to reproduce the error.

Additionally, its not clear: does the error occour during the tests of the truffle-core code itself, or when a project is created and tested?

Dev System Setup

Instructions on how to setup a development environment in order to contribute to truffle (in general, looks like this is generally missing in the overall project).

[gnidan]

@lazaridiscom:

Thanks for looking to take this on! To address your points:

Dev System Setup: Please see the Truffle project's CONTRIBUTING guide, which should outline the steps you need to get a working local copy of Truffle for development.

Reproduce Steps: Although this won't reliably reproduce the issue every time, you can increase the failure rate of the test by removing this line: https://github.com/trufflesuite/truffle-core/blob/develop/test/ethpm.js#L121. The error occurs in truffle-core, as part of the Package.install() operation, caused by intermittent network problems.

Obtaining more reliable reproduce steps can be considered part of the work if it helps you achieve the goal, but it may not be necessary—mocking EthPM in this test would be sufficient to meet the terms of the bounty, since removing the test's dependency on the network would also remove this intermittent failure.

Let me know if there's anything else I can help with. Thank you!

[ghost]

Thanks for all the info. The contributing guide gives good instructions, project setup is quite easy.

As for the issue, the expected Time-frame: 7 to 14 days (I will dive-in in tiny daily increments, in parallel to other tasks).

For now all fine from my side.

[ghost]

Note that I'm running into WinOS related problems (#105, #106, #107).

As the Truffle website states "Requires NodeJS 5.0+. Works on Linux, macOS, or Windows.", those must be fixed anyway before working on the main problem.

[ghost]

I've isolated the first problem-root, which is surprisingly the sandbox-creation. The reliance on github repos (not git but github) should be reduced, ideally to zero. The problems with http://raw.githubusercontent.com/ seem to be even worser, so this one should be avoided, too (coming to this next days).

[ghost]

TruffleBox.sandbox() works now from local data (on my machine, quick-n-dirty workaround). The related sporadic (or intermittent) errors are gone.

I am now at the point where the sporadic test-error related to EthPM happens. Mocking EthPM will not be necessary, just the GithubExamples functionality of EthPM must be relayed to a local source (away from raw.githubusercontent.com).

For today I take a break to overthink all this, possibly to come up with a conceptually clean concept to avoid such errors in future completely.

[ghost]

GithubExamples.initialize() works now on local data.

After fixing/working around #105, #106, #107, #108, #109, the tests run now off-line on my side.

Now the 2 surprises:

a) Essentially, a EthPM Mock is already available ("Create a fake EthPM host and memory registry").
b) The sporadic test failure is still there.

Further Tasks/Issues

• Debug the sporadic EthPM test-failure in off-line mode
• Provide a unified method to work on local data
• Document Source Code

Those tasks/issues would be in essence new 'bounties', as they are quite important.

(I will continue at this point with the debugging anyway, but i expect that i will hit on more and more issues)

[ghost]

Further Isolation: The sporadic error (causing a timeout, even off-line) is caused here:

https://github.com/trufflesuite/truffle-core/blob/develop/lib/package.js#L69

and goes further into:

https://github.com/trufflesuite/ethpm-js/blob/master/lib/installer.js#L51

I don't think that this should be in any way "mocked away" (thats why I didn't went this path). It should be pinpointed and fixed at all cost, cause such deeply hidden sporadic errors can bring up more trouble, possibly at customer installations.

(giving up for today, headaches...)

[ghost]

The final cause: https://github.com/trufflesuite/ethpm-js/blob/master/lib/preflight.js#L9

[gnidan]

@lazaridiscom: This is quite excellent. Thank you very much for all the research on this, it's extremely helpful. I'm out of town this week, but I will review your efforts in more detail when I get back.

If you get to a point where you can open a PR with your discovered fixes up to this point, we can get that merged and get you the bounty. It looks like the follow-up work is substantial, so let's plan on getting an additional bounty up for all the effort you've put in. (cc @owocki @vs77bb)

[ghost]

@gnidan, you're welcome, and I have to thank you for having this opportunity here. I simply continue here, filing new issues that pop up. As for the bounty stuff, I trust you.

I'll see to it to get everything into PRs, just please be aware that in some cases its not the the PR that is relevant, but the message "that's the solution" or "there's the bug" (leaving the implementation to the core-developers who know the code/architecture best). This could be relevant for your Gitcoin system, too.

[ghost]

• https://github.com/trufflesuite/ethpm-js/blob/master/lib/preflight.js#L43
• https://github.com/trufflesuite/ethpm-js/blob/master/lib/hosts/ipfshost.js#L60

The wget.request within ipfshost causes the timeout (http.request internally). Setting a timeout: 10000 a few lines above wget confirms this.

IPFS server: problem is not with the IPFS local server, as the relevant URI gives a normal response from a browser (during the timeout period).

IPFS host: possibly the ipfs client/host code is responsible (hidden errors like connection-resets etc.)

[ghost]

FINAL ERROR

The sporadic error is caused by the parallel read-operations within ethpm/lib/indexes/github-examples.js, which disrupted the order of the examples. Whenever package transferable was before owner, transferable got a wrong package_name assigned, causing the system to hang (no error message).

FIX

The solution is simple: ensure that the order of the examples is kept:

https://github.com/trufflesuite/ethpm-js/blob/master/lib/indexes/github-examples.js#L55

change:

Object.keys(results).forEach(function(package_name) {

to

examples.forEach(function(package_name) {

Thus the original package-name order is kept.

RESUME

The initial happiness is gone, now comes the bitterness of the so so many time spend on this tiny thing.

All raised issues should be solved, and of course new test-cases should be implemented, in order to catch this bug within the the EthPM test-suite (for the case of future regression).

[ghost]

@gnidan , this gitcoin process kills my creativity/productivity. Possibly you can just use this address here for a manual transfer: 0x61cbbb74a5d61a218c9d39ca270b3232de6a9ebd - thank you!

Edit: or drop me the necessary fees 0xcAB13E6D9a85296e332dBE0ACDfbB917Ae86f111 (that's my MetaMask account).

[gnidan]

@owocki could we get @lazaridiscom this bounty?

@lazaridiscom - sorry that this process has been stressful. I know there is a budget and smart contract functionality for "tipping", but I'll have to sync up with @owocki about how I can do that. definitely want to get you something more than the 0.3 ETH for all the work you've put in. it's been enormously helpful. just please, bear with us as this collaborative effort between Truffle and Gitcoin becomes more fluid. no need to stress about whether you will be paid - you've gone far and above what was expected here, and I will personally make sure you are rewarded accordingly.

[ghost]

@gnidan , no worries, I trust you folks 100%-ly. I was "ranting" (or "communicating unfiltered") a little bit, simply because I really like this gitcoin thing, and I would like the folks there to feel my ( a new users) pain a little bit, thus they fix things.

From you, at this point I would just need the "necessary fees" (the gas, $5 or so) on my MetaMask 0xcAB13E6D9a85296e332dBE0ACDfbB917Ae86f111, so I can continue the usual process on gitcoin (yeah, I fixed this hellish sporadic bug, but I failed to purchase Ether on the exchanges... well...).

(You can place new bounties on the issues I've opened. The workarounds need some work to be converted to real patches.)

[gnidan]

@lazaridiscom sent you some ether from my personal account. consider it the equivalent of me buying you lunch :)

[ghost]

I still don't get all this cryptocoin-stuff. I thought thought that a transfere takes some time (minutes to hours)... but the "lunch" is already available! Thanks a lot!

[gitcoinbot]

Work for 0.3 ETH (261.08 USD) has been submitted by @lazaridiscom.

@lazaridiscom, please leave a comment to let the funder (@owocki) (and the other parties involved) that you've submitted you work. If you don't leave a comment, the funder may expire your submission at their discretion.

• Learn more on the gitcoin issue page
• Questions? Get help on the Gitcoin Slack
• $9056.14 more Funded OSS Work Available at: https://gitcoin.co/explorer

[owocki]

@owocki could we get @lazaridiscom this bounty?

paying out now

@lazaridiscom - sorry that this process has been stressful. I know there is a budget and smart contract functionality for "tipping", but I'll have to sync up with @owocki about how I can do that. definitely want to get you something more than the 0.3 ETH for all the work you've put in.

@gnidan hit me up on slack and we can work it out

[gitcoinbot]

The funding of 0.3 ETH (259.05 USD) attached to this issue has been approved & issued to @lazaridiscom.

• Learn more at on the gitcoin issue page
• Questions? Get help on the Gitcoin Slack
• $8083.29 more Funded OSS Work Available at: https://gitcoin.co/explorer

[ghost]

@owocki , @gnidan , thanks a lot!

[ghost]

@gnidan , this fix is still not applied. I can bet that this causes some other hidded errors somewhere else (caused by ethpm-js):

trufflesuite/ethpm-js#6