Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test improvements #21

Closed
zifeo opened this issue Dec 17, 2023 · 4 comments
Closed

Test improvements #21

zifeo opened this issue Dec 17, 2023 · 4 comments

Comments

@zifeo
Copy link
Member

zifeo commented Dec 17, 2023

  • the installs folder should be persisted between restart so it can be cached (locally and in the github runner)
  • how to run a specific test only?
  • write a test for each os/shell combination to test the hooks
  • should the test fail if there is an error in the ghjk logs?
@Yohe-Am
Copy link
Contributor

Yohe-Am commented Dec 20, 2023

@zifeo

  1. There's actually not much to cache in the installs folder as each test is run in isolation and there's not a lot of setup beforehand. I.e. there isn't much to test if we cache the install.
    • For the local runner, (enabled using $GHJK_TEST_E2E_TYPE=local and used on macos runners), we use the deno bin on the system and thus share it's module cache. I suppose I can add a cache step on DENO_DIR in the runners but currently, it's very lightweight
    • For the docker based suite (enabled using $GHJK_TEST_E2E_TYPE=docker and used on ubuntu-latest runners), I've changed the debian based deno image to alpine to make it even more lightweight. We can add docker image caching if the tests get heavier but I don't expect that to happen. And yeah, most of the intermediate steps like pulling in the deno dependencies are cached across a single run for the e2e tests (through the default docker layer caching mechanism).
  2. This is possible using deno task test --filter filter. That task straights goes to the deno test command and the suite is built upon Deno.test.
  3. WIP in refactor(ports)!: streamlining and python support #14.
  4. WIP in refactor(ports)!: streamlining and python support #14.

@Yohe-Am Yohe-Am linked a pull request Dec 20, 2023 that will close this issue
8 tasks
@Yohe-Am Yohe-Am removed a link to a pull request Dec 21, 2023
8 tasks
@zifeo
Copy link
Member Author

zifeo commented Dec 21, 2023

@Yohe-Am 👍

  1. I thought more of caching the actual ports download, maybe this needs a micro "proxy" to work efficiently on local and docker test driver

@Yohe-Am
Copy link
Contributor

Yohe-Am commented Dec 21, 2023

caching downloads

I'd been thinking about that myself. Nix demands that you provide a check sum of sorts for any network download. I considered this and the donwloadUrls helpers avail on the GithubReleasePorts are an aborted attempt in that direction after I the asdf edge case showed itself again. Maybe we can provide a Deno based curl tool that adheres to the interface or maybe there are environment vars we can set for actual curl but the whole thing seemed like a best effort hack.

In the context of testing,same complications arise. Since the ports might use plethora of methods for their downloads, (curl, wget, aria2), we'll have to mess with iptables or something system specific like that to make sure they use the proxy. We also want to test that the addresses they're calling out are still valid.

@zifeo
Copy link
Member Author

zifeo commented Dec 21, 2023

maybe there is an easy way with https://docs.deno.com/runtime/manual/basics/modules/proxies + https://gist.github.com/osy/30c5c96d7575efd1d2a2db5e3def0815 (no need to patch squid though, or use another local caching proxy like https://github.com/hauke96/tiny-http-proxy)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants