Overlay never using binary cache #122
Comments
|
I'm having the same issue here! |
|
At the risk of piggy-backing on a tangentially-related issue, it also seems that the jobsets don't include darwin, so MacOS users won't ever get cache hits currently. I wonder what options might exist for that, e.g. just using github actions with cachix to routinely build and push a variety of popular outputs for that platform. |
|
seems OK now (for |
|
I think I managed to find the culprit earlier today. It's been running smoothly since nix-community/infra@b0a9ac3. I'm closing this for now. Please reopen if this issue still persists (I will monitor the build infra to see if this issue pops up again). |
@purcell The I'm reluctant to use github actions for this for various reasons:
|
|
Works fine for me now, thank you @adisbladis !! |
|
@adisbladis how can you extract the |
|
@adisbladis Ah yes, I remember now that that was a concern. @mjlbach Does that mean you have a cachix with Darwin emacsGcc artifacts that other people (me!) could use? |
|
Yes :) You have to use my fork of the overlay, but it should otherwise work! The install instructions aren't accurate since I didn't modify them from nix-community, but if you s/nix-community/mjlbach in all the right places it should work fine (and let me know if it doesn't!). |
|
Thanks, I'll give that a go. |
|
Thanks for looking into this! 😄 Getting the cache up and running again is really a nice quality of life improvement. However, I wanted to report that I'm still experiencing this today. That said, I thought I saw the cache working yesterday, so it might be something wrong with the configuration between two different systems. I use the same config as OP (just with different packages), so I'd expect it to work on my end too. Could there be any more issues/regressions that have cropped up or am I the only one experiencing this now (in which case it's probably just user error)? Thanks! |
Some cache misses are always expected as NixOS channels and the nix-community hydra instance are not in perfect sync. It doesn't look like we have any more regressions in the build infra. |
|
Ah, okay, thanks for the clarification! Trying again now it seems to have worked. Thanks! 🙇 |
|
It was also working for me, but I just tried upgrading the system and got a cache miss. Presumably, because GitLab CI is still running on the most recent commit (or does it have nothing to do with the binary cache)?
In case of flakes would pinning the NixOS channel solve that? I just noticed that this project doesn't have Also, is there a way to specify an older cached commit and use it without rebuilding Emacs? I looked over https://hydra.nix-community.org/jobset/emacs-overlay/unstable-gcc-pkgs and tried using the latest evaluation that did not error (which at the time was on this commit dcb4f8e). But that didn't seem to help. Am I doing something wrong? EDIT: I think I solved the issue with using an older cached commit, in my NixOS configuration I changed the nixpkgs revision to 1f77a4c8c74bbe896053994836790aa9bf6dc5ba (which I found in evaluation inputs). |
|
@jul1u5 I had to do the same. It would be nice if there was a way to "match" this repo automatically, but idk if that's a thing. |
|
This issue has been mentioned on NixOS Discourse. There might be relevant details there: https://discourse.nixos.org/t/use-nix-community-cachix-cache-for-emacs-overlay/11710/5 |
Adding note: I am amazed and extremely grateful for the work that had been done in the nix world to make this all possible! |
|
I just found an elegant solution to this problem: instead of asking for the tarball of the current More specifically, I changed the overlay's URL in my config to If the latest commit on If the latest commit on This way I'm still tracking |
|
Note that the above works only if your channel is at the same version as the one used for the build, so if you're still getting a cache miss with the above, try updating the channel first. |
|
@zeorin Nice! That seems to work well 😄 I didn't realize you could do that! can you explain the syntax? It seems to just be a URL-encoded "2 hours ago". Is that a github thing or a nix thing? Must be the former, right? |
|
@thomasheartman it's a git thing, and it turns out that GitHub supports it, too. From I did try it without the URL encoding, first, but that didn't work. |
|
Oh, cool! The more you know 😄 Thanks! That could very well come in handy at some point in the future. |
|
Note that hydra is currently broken: #204, so for the time being I've pinned it like this: import (builtins.fetchTarball {
url =
"https://github.com/nixos/nixpkgs/archive/0ecf7d414811f831060cf55707c374d54fbb1dec.tar.gz";
}) {
config = config.nixpkgs.config;
overlays = [
(import (builtins.fetchTarball {
url =
"https://github.com/nix-community/emacs-overlay/archive/46c8425e72efd4002343df959ba1ce47381e1eca.tar.gz";
}))
];
}; |
|
Will write down how to find combination of nixpkgs and emacs-overlay for which cachix has prebuilt artifact:
Suspect that everyone in this thread already knows this, but I only found out about that 6 months after first encountering this |
so that really isn't guaranteed to be in nix-community cache. what you can do is try in nix shell first. example, as of 2022-07-23T07:45 UTC, latest build for only if i try one of the 2 in the end of the first page it will hit cache so try that method with nix shell first before setting to your inputs |
|
@yuuyins Do you know if there is any way to automate that process or otherwise make it so that you don't need to do it manually everytime? Preferebly just make it so that when building, it's checked automatically and adjusted to the latest revision in the cache? |
|
@thomasheartman well that's not so simple; because a flake input can have multiple derivations and how simple would it be to get a revision that has available cache for all specified derivations? let's say you just want for one specific derivation. it would need to use cachix and and hydra apis to find which revision has cache and then put stuff into flake.cache that flake.nix inputs would reference which flake lock would then do its magic. this is a workaround; the best solution would be this integrated directly into |
just investigating this more, this is wrong. https://hydra.nix-community.org/build/16566209 and i can get the nar.xz but for whatever the reason with nix it won't hit cache |
|
Maybe you have a negative cache hit stored in your nix. will get rid of all caches. |
|
@Mic92 i got rid of that directory. then https://hydra.nix-community.org/build/16583641 but same issue can't you reproduce this issue? |
|
Is it not just the case you are just using a different nixpkgs version from what CI used? The nixpkgs version is not pinned here. |
|
@Mic92 yes. So we really need to have both nixpkgs and emacs-overlay pinned to the same revision as the one used by Hydra? It's not feasible to me to update nixpkgs nixos revision whenever i need to update emacs-overlay. can't i use an extra nixpkgs flake input just for emacs-overlay to use? is there an option for it? is there a way to test it in |
|
I think it would help if we would build emacs from the flake with nixpkgs beeing pinged, but I think @adisbladis does not like flakes for some reason. Anyway, here you can see that cachix has emacs packages cached: |
|
i've added the e494a908e8895b9cba18e21d5fc83362f64b3f6a to nixpkgs, and now indeed i hit cache in nix shell. so guess that was it, really needs to be in sync. but i also just came to know it's possible to specify a nixpkgs flake input for a flake input like this. edit: this actually doesn't work for hitting the cache... for some reason only specifying top level nixpkgs input will work |
|
This issue has been mentioned on NixOS Discourse. There might be relevant details there: |
|
@Mic92, @adisbladis. Is it possible to just generate/push the flake.lock? This way i believe when we set emacs-overlay, its inputs.nixpkgs would follow what is set in the nix-community/emacs-overlay/flake.lock? wouldn't it? edit: forgot the part in which you implied emacs-overlay isn't built from the flake.. |
|
I am using this script to fetch the latest (2h ago) rev and extracting the relavant nixpkgs rev from the flake.lock #!/usr/bin/env bash
overlay_url_rev="master@{2%20hours%20ago}"
rm -rf /tmp/emacs-overlay/
mkdir -p /tmp/emacs-overlay/
http -F --download \
https://github.com/nix-community/emacs-overlay/archive/${overlay_url_rev}.tar.gz \
| tar xzvf - -C /tmp/emacs-overlay/
overlay_filename=$(ls /tmp/emacs-overlay/)
overlay_rev=${overlay_filename##*-}
nixpkgs_rev=$(cat /tmp/emacs-overlay/${overlay_filename}/flake.lock | jq -r '.nodes.nixpkgs.locked.rev')
nix flake lock --override-input nixpkgs github:NixOS/nixpkgs/${nixpkgs_rev}
nix flake lock --override-input emacs-overlay github:nix-community/emacs-overlay/${overlay_rev} |
In my home manager I have the following:
I have my channels updated (not that I think it would matter since I'm importing it directly) and have my cachix set to
nix-community.Still, though, nix refuses to pull from the cache and tries to build emacs itself. Is there anything I'm missing here?
The text was updated successfully, but these errors were encountered: