Base for devos

[blaggacao]

I'm speaking purely on behalf of myself, but I would find it excitinh if this can be brought into a shape that it will be easy for devos to built upon.

I see the great powers that both projects combined could yield out of flakes and for its users.

I sense devos is at a moment where it would wish to consolidate its library and declare concise and reusable interfaces.

It looks like you (coming from the other end of flake-utils) already have gone down that route.

Some time ago, I tried to achive something similar myself with numtide/flake-utils#18. But the whole PR and it's shape perfectly reveals where I stand in terms of nix-skills 😉

So I would love to see a good bunch of collaboration striving for getting this and devos, both in a shape where devos could adopt / inherit this library without braking any features.

I think for devos it would be good / convenient, until reaching 1.0 to have such a well-defined (and well-guarded) library in-tree.

@gytis-ivaskevicius what do you think?

@nrdxp what do you think?

@Pacman99 what do you think?

[Pacman99]

I like the idea and I think it would be really cool, but I do see some potential issues with this. I think @gytis-ivaskevicius direction for this project is fairly different from devos and if we rely on this is as a base it would likely hurt their ability to experiment and solidify this project. It is nice for devos library to be in tree since we still have a long way to go for a stable api, and being able to change library functions and add new features in the same PR/commit is really helpful.

But I do love that we have connected these two projects. I think we can all benefit from bouncing ideas off of each other to figure out the best practices. As @nrdxp puts it, the goal is a "community consensus". So I think this issue thread would be a great place to communicate between each other. And eventually decide if we want to incorporate the two somehow, or if we decide not to then at least maintain some collaboration.

[blaggacao]

• I think we can steal the naming variant of shared prefix, like in sharedOverlays, etc. instead of extern. "Shared" is "shared in my whole environment", "extern" is "extern to my own stuff", but inputs.something-different-than-self does make rhis already clear. So while "extern" is somewhat redundant "shared" is extra information.

• I like the Builder suffix as it clearly reveals the builder paradigm. In devos the ideas of the builder paradigm are somewhat blurred. The task a builder acomplishes (dependency injection into a function) is still distributed across many places and the api is diluted vs. the clarity of a builder mkSystemFlake = nixosConfigurationBuilder { channel } mkHomeManagerFlake = hmConfigurationBuilder { channel }. The builder pattern makes high level dependency injection (channels) really blissfully self-evident. Big 👍

• The code example leads me to another point: the layered approach (genericConfigurationBuilder / nixosConfigurationBuilder). I think @Pacman99, reading you between the lines in dofferent places, my understanding is that you also have a need or would very much welcome such a layered approch (e.g. mkHmConfiguration = hmConfigurationBuilder)

[Pacman99]

Yeah I like the builder too. But I was thinking for devos to instead take a more nixpkgs style approach. Maybe you could pass overlays for devos that override lib functions. Devos 'builders' are blurred because each builder is very unique and does a lot, so I'm not sure builders is the best approach there.

Yes I would! Proper home manager support is something I want in devos(we're almost there!). And I have a PR open to add mkHomeConfiguration as a lib function. And a layered approach would be nice and a generic configuration builder would be cool. Perhaps it could be used for the isoConfig too.

Also we should probably move this discussion to the devos repo, since this issue is more relevant to how devos should work than about flake-utils-plus.

[gytis-ivaskevicius]

I think that it would be really cool to collaborate on something like this and I would like to note that I do not see much if any issues with DevOS implementing this particular flake, but that would quite possibly result in code redundancy due to the fact that DevOS strongly enforces file structure and flake-utils-plus leaves it up to the user to decide.

Anyways, since you guys piqued my curiosity (And that part of me that wishes to show off Nix superiority :D ) I will spend some time working on my version of DevOS and I guess after that we will compare the results :)
So I guess here are the candidates:

1. Original
2. Pacman99 take
3. My take (and yes, this is THE greatest repository name anyone can think of)

If anyone wishes to reach me - probably the best way to do so is via Discord @Gytis#0001 (Tho I am not closing this issue, or at very least not until the issue on devos GitHub gets created)

Also, later today I will create some new branch on this repo in which ill get rid of deprecated code (I left most of the old code there to preserve a smile on early adopters faces), and also I will tweak the existing code structure so it would be easier to customize it and implement in use cases like this one. So if anyone wishes to join in - use that

[blaggacao]

Sounds all pretty good!

DevOS strongly enforces file structure and flake-utils-plus leaves it up to the user to decide.

DevOS strives to also offer a congruent file system layout to a broader community. In a global social sharing model — one of the (social) dimensions of nix superiority — it will be (quite) easier for people to browse over each other's config.

On the other hand, I do think that the scattered way it is currently implemented is in need of some abstraction. Or in other words, the FSH needs to be implemented upfront of builders and builders need to become strictly unaware of this concern. @Pacman99 's take already nicely implements this.

[nrdxp]

I'd be glad to move forward on this and build off existing solutions rather than try and do everything in house (exactly what I was trying to eliminate with devos in the first place) and I'm well aware of the uninviting complexity that has evolved over time.

The main thing I've been pondering lately as well (part of the reason I haven't made any major changes recently) is how to properly reduce it. I'm glad I'm not the only one thinking about this and I'm sure we'd come up with a better solution together.

I will probably be fairly short on time for the coming month at least though, so I'm thinking about dishing out write access to large contributors of DevOS to keep the PR's moving. @gytis-ivaskevicius, if you are really willing to use your work and integrate it, I'd be glad to include you in this, as it seems our goals are aligned.

[gytis-ivaskevicius]

Finally got around to all this. Just reviewed @Pacman99 PR, looks really nice 👍 divnix/digga#206 (review)

@gytis-ivaskevicius, if you are really willing to use your work and integrate it, I'd be glad to include you in this, as it seems our goals are aligned.

Our goals seem to be aligned, but flake-utils-plus will never beat a custom-built solution like @Pacman99 built since our approach is a little different, here are few core differences:

• flake-utils-plus is a simple abstraction that is more of a "choose your own adventure" kind of thing.
• DevOS - is a whole framework that introduced additional structure and concepts - for example "suites" - in flake-utils-plus world this would be something along the lines of this https://github.com/gytis-ivaskevicius/nixfiles/blob/master/flake.nix#L46-L55 or in case of a user having lots of suites to deal with - it could be specified in some suites.nix file or something along those lines. Heck, maybe some users prefer mySuiteName.nix with content as such: {...}: { imports = [ ./modules/a.nix ./modules/b.nix]; }. In other words - I leave it up to the user to figure out.
• flake-utils-plus has a limitation of not depending on nixpkgs (Its because I hope one day to merge it all to flake-utils repo)
• @Pacman99 did an excellent job with adding type safety which I can not do due to the point above

[Pacman99]

That is a really good comparison and covers my exact opinion on the issue. I don't know if I personally will use flake-utils-plus but I really like the project and I think it is a very positive addition to the nix flakes ecosystem.
I think a similar analogy can be gnome vs tiling WM/kde:

• When you use gnome, you end up following a specific format of desktop usage that the gnome devs prescribe and if you don't like it, you usually have ask for the format to be changed upstream - or send changes upstream yourself.
• While with i3 or other tiling wm's you can change pretty much anything without an actual change in upstream i3. And similarly when using kde, there are so many options to tweak that you can change your desktop format without an upstream kde change.

analogy probably isn't perfect, but I think it works

Devos is some people working together towards a common format(gnome), while flake-utils-plus is more of a simpler library that gives you the tools to create your own format(i3/kde). Even though my PR does attempt to allow you to change the filesystem format, in the end you are still following the devos method of suites/profiles and other devos concepts.

[blaggacao]

Its because I hope one day to merge it all to flake-utils repo.

Nice! Strategically, to meet the "community consensus" goals, anything DevOS does needs to be in a position to be rebased on your work, once that happens.

All in all, according to the goals, DevOS' framework-ish parts are better suited to be a value-added instead of an alternative.

So let's make sure DevOS is a prime stakeholder of any emerging library implementation, that might "graduate" into numtide/flake-utils.

numtide/flake-utils — power of its widespread adoption — in a way acts as a quality gateway and strong community signal, here, and therefore is an important building block for achieving our "community consensus" goals.

[blaggacao]

divnix/digga#232

I also created a couple of issues, that I think we need to let settle in and evaluate in order to polish the mutual interfaces:

• Importers #8 (importers)
• config generator (nixos) #9 (config generator nixos → we might be able to implement everything as extra modules, though)
• config generators (generalized) #10 since fup cannot depend on any additional input (to get upstream consideration), we might agree on barring other configurators from fup itself?
• sharing model #12 (inputs is a strategically dangerous dilution of the very flake's top level api, is it? — no clear emerging answer on the flake sharing model, yet.)
• extraArgs is marked with a warning in upstream #13 (extraArgs vs _module.args & specialArgs)
• nixosConfiguration escape hatch #14 (fixed by Fix #14 #15 — put trivial escape hatch out of service)
• profiles #16 (nixosProfile arg naming issue) solved via ref: profile -> hostAttrs #23 and folks

[gytis-ivaskevicius]

Will go over all that later today and maybe ill finally put some work into 1.1.0 release (staging branch). I am a little busy these days but I'm sure ill find some time for it.

[blaggacao]

At any rate, take them with a grain of salt. On the API PR that I also linked above, you might find some more context to see where we're currently standing.

[gytis-ivaskevicius]

I'll look into it once I get home. I was under impression that DevOS will use purpose-built implementation written by pacman99.
If this is going to become part of DevOS - I guess quite a bit more cooperation from my side is required.
Also, I might as well hook you up with write access to the repo. (Tho you will not be allowed to push to master or anything like that)

[Pacman99]

To be honest, you have already made a good enough api which fits well with our goal. I have a few changes I plan to send as PR's, but I would say those are overall improvements to flake-utils-plus and not even specific to devos.
So consider us just a larger user of flake-utils-plus, and users who are very willing to contribute back :)
Other than reviewing, we will try not to put too much work for this on you.

[gytis-ivaskevicius]

Okay, a couple of things:

• Both of you have write access, master branch is protected
• I trust you guys enough to do merges yourselves
• No matter the change I'd like to see examples updated accordingly, its just a nice overview
• I don't think that patching currently works on staging (It is quite an annoying thing to maintain, not sure if we are supporting it :/ ) Related: Support flake references to patches NixOS/nix#3920
• If you guys have a Discord or something along those lines - I feel like it would be useful to create a bit more intimate communication channel between us ;) My Discord tag is Gytis#0001
• For a proper review of PR's - I think the only solution is to actually properly test it. Maybe at this point, some actual tests should be created, even if it's semi-manual. Main test cases:

1. nix flake show
2. nix build /path/to/flake#nixosConfigurations.HOSTNAME.config.system.build.toplevel
3. Possibly evaluate a few attributes nixosConfigurations.HOSTNAME.config (honestly, probably this test case is not necessary, not sure :/ )

• If you guys are up for it - I'm pretty sure that lib argument is kinda needed for all this. (It should be passed as flake output and should become part of user system configurations as well)

What exactly would you guys like me to do/work on?

[Pacman99]

No matter the change I'd like to see examples updated accordingly, its just a nice overview

I realize now that examples are your method of documentation, so I will make sure to update existing PR's before merging.

I don't think that patching currently works on staging (It is quite an annoying thing to maintain, not sure if we are supporting it :/ ) Related: NixOS/nix#3920

patching is cool, but overlays usually does the job for most cases. And then there are those instances when you just copy a file from nixpkgs and edit it locally. So I'm not against removing it, but its up to you.

If you guys have a Discord or something along those lines - I feel like it would be useful to create a bit more intimate communication channel between us ;) My Discord tag is Gytis#1

Mine's ParthivS#2050, I think your missing something, that username doesn't exist for me. But go ahead and send me a request.

For a proper review of PR's - I think the only solution is to actually properly test it. Maybe at this point, some actual tests should be created, even if it's semi-manual. Main test cases:

My brother uses flake-utils-plus, so I just set his utils input to a path on my system and have been testing with that. So my PR's should be tested thoroughly. I usually just do a nix flake check as that does most testing for you. But I can start checking if the configuration builds too.

What exactly would you guys like me to do/work on?

For the host rename, it might be better for you to do them, since we don't know all about where profiles is used in the repo, and also I want to leave the final rename decision to you. But I think @blaggacao has already outlined a lot of goals as github issues.

[gytis-ivaskevicius]

May I get some sort of update from DevOS side of things as well as some update on fup issues and status of these branches - what's holding them back? https://github.com/gytis-ivaskevicius/flake-utils-plus/branches (Also some of those branches are crying for rebase)

[blaggacao]

We are already sketching on first attempts to rebase, I think what currently blocks us is:

• imp: provide per channel modules paths to make it easy to "backport" #42 General exporter and builderConstructor improvements #43 embed info in channels and pass host's channel as specialArg #44

Overall good progress and we are in the final round.

[gytis-ivaskevicius]

I was looking over devos/lib just now. I really don't understand a lot of it :D (like use-cases of half of those functions) but I am quite confident that flake-utils-plus would like to steal these two specimens:
https://github.com/divnix/devos/blob/api-next/lib/devos/recImport.nix
https://github.com/divnix/devos/blob/api-next/lib/devos/devosSystem.nix (I'm thinking about filterAttrs over inputs and if required input exist - add hm/iso entries. WIll play around with its implementation someday)

[blaggacao]

Make sure to look at api-next branch :-)

The recImport is an "importer", which could become a flake-utils-plus lib category of its own ("importer"). But there are a couple more.

On the other hand that's probably fup becoming slowly equivalent to devos so at some point merging fup into devos might just become the easier (and wiser *) thing to do.

* we should avoid to present the community with equivalent alternatives, since in open source there is no competition in price. And the emergence of equivalent alternatives to do all sorts of things all over the place is what already makes this community a bit hostile to newcomers in the first place.

[gytis-ivaskevicius]

Beginning of the project:

Small little helper that will probably get merged with actual flake-utils repo

The current state of the project:

Uhhh, will numtide even going to consider accepting this much of code 👀

[Pacman99]

from the other side: divnix/digga#270

[gytis-ivaskevicius]

Since fup is merged in - do you guys think that this should be closed? I guess the same goes for the issue above?

[blaggacao]

Immediate goal achieved.
👏 👏 👏 👏 👏 👏 👏
🍸 🍸 🍸 🍸 🍸 🍸 🍸