`lorri daemon` #81

Profpatsch · 2019-05-14T16:47:28Z

Implements a lorri daemon command listening on a unix socket and a lorri ping command to send a build job to the daemon.

The test is still WIP, blocking on some action.

There’s still a few TODOs left, which will be addressed in a successive PR.

Profpatsch · 2019-05-15T16:50:26Z

Okay, the test should now work.

nix/pre-check.sh

Profpatsch · 2019-05-27T17:38:15Z

Heck! It’s green!

Profpatsch · 2019-05-28T12:29:51Z

Btw, I’m not sure whether we really need the dependency on nix just to get access to the poll function. It’s a very, very thin wrapper around https://docs.rs/crate/libc/0.2.55, so might be best to just copy that one function over.

grahamc

Looks like there are a lot of TODOs to be done. I think it'd be good to get those done prior to a thorough review?

Profpatsch · 2019-05-28T13:36:19Z

I want to get this PR merged, it’s been open for too long.

Since I documented every TODO in the comments, we can fix them in separate PRs.

grahamc · 2019-05-28T14:01:17Z

Getting the existing TODOs todone won't be a substantial lift, and when possible, it is always better to get it done before integration. Let's do it in this PR.

shlevy · 2019-06-15T11:39:13Z

Anything we're waiting on here?

grahamc

Code review done, giving it a go

src/cli.rs

src/socket/mod.rs

grahamc · 2019-06-19T09:32:49Z

src/socket/mod.rs

+}
+
+/// Wrap a socket with a timeout. Inspired by https://docs.rs/crate/timeout-readwrite/0.2.0/.
+mod timeout {


I don't love the code in this module, it feels quite complex and I don't really want to "own" it.

It’s sadly the only thing to not break the Darwin socket implementation in non-debuggable ways. Just setting the socket timeout option worked fine on Linux, but not on Darwin. It’s just a small wrapper around the poll(2) syscall, nothing extremely complex or error-prone.

src/main.rs

This way we keep all project-wide constant setup in one place.

Creates a dedicated directory for GC roots, because we need a collision-free space for our socket files in the toplevel directory.

We can watch an arbitrary nix file. The rest of the codebase already uses `nix_file`, so rename for consistency.

The `id` function hashed the full path and the name. The latter was completely determined by the full path, so name is redundant. When we want to allow any nix file, we cannot determine name anyway (without it panicking for some file paths).

lorri cannot have a sensible concept of a project directory. Builders need only a valid nix file to function. We can still expose the convenience that `shell.nix` is automatically discovered and used as the default nix file, even without explicitely tracking the parent directory of that shell file.

Slight refactoring of `main()` is necessary to correctly set up the roots.

Projects are just a trivial wrapper around a nix file path and gc_root directory, so we don’t need to actually clone that info.

After the client opens a connection, the server won’t wait forever for a message to free its resouces.

The daemon gets the files to evaluate over the socket instead of from `shell.nix` on startup.

Co-authored-by: Graham Christensen <[email protected]>

Internal & experimental commands should be differentiable from user-commands.

Co-authored-by: Graham Christensen <[email protected]>

Profpatsch requested a review from grahamc May 14, 2019 16:47

Profpatsch force-pushed the message-over-socket branch from f95f17c to 42aa124 Compare May 15, 2019 16:49

Profpatsch changed the title ~~WIP lorri daemon~~ lorri daemon May 15, 2019

Profpatsch force-pushed the message-over-socket branch from 42aa124 to a54f395 Compare May 15, 2019 16:58

grahamc reviewed May 16, 2019

View reviewed changes

nix/pre-check.sh Outdated Show resolved Hide resolved

Profpatsch force-pushed the message-over-socket branch 4 times, most recently from 2b3a744 to d7187cc Compare May 22, 2019 16:52

Profpatsch force-pushed the message-over-socket branch from 0476708 to dcc55ab Compare May 27, 2019 16:41

Profpatsch requested a review from grahamc May 27, 2019 17:38

grahamc reviewed May 28, 2019

View reviewed changes

Profpatsch force-pushed the message-over-socket branch 2 times, most recently from b94bf20 to 31aae02 Compare May 29, 2019 18:27

Profpatsch force-pushed the message-over-socket branch 5 times, most recently from 69c2e5f to a672eba Compare June 7, 2019 22:26

Kiwi mentioned this pull request Jun 13, 2019

lorri daemon: systemd service #96

Closed

grahamc reviewed Jun 19, 2019

View reviewed changes

src/main.rs Outdated Show resolved Hide resolved

Profpatsch force-pushed the message-over-socket branch from a1d5415 to 07eac0d Compare June 19, 2019 14:51

Profpatsch and others added 25 commits June 20, 2019 18:46

daemon: Display error message when binding to locked socket

461d19e

daemon: factor out static socket string

b346b37

project: Move gc_path constant to new constants module

4fd565e

This way we keep all project-wide constant setup in one place.

Move GC roots from ~/.cache/lorri to ~/.cache/lorri/gc_roots

162fce8

Creates a dedicated directory for GC roots, because we need a collision-free space for our socket files in the toplevel directory.

constants, daemon: Move daemon socket to cache dir

fcd0588

constants/Paths: create directories if they don’t exist

778305d

ops/daemon: move Daemon struct into daemon module

a3e2052

project: rename shell_nix to nix_file

52034e6

We can watch an arbitrary nix file. The rest of the codebase already uses `nix_file`, so rename for consistency.

Move ops/info and ops/init to plain rust files

4ca9c4a

roots: Remove new() because unused

070cb99

Add NixFile newtype to distinguish from other PathBufs

e90deb8

project/Project.from_cwd: move gc_root out to call site

31c5a06

Slight refactoring of `main()` is necessary to correctly set up the roots.

project/Project: make a shallow wrapper

ecea7b4

Projects are just a trivial wrapper around a nix file path and gc_root directory, so we don’t need to actually clone that info.

daemon: Set default message socket read timeout of 1000ms

614793f

After the client opens a connection, the server won’t wait forever for a message to free its resouces.

default.nix: fix cargoSha256

395a4a9

socket/communicate: introduce a DEFAULT_READ_TIMEOUT

02f9fef

main.rs: daemon subcommand doesn’t require a shell.nix in cwd

28be34a

The daemon gets the files to evaluate over the socket instead of from `shell.nix` on startup.

cli.rs: add documentation to Daemon and Ping

17e303d

Co-authored-by: Graham Christensen <[email protected]>

cli.rs: postfix Ping with _ to indicate it’s plumbing

12a0551

Internal & experimental commands should be differentiable from user-commands.

socket/communicate: more declarative names for PhantomData fields

ee70572

constants/Paths: move daemon socket to the runtime dir if possible

9150ed3

daemon/StartBuild: rename to IndicateActivity

5dc0a56

ops/direnv: improve error message for missing shell.nix

40dc0d3

Co-authored-by: Graham Christensen <[email protected]>

Profpatsch force-pushed the message-over-socket branch from 1fdd65d to 40dc0d3 Compare June 20, 2019 17:10

Profpatsch merged commit 2b9c420 into master Jun 20, 2019

grahamc deleted the message-over-socket branch June 21, 2019 09:16

Profpatsch mentioned this pull request Jun 25, 2019

Implement a socket API for the background evaluator #32

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`lorri daemon` #81

`lorri daemon` #81

Profpatsch commented May 14, 2019

Profpatsch commented May 15, 2019

Profpatsch commented May 27, 2019

Profpatsch commented May 28, 2019

grahamc left a comment

Profpatsch commented May 28, 2019

grahamc commented May 28, 2019

shlevy commented Jun 15, 2019

grahamc left a comment

grahamc Jun 19, 2019

Profpatsch Jun 19, 2019 •

edited

Loading

lorri daemon #81

lorri daemon #81

Conversation

Profpatsch commented May 14, 2019

Profpatsch commented May 15, 2019

Profpatsch commented May 27, 2019

Profpatsch commented May 28, 2019

grahamc left a comment

Choose a reason for hiding this comment

Profpatsch commented May 28, 2019

grahamc commented May 28, 2019

shlevy commented Jun 15, 2019

grahamc left a comment

Choose a reason for hiding this comment

grahamc Jun 19, 2019

Choose a reason for hiding this comment

Profpatsch Jun 19, 2019 • edited Loading

Choose a reason for hiding this comment

`lorri daemon` #81

`lorri daemon` #81

Profpatsch Jun 19, 2019 •

edited

Loading