Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve the ParseSink interface #9653

Merged
merged 2 commits into from
Jan 23, 2024

Conversation

Ericson2314
Copy link
Member

Motivation

More invariants are enforced in the type, and less state needs to be stored in the main sink itself. The method here is roughly that known as "session types".

Context

The separation of concerns I would like to see in #9485 depends on this step.

Priorities and Process

Add 👍 to pull requests you find important.

The Nix maintainer team uses a GitHub project board to schedule and track reviews.


if (mode == Mode::Executable)
sink.isExecutable();
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was a bad hack that no longer works. The changes to this file are to make a better solution instead.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file will be easier to review with whitespace disabled

@@ -171,93 +171,105 @@ static void parse(ParseSink & sink, Source & source, const Path & path)
s = readString(source);
if (s != "(") throw badArchive("expected open tag");

enum { tpUnknown, tpRegular, tpDirectory, tpSymlink } type = tpUnknown;
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The change I did to the parser is basically to replace this extra state variable with control flow. This is needed so that the "content" and "executable" parsing happens within the callback. I then changed the others to keep the code design consistent (and get rid of this enum entirely, as opposed to confusingly just getting rid of tpRegular).

@@ -9,27 +9,38 @@
namespace nix {

/**
* \todo Fix this API, it sucks.
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was originally your comment, @edolstra, but I removed it as I am now decently happy with this interface :)

Comment on lines +237 to +245
if (archiveSettings.useCaseHack) {
auto i = names.find(name);
if (i != names.end()) {
debug("case collision between '%1%' and '%2%'", i->first, name);
name += caseHackSuffix;
name += std::to_string(++i->second);
} else
names[name] = 0;
}
Copy link
Member

@roberth roberth Jan 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thought: This could be factored out into a sink to sink transformation that performs the renaming.

if (name <= prevName)
throw Error("NAR directory is not sorted");
prevName = name;
if (archiveSettings.useCaseHack) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(pre-existing)
This is only necessary when writing to a store that needs it.
It must not be done in other situations, e.g. when writing NAR contents to Git trees.

src/libutil/fs-sink.hh Show resolved Hide resolved
src/libutil/fs-sink.hh Outdated Show resolved Hide resolved
src/libutil/fs-sink.hh Outdated Show resolved Hide resolved
virtual void createDirectory(const Path & path) = 0;

virtual void createRegularFile(const Path & path) = 0;
virtual void receiveContents(std::string_view data) = 0;
virtual void isExecutable() = 0;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should isExecutable be called before, during or after the sink operations?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think any order is fine, actually.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if a NAR has multiple "executable" strings per file? That'd not be canonical. It doesn't quite smell right, but I don't have a suggestion for this PR.

Ericson2314 and others added 2 commits January 22, 2024 18:01
More invariants are enforced in the type, and less state needs to be
stored in the main sink itself. The method here is roughly that known as
"session types".

Co-authored-by: Robert Hensing <[email protected]>
@github-actions github-actions bot added the store Issues and pull requests concerning the Nix store label Jan 22, 2024
@roberth roberth enabled auto-merge January 22, 2024 23:31
@roberth roberth merged commit 08bf284 into NixOS:master Jan 23, 2024
7 checks passed
@Ericson2314 Ericson2314 deleted the improve-parse-sink branch January 23, 2024 03:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
store Issues and pull requests concerning the Nix store
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

3 participants