Importing a package in an ancestor package instance, then evaluating it in a sub folder

[WaywardWizard]

Use Case

Configuration goes with code. I want to put cue files in my directory heirarchy, and fill them with configuration relevant to the directory in which they are found.

Later, i want to do something with that configuration. For example use it inside a Makefile. If i run cue in a directory, for a package, i want to 'see' all of the unified configuration for that package from all ancestor directories up to the module root regardless of wether or not there is a cue file for that package in the current directory.

Perhaps i need to rethink how i am using cue abit.

Problem

I want to be able to evaluate a cue package (a), in some subdirectory of my cue module (./test), where in some ancestor directory (.) instance of that package (a) another package (b) in the same module (broken.io) is imported. Where in the leaf directory (./test) there is no cue file in the package (a) i am trying to evaluate.

I dont want to specify the module and path to some subfolder to explicitly define the package instance i am evaluating. I want cue to know the package instance is for the directory in which it is being run regardless of wether or not there is a cue file belonging to the package of interest in the current directory.

Problem Demonstration

But am getting this error. I have distilled the problem down to the basics. See my example;

mkdir test; cue mod init "broken.io";touch {a,b}.cue test/c.cue;

Yields the following;

drwxr-xr-x 162 ben 20 Dec 12:01 .
.rw-r--r--  56 ben 20 Dec 12:01 ├── a.cue
.rw-r--r--  18 ben 20 Dec 12:00 ├── b.cue
drwxr-xr-x  54 ben 20 Dec 11:58 ├── cue.mod
.rw-r--r--  54 ben 20 Dec 11:58 │   ├── module.cue
drwxr-xr-x   0 ben 20 Dec 11:58 │   ├── pkg
drwxr-xr-x   0 ben 20 Dec 11:58 │   └── usr
drwxr-xr-x  23 ben 20 Dec 12:00 └── test
.rw-r--r--  18 ben 20 Dec 12:00     └── c.cue

With file contents (commands writing content to file omitted);
a.cue

package a

import B "broken.io:b"

a:  "a"
bb: "\(B.b)"

b.cue

package b

b: "b"

c.cue

package c

c: "c"

When i
pushd test; cue export :a

I get

broken.io/test@v0: import failed: no dependency found for package "broken.io":
    ../a.cue:3:8

And try;
popd;cue export :a
Yielding

    "a": "a",
    "bb": "b"
}

As expected.

Thoughts

I think cue is expecting this package to be a dependency of the module rather than an internal package, it searches for the dependency cue.mod rather than the module filetree, cannot find it and fails.

Question

What is going wrong? Why cant i cue export in the subdirectory?

[WaywardWizard]

cue export "broken.io:a" will work... Thats an answer i think but wont close discussion just yet to leave some space for other suggestions

[WaywardWizard]

I think its logical to assume that a package name, not qualified with a module identifier (:a), should refer to the module of the current directory (which, is resolved by stepping up the directory heirarchy until a cue.mod folder is located).

Logical assumptions holding save mental load of having to remember quirks like prefix with the module name and contribute to the mental ergonomics of a language.

For this reason i dont consider the above an answer and i propose converting this discussion to an issue. Will wait to hear others thoughts.

[myitcv]

Thanks for the question, @WaywardWizard. I think you have simply run into a bug here, which I have raised over at #3640.

A couple of thoughts/comments looking at your example:

• Multiple CUE files can belong to the same package. This might already be clear to you, indeed the multiple packages might be a key part of reproducing the issue I raised at cmd/cue: attempt to import package from main module fails #3640.
• CUE packages in a subdirectory unify with packages of the same name in ancestor directories. Your description indicates this is the behaviour your want to rely on, but just making it explicit by an example:

# export in root
exec cue export .
cmp stdout $WORK/stdout.root.golden

# cd test
cd test

# export in test
exec cue export .
cmp stdout $WORK/stdout.test.golden

-- cue.mod/module.cue --
module: "mod.example"
language: {
	version: "v0.12.0"
}
-- x.cue --
package x

root: 5
-- test/x.cue --
package x

test: 4
-- stdout.root.golden --
{
    "root": 5
}
-- stdout.test.golden --
{
    "root": 5,
    "test": 4
}

If i run cue in a directory, for a package, i want to 'see' all of the unified configuration for that package from all ancestor directories up to the module root regardless of wether or not there is a cue file for that package in the current directory.

This should be the behaviour of CUE by default today and you can rely on this. Please raise issues for any bugs you find, like the one you uncovered in #3640.

Please note that in #2000 we might change the default behaviour to not load ancestor directories by default. Whilst it fits the use case that you want here, it isn't always what people want. And the fact that it's not possible to "turn off" that behaviour today is a problem for many.

I dont want to specify the module and path to some subfolder to explicitly define the package instance i am evaluating. I want cue to know the package instance is for the directory in which it is being run regardless of wether or not there is a cue file belonging to the package of interest in the current directory.

This should be possible, as my repro in #3640 demonstrates. The use of .:a says "load the package a from the current directory." The relative directory means of specifying a package path to CUE is documented in https://cuelang.org/docs/reference/command/cue-help-inputs/.

Finally, thank you for taking the time to create such a detailed repro. As you can see from the issue I raised in #3640, repros that involve multiple files can most easily be communicated via a txtar archive. Not essential, but in case you are interested you can read more at https://github.com/cue-lang/cue/wiki/Creating-test-or-performance-reproducers#testing-a-repro-using-cmdtestscript.

I'll leave you to close this discussion, or ask further questions.

[WaywardWizard]

Thanks for the response @myitcv

I do rely on the behaviour of unifying with packages of same name in ancestor directories yes, and the approach i have to using cue has multiple cue files belonging to the same package.

Thankyou for the pointer of txtar, that looks like a better, easier way.

My gratitudes again for all the cuelang people and the work that you do.

I'll be following issue #3640 and close this discussion.