--experimental_repository_cache should allow a relative path #3516

kchodorow · 2017-08-07T14:53:30Z

Or at least not print a stack trace:

java.lang.IllegalArgumentException: my_cache (not an absolute path)
	at com.google.devtools.build.lib.vfs.FileSystem.getPath(FileSystem.java:146)
	at com.google.devtools.build.lib.bazel.BazelRepositoryModule.handleOptions(BazelRepositoryModule.java:189)
	at com.google.devtools.build.lib.runtime.BlazeCommandDispatcher.execExclusively(BlazeCommandDispatcher.java:511)
	at com.google.devtools.build.lib.runtime.BlazeCommandDispatcher.exec(BlazeCommandDispatcher.java:314)
	at com.google.devtools.build.lib.runtime.CommandExecutor.exec(CommandExecutor.java:53)
	at com.google.devtools.build.lib.server.GrpcServerImpl.executeCommand(GrpcServerImpl.java:849)
	at com.google.devtools.build.lib.server.GrpcServerImpl.access$2100(GrpcServerImpl.java:104)
	at com.google.devtools.build.lib.server.GrpcServerImpl$4$1.run(GrpcServerImpl.java:915)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
java.lang.IllegalArgumentException: my_cache (not an absolute path)
	at com.google.devtools.build.lib.vfs.FileSystem.getPath(FileSystem.java:146)
	at com.google.devtools.build.lib.bazel.BazelRepositoryModule.handleOptions(BazelRepositoryModule.java:189)
	at com.google.devtools.build.lib.runtime.BlazeCommandDispatcher.execExclusively(BlazeCommandDispatcher.java:511)
	at com.google.devtools.build.lib.runtime.BlazeCommandDispatcher.exec(BlazeCommandDispatcher.java:314)
	at com.google.devtools.build.lib.runtime.CommandExecutor.exec(CommandExecutor.java:53)
	at com.google.devtools.build.lib.server.GrpcServerImpl.executeCommand(GrpcServerImpl.java:849)
	at com.google.devtools.build.lib.server.GrpcServerImpl.access$2100(GrpcServerImpl.java:104)
	at com.google.devtools.build.lib.server.GrpcServerImpl$4$1.run(GrpcServerImpl.java:915)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

The text was updated successfully, but these errors were encountered:

kchodorow · 2017-08-07T14:54:13Z

See https://stackoverflow.com/questions/45425528/saving-external-dependencies-to-projects-repository for a use case.

davido · 2017-08-08T07:33:28Z

We have a feature request to allow user specific directory: ~/.gerritcodereview/download-cache.

ittaiz · 2017-08-08T11:46:41Z

+1

…

On Tue, Aug 8, 2017 at 10:33 AM David Ostrovsky ***@***.***> wrote: We have a feature request to allow user specific directory: ~/.gerritcodereview/download-cache. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#3516 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABUIF3ByOVsAbfkJKJko1Jq7fhnqc5wOks5sWA9LgaJpZM4Ovi2a> .

davido · 2017-08-08T12:15:35Z

#2054

nevion · 2017-10-14T13:04:32Z

+1

ensonic · 2017-11-02T13:49:55Z

It only it would resolve env-vars like $HOME or $PWD as a workaround.

dslomov · 2018-01-10T14:21:31Z

crash is P1, feature request seems to be not that hard to implement either

ittaiz · 2018-01-10T16:12:04Z

@dslomov any chance this will allow ~ or $HOME?
@davido did something like this in a different CL he got in a few months ago but I think it's only relevant for his other experimental flag (I think related to local remote cache)

davido · 2018-01-10T18:15:29Z

@ittaiz yeah and it was undone in later patch sets of that CL: [1].

[1] https://bazel-review.googlesource.com/#/c/bazel/+/16810/7/src/main/java/com/google/devtools/build/lib/remote/SimpleBlobStoreFactory.java@86

ittaiz · 2018-01-11T05:54:01Z

Why was it undone?

…

On Wed, 10 Jan 2018 at 20:15 David Ostrovsky ***@***.***> wrote: @ittaiz <https://github.com/ittaiz> , yeah and it was undone in later patch sets of that CL: [1]. - [1] ***@***.*** — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#3516 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABUIF2LJ4X7XmVopbQhd4Zu4aa-XPfcRks5tJP5EgaJpZM4Ovi2a> .

davido · 2018-01-11T07:06:02Z

Because reviewers pointed out, that support for HOME-directory aware paths should be generally integrated in Bazel infrastructure and not hard coded in this specific parameter.

ittaiz · 2018-01-11T07:07:38Z

I see. Unlikely you'll pick that one up, right?

…

On Thu, 11 Jan 2018 at 9:06 David Ostrovsky ***@***.***> wrote: Because reviewers pointed out, that support for HOME-directory should be generally integrated in Bazel infrastructure. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#3516 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABUIF-9pcZGn-AmQ2HCQMr3G8XspOn1-ks5tJbLdgaJpZM4Ovi2a> .

davido · 2018-01-11T07:14:07Z

Unfortunately, I cannot work on that atm.

...and interpret them relative to the workspace directory. Improves on #3516. In fact, fixes the original request of supporting relative paths. Change-Id: Ibbb6fd43179d589ad477427e47e26c773c7a04de PiperOrigin-RevId: 185121629

aehlig · 2018-02-12T15:27:16Z

The actual crash is fixed in 157caed and relative paths are interpreted relative to the WORKSPACE directory. So the intended use case 'bazel build --experimental_repository_cache=my_cache' works. Alternatively, 8837461 and 220d650 add an option --experimental_distdir allowing to specify a list of read-only directories containing flatly the archives instead of a cache directory in bazel's internal repository cache format. Again, relative paths work (interpreted relative to the WORKSPACE, e.g., 'bazel build --experimental_distdir=distfiles'), as do absolute path ('bazel build --experimental_distdir=/usr/ports/distfiles'). So, the only remaining part of this issue is what is already tracked in #2054. Therefore, unless someone objects, I will close this issue leaving only feature request #2054 open. (For the latter, I'm not sure if it is really necessary. Options passed on the command line are already expanded by the shell, and options referring to the user's home directory are best set in the user's .bazelrc file (which is individual per user); documentation can still suggest commands like 'echo $HOME/.gerritcodereview/download-cache >> $HOME/.bazelrc' which again will be expanded correctly by the shell).

…

-- Klaus Aehlig Google Germany GmbH, Erika-Mann-Str. 33, 80636 Muenchen Registergericht und -nummer: Hamburg, HRB 86891 Sitz der Gesellschaft: Hamburg Geschaeftsfuehrer: Paul Terence Manicle, Halimah DeLaine Prado

ittaiz · 2018-02-12T17:13:02Z

How does this play with a bazelrc file which is checked in to the project? Unfortunately bazelrc files aren’t hierarchical so one has to use one or the other and we’re using the checked in version to not have tribal knowledge of what goes in

…

On Mon, 12 Feb 2018 at 17:27 Klaus Aehlig ***@***.***> wrote: The actual crash is fixed in 157caed and relative paths are interpreted relative to the WORKSPACE directory. So the intended use case 'bazel build --experimental_repository_cache=my_cache' works. Alternatively, 8837461 and 220d650 add an option --experimental_distdir allowing to specify a list of read-only directories containing flatly the archives instead of a cache directory in bazel's internal repository cache format. Again, relative paths work (interpreted relative to the WORKSPACE, e.g., 'bazel build --experimental_distdir=distfiles'), as do absolute path ('bazel build --experimental_distdir=/usr/ports/distfiles'). So, the only remaining part of this issue is what is already tracked in #2054. Therefore, unless someone objects, I will close this issue leaving only feature request #2054 open. (For the latter, I'm not sure if it is really necessary. Options passed on the command line are already expanded by the shell, and options referring to the user's home directory are best set in the user's .bazelrc file (which is individual per user); documentation can still suggest commands like 'echo $HOME/.gerritcodereview/download-cache >> $HOME/.bazelrc' which again will be expanded correctly by the shell). -- Klaus Aehlig Google Germany GmbH, Erika-Mann-Str. 33, 80636 Muenchen <https://maps.google.com/?q=Erika-Mann-Str.+33,+80636+Muenchen&entry=gmail&source=g> Registergericht und -nummer: Hamburg, HRB 86891 Sitz der Gesellschaft: Hamburg Geschaeftsfuehrer: Paul Terence Manicle, Halimah DeLaine Prado — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#3516 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABUIF6ptvj-7jq6ykxeHOJuknmNxXHcnks5tUFhYgaJpZM4Ovi2a> .

davido · 2018-02-12T19:53:38Z

@aehlig Yeah, there is also #3042, where a similar feature request, to provide relative class path for --experimental_local_disk_cache_path option is tracked, see my last comment.

aehlig · 2018-02-12T22:33:08Z

Unfortunately bazelrc files aren’t hierarchical so one has to use one or the other

Indeed, this is yet another independent issue. Filed #4629 to keep track of it.

aehlig · 2018-02-12T22:33:48Z

The actual issue is solved, and all other feature requests that came up in the discussion are tracked in separate issues. Closing.

ensonic · 2018-02-13T14:23:36Z

Which change fixes it? According to the comments in #3042 is is not fixed.

davido · 2018-02-13T16:53:44Z

@ensonic It's not fixed yet, and this feature request is tracked in #2054. Once fixed, it should work for --experimental_repository_cache, --experimental_local_disk_cache_path and others.

hlopko added category: extensibility > external repositories P3 We're not considering working on this, but happy to review a PR. (No assignee) type: feature request labels Aug 9, 2017

dslomov added P1 I'll work on this now. (Assignee required) and removed P3 We're not considering working on this, but happy to review a PR. (No assignee) labels Jan 10, 2018

dslomov added the external-repos-triaged label Jan 10, 2018

aehlig mentioned this issue Feb 12, 2018

support reading multiple rc files #4629

Closed

aehlig closed this as completed Feb 12, 2018

aehlig mentioned this issue Feb 13, 2018

Change where bazelrcs are read from, and make it easier to control which ones are read. #4502

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

--experimental_repository_cache should allow a relative path #3516

--experimental_repository_cache should allow a relative path #3516

kchodorow commented Aug 7, 2017

kchodorow commented Aug 7, 2017

davido commented Aug 8, 2017

ittaiz commented Aug 8, 2017 via email

davido commented Aug 8, 2017

nevion commented Oct 14, 2017

ensonic commented Nov 2, 2017

dslomov commented Jan 10, 2018

ittaiz commented Jan 10, 2018

davido commented Jan 10, 2018 •

edited

Loading

ittaiz commented Jan 11, 2018 via email

davido commented Jan 11, 2018 •

edited

Loading

ittaiz commented Jan 11, 2018 via email

davido commented Jan 11, 2018

aehlig commented Feb 12, 2018 via email

ittaiz commented Feb 12, 2018 via email

davido commented Feb 12, 2018 •

edited

Loading

aehlig commented Feb 12, 2018

aehlig commented Feb 12, 2018

ensonic commented Feb 13, 2018

davido commented Feb 13, 2018 •

edited

Loading

--experimental_repository_cache should allow a relative path #3516

--experimental_repository_cache should allow a relative path #3516

Comments

kchodorow commented Aug 7, 2017

kchodorow commented Aug 7, 2017

davido commented Aug 8, 2017

ittaiz commented Aug 8, 2017 via email

davido commented Aug 8, 2017

nevion commented Oct 14, 2017

ensonic commented Nov 2, 2017

dslomov commented Jan 10, 2018

ittaiz commented Jan 10, 2018

davido commented Jan 10, 2018 • edited Loading

ittaiz commented Jan 11, 2018 via email

davido commented Jan 11, 2018 • edited Loading

ittaiz commented Jan 11, 2018 via email

davido commented Jan 11, 2018

aehlig commented Feb 12, 2018 via email

ittaiz commented Feb 12, 2018 via email

davido commented Feb 12, 2018 • edited Loading

aehlig commented Feb 12, 2018

aehlig commented Feb 12, 2018

ensonic commented Feb 13, 2018

davido commented Feb 13, 2018 • edited Loading

davido commented Jan 10, 2018 •

edited

Loading

davido commented Jan 11, 2018 •

edited

Loading

davido commented Feb 12, 2018 •

edited

Loading

davido commented Feb 13, 2018 •

edited

Loading