Provide a standard way to find the default julia install

[davidanthoff]

There are now many pieces of software that are started directly by the user and that then need to start a julia.exe internally. IJulia, the various editor plugins (e.g. Sublime-IJulia and Jewel) are good examples of that. Right now each of these somewhere saves the hard coded full path to an ijulia.exe for that purpose. For some of these (the editor plugins) you actually have to manually edit a config file to tell it where the julia.exe is located, for others the path is picked up during setup (IJulia) and then stored.

I think a better model would be to provide some central registration for the "active" julia installation, e.g. something like a JULIAHOME environmental variable that points to the currently active julia.exe, so that not every one of these pieces of software has to store the path to julia. The JULIAHOME variable could be set during setup of julia.

That would allow a couple of neat things: one could have multiple julia versions installed in parallel, with one being "active". If the active one is changed, all the ide plugins and ijulia would pick that one up. Also, right now if one installs a new version of julia that puts things into a different directory than a previous install (e.g. for a while the directory was julia-0.3.0-prelease or something, not it is julia-0.3.0-rc1) these other pieces of software would still work because they could automatically pick up the new install location of julia.

This would also help with something like the chocolatey package that I'm building (#7386) that will probably end up installing julia prerelease versions into directories that contain the git commit sha, i.e. where the install directory name will change for every new build.

[davidanthoff]

Pinging @stevengj, @tkelman, @ihnorton, @quinnj and @one-more-minute to get your opinion whether that would actually be a useful thing from your point of view or not.

[quinnj]

Seems like it would be generally useful, though does it complicate things if we start messing with environment variables? (i.e. require additional permissions, etc.?)

[nalimilan]

FWIW R supports installing several versions in parallel, and it appears to save the "active" version in the Windows registry. Doesn't sound like a bad solution (working with the registry is OK as long as you don't use it for complex tasks), better than environment variables at least since the registry is exactly supposed to be used for this kind of things.

[stevengj]

@davidanthoff, setting an environment variable wouldn't work, because an environment variable is only visible to child processes.

Of course, there already is a standard way to make a certain version of an executable globally visible: put it into your PATH.

[tkelman]

And adding things to PATH gets horribly abused on Windows, causing no end of mysterious problems from dll shadowing, performance problems and bugs in git and other things from PATH being really long, and so on. I don't like touching PATH, or the registry, or making a new special environment variable just for Julia on principle, but also can't think of a better solution to the problem. I prefer using explicit absolute paths wherever possible, but this can get annoying if it has to be constantly reconfigured.

We do have a fairly standardized location where we're putting packages. What if, on a per-user basis, we have a ~/.julia/.active_julia file that contains the absolute path to the currently active Julia installation? Maybe under the versioned package dir instead so it can be independent for each minor version of Julia.

[MikeInnes]

I'd really like to see something like this, if only on Windows; it's pretty standard practice to set up user environment variables like JAVA_HOME, PYTHON_HOME etc., so that seems like a reasonable solution to me. I'm also not completely against modifying the path. Given that not everyone might want this it could perhaps be an install-time option.

Another option would be having a Julia function to do either of those things – set_default_julia() or add_julia_to_path(). Even that would be a big step up from having to edit config files, which is unfortunately fiddly and complicated.

[ihnorton]

Another option would be having a Julia function to do either of those
things – set_julia_home() or add_julia_to_path().

If Julia is running it should already know what the home directory and path
are...

On Tue, Aug 5, 2014 at 2:21 PM, Mike Innes [email protected] wrote:

I'd really like to see something like this, if only on Windows; it's
pretty standard practice to set up user environment variables like
JAVA_HOME, PYTHON_HOME etc., so that seems like a reasonable solution to
me. I'm also not completely against modifying the path. Given that not
everyone might want this it could perhaps be an install-time option.

Another option would be having a Julia function to do either of those
things – set_julia_home() or add_julia_to_path(). Even that would be a
big step up from having to edit config files, which is unfortunately fiddly
and complicated.

—
Reply to this email directly or view it on GitHub
#7835 (comment).

[tkelman]

Haskell Platform adds itself to the path on Windows. And it gets yelled at by every single other project that uses MinGW in any way shape or form by introducing difficult-to-debug incompatibilities on users' machines. We have so many dependency dll's, adding Julia's bin directory to the Windows PATH would be an enormous act of poor open-source citizenship.

It's not really feasible to make any permanent changes to environment variables from inside Julia. Is coordinating, centralizing, and simplifying the configuration files a viable solution here?

[quinnj]

Totally agree with @tkelman on modifying PATH, some of my most frustrating moments have dealt with programs that modified path.

I think setting a JULIAHOME environment variable would be reasonable, or some kind of config file might work well also.

[ihnorton]

@tkelman - I agree about PATH, but I don't have a problem with adding a
JULIA_HOME environment variable.

The other option is to put something in the registry, but that makes life
more difficult for everyone.

On Tue, Aug 5, 2014 at 2:35 PM, Tony Kelman [email protected]
wrote:

Haskell Platform adds itself to the path on Windows. And it gets yelled at
by every single other project that uses MinGW in any way shape or form by
introducing difficult-to-debug incompatibilities on users' machines. We
have so many dependency dll's, adding Julia's bin directory to the Windows
PATH would be an enormous act of poor open-source citizenship.

It's not really feasible to make any permanent changes to environment
variables from inside Julia.

—
Reply to this email directly or view it on GitHub
#7835 (comment).

[tkelman]

Adding a dedicated environment variable for this could probably work, but making permanent manual changes to environment variables is not very user-friendly on Windows, I can never remember which set of menus it's buried under in control panel. Evidently setx is how to permanently set env vars from the command line, and I guess that could be made to work inside Julia too, but I had to google it - I've never used that command before.

[MikeInnes]

If Julia is running it should already know what the home directory and path
are...

@ihnorton Sure, but just because Julia is running doesn't mean that it's accessible to other programs. Making Julia run once is easy enough (just download and click the executable), but you don't want to navigate to the download folder / git repository every time, you want to make it accessible on your path or to your editor. Adding a utility for this seems pretty reasonable to me (especially on Windows, where adding to the path is not exactly user-friendly).

adding Julia's bin directory to the Windows PATH would be an enormous act of poor open-source citizenship.

@tkelman I think it's possible to add just the Julia executable to the path without all of its dependencies, which shouldn't cause any name-clashing issues.

[tkelman]

I think it's possible to add just the Julia executable to the path without all of its dependencies, which shouldn't cause any name-clashing issues.

Sort of. julia.exe has to live in the same directory as libjulia.dll, and libjulia.dll depends on MinGW libgcc and libstdc++ dll's that we do not want to put on the path. There used to be a julia.bat that we could resurrect, and follow a Matlab-like approach where we put a directory with only bat files on the path, with all the dll's placed somewhere else. Still, messing around with PATH or other environment variables to change this around between different versions is fairly tedious. And Julia itself (due to a limitation in libuv) is incapable of spawning bat files, other applications may have similar problems with them. We could go down the road of writing small dependency-free launcher exe's whose job is to be on the path and start up the real julia exe that isn't, but this is extra code to write and maintain. I don't think it's worth it - what's wrong with tracking this small piece of state through an agreed-upon file in a standard location?

Chocolatey would do some of this for us, it adds one specific folder to the path and then everything you install through that package manager puts bat file shortcuts in that one place. But it's still rather new, not that many people use it, and for a lot of packages it doesn't do anything more complicated than running an installer in silent mode to a specific location. Chocolatey support for Julia is still a WIP PR, and I'm not sure it would ever get 100% adoption since people are so used to conventional installers on Windows.

[MikeInnes]

Yeah, I thought it might be possible to add julia.exe directly to the path, but it looks like I was mistaken, in which case it's not worth the effort. As far as this issue is concerned a config file of some sort seems like a pretty good solution.

Being able to add Julia to the path conveniently still seems valuable in itself, though, so I'll continue looking into this.

[davidanthoff]

Bump, it would be really great if we could find some solution for this before 1.0.

This is the kind of usability thing that really makes life difficult for inexperienced users. I'm seeing these problems again with the VS Code extension for julia: one now has to configure a setting with the path to the julia binary at first start, and then whenever one installs a new version of julia one has to update that setting in VS Code, because the Windows installer uses a different directory name for different julia versions. Not a problem for me, but the amount of friction this is causing for new users is quite amazing (I'm bringing a lot of new folks to julia lately that don't have a strong programming background).

I think my favorite solution for now would be what @tkelman suggested above:

What if, on a per-user basis, we have a ~/.julia/.active_julia file that contains the absolute path to the currently active Julia installation?

I think that would do the trick, would be cross-platform essentially just solve this problem.

Tony also suggested this

Maybe under the versioned package dir instead so it can be independent for each minor version of Julia.

I think I would prefer just one active julia. At the end of the day, I need to pick one julia version in something like the VS Code extension, so having different active julias for different minor versions just pushes the problem some level down.

[StefanKarpinski]

I think we'd want this to be an environment variable, a la virtual-env. The current active Julia should be a dynamically scoped property, not a global one.

[tkelman]

I don't think Julia installation should be making permanent environment variable modifications. Is that even doable with a dmg or tarball?

[StefanKarpinski]

I meant in a shell session. Perhaps we need a global "default starting Julia" and environment variable for overriding that with a dynamic scope across processes. However, that seems a bit redundant since the user can always put it in their RC files and something like Juno can set it in its own environment.

[tkelman]

any shell centric setting would be inappropriate for windows, or editor applications that are launched from somewhere other than a shell

[StefanKarpinski]

Ok, so propose something that satisfies your requirements and also allows dynamic scoping.

[nalimilan]

Environment variables are great to customize/override settings, but we also need a permanent one. On Windows, I guess the registry would be the standard way of doing that. On Unix, PATH should be enough (it normally includes ~/.local/bin so that one can override /usr/bin with custom executables/symlinks if needed).

[davidanthoff]

For VS Code, we would still have a setting where users could pick some other julia binary than the global active one, so at least for our purposes just one global active julia binary that we can easily find would be good enough.

A more complete solution would probably have some sort of central information about all julia versions that are currently installed, plus an indication which of those is the default one.

What is the use case for dynamic scoping?

[davidanthoff]

This shows the current (future?) Python approach to registering all installed Python versions. Not clear to me (from a hasty read) how they go about the default or active version...

[StefanKarpinski]

What is the use case for dynamic scoping?

Different processes (shells, IDEs) should be able to set their preferred Julia and have child processes respect that. That's why environment variables are good for this use case.

[tkelman]

I don't see this needing to be changed all that frequently, or applications wanting to modify this in a scoped way. How many applications that need this kind of configuration are going to spawn other distinct applications that also need this, while wanting to make process-local modifications to this information?

There's already a perfectly good mechanism for controlling aliases and shortcuts within shells. For non-shell applications, I think a filesystem config file is the most portable and easy to modify option. Users might need to modify this configuration once in a while, but otherwise the Julia installer is the only thing that would really need to ever touch this.

[davidanthoff]

I agree with @tkelman, I don't really see the need for dynamic scoping, at least not for the use case of an IDE/editor plugin.

Thinking through this for the VS Code extension situation, I think the only case where this might come up is if someone wants to start say another julia instance from the julia REPL inside VS Code. But Base.julia-cmd() should be good enough for that, I think.

[digital-carver]

On the Windows side of things:

does it complicate things if we start messing with environment variables? (i.e. require additional permissions, etc.?)

Setting a user-level (permanent) environment variable doesn't require any special permissions (afaict). So the installer can set JULIA_HOME (or ACTIVE_JULIA) as a user-level environment variable by default, and have an option in the UI for the user to set it at the system level (with UAC approval from the user) if desired.

better than environment variables at least since the registry is exactly supposed to be used for this kind of things.

I don't have a problem with adding a JULIA_HOME environment variable. The other option is to put something in the registry, but that makes life more difficult for everyone.

Setting permanent environment variables is editing the registry on Windows, though doing it via environment variables is probably the better, simpler option (compared to directly adding registry keys).

I'd really like to see something like this, if only on Windows; it's pretty standard practice to set up user environment variables like JAVA_HOME, PYTHON_HOME etc., so that seems like a reasonable solution to me.

+1

making permanent manual changes to environment variables is not very user-friendly on Windows, I can never remember which set of menus it's buried under in control panel

This is subjective, but I think handling environment variables is pretty standard stuff for most regular Windows-based developers (right click My Computer->Properties->Advanced->Environment variables). It's definitely more intuitive to a Windows user than editing a dotfile like .active_julia. (And there are tools like Rapid Environment Editor.)

I think adding an environment variable is probably the way to go here (at least on Windows). Adding to PATH can be another option on the installer that the user can choose to enable (defaulting to off), independent of this feature.

[davidanthoff]

The way we handle this in the VS Code extension now is that we just look in some standard locations on Windows and Mac. It mostly works for auto detecting a julia installation, but something that is more official would be really great.

[ViralBShah]

In general, given that we do not modify the user's environment or system files - I don't know if there is anything more official that can be done. I think you're doing the right thing by looking in the standard locations. Please reopen if you don't agree with me closing this.