Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Update installation page #595
Update installation page #595
Changes from 19 commits
6d0a05b
fcffff3
cd5f1a1
a9db33d
cdef452
8b6532e
994fc32
c040bb3
a4eda1e
764a99e
ca78fea
93a937e
d520d0a
9c066ec
51a4b69
2ff3679
359cf4e
0013531
2467c06
6280bbb
c068a96
22195b5
16b80c5
29d7ed3
7ab75e7
73a5f49
File filter
Filter by extension
Conversations
Jump to
There are no files selected for viewing
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do they need any warnings here about losing their work if they clear their cache etc?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
not sure - suggestions welcome!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
More actively maintained than Anaconda?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it looks like that was originally in relation to Python(x, y) - see #259. I guess we just remove that part now.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@CAM-Gerlach thought I'd ping in case you have any opinions on the Python Distributions section here! Hasn't been updated in a while...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@jarrodmillman, I believe the linter is suggesting changes which would break the formatting.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You could try something like
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Of course, you can use
uv
to do this as well and then switch back to being a simplevenv
dinosaur like me. But there are indeed already some comments about choice paralysis I think.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, I considered leaving out the hyperlink and letting users google "how to install Python" - could do that if people prefer. I don't think we should be laying out different options for installing Python.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It looks like the uv folks are urging people to switch away from activating virtual environments. And they have good technical reasons for it.
That's all fine if you've decided to switch everything to UV, but it feels problematic from a higher level UI perspective.
python myscript.py
now becomessome-tool-name run myscript.py
, so it (a) introduces a layer on top of Python (not ideal for newcomers) and (b) binds you into a specific, non-general invocation pattern.As I mentioned before, I suspect this pattern might break a lot of existing tutorial advice, scripts with
#!/usr/bin/env python
, etc. In fact, that's a question I'm very curious about: how doesuv
intend for developers to use it if they requireuv run
for everything? (I'm speaking from complete ignorance here, of course, since I haven't studied uv at all.)I also don't like that users now need to know about
uv
andpip
anduv pip
. It's often possible to plot some narrow trajectory with these tools where you hide much of the complexity (and we should have a community guide from that perspective, perhaps via PyOpenSci, if they don't already). But then things break and you have a lot of concepts to understand before you can debug, and only uv-specific resources are helpful.Anyway, these are just my high-level concerns; I'm sure they can all be addressed in one way for another, and I appreciate you having patience with me as I iterate them.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Adding to what Stefan said --- and I also never used uv --- how does the workflow with a test script in a separate repository work with auto-activation?
Same question about working on a project (scipy) and its dependency (scipy-doctest):
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you clarify the situation and expected behaviour here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The one word answer is "workspaces" (implemented in
uv
, see https://docs.astral.sh/uv/concepts/projects/workspaces/, work in progress inpixi
). Inpixi
right now you can do what you must also do with pip/conda: you manually install one then develop the other, without tracking the dependency between these packages. There is no good way to do it today outside of workspaces (and no, editable installs don't count since those are inherently incomplete).There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Uhm, it can't be that uncommon to have to fix a dependency? The behavior is
That it's an editable install is not very relevant (can be any other way to install the dependency in this specific environment), the choice of the env manager is irrelevant (can be mamba or venv or virtualenvwrapper or ...), the pair of projects can be anything. The main point is the ability to have two unrelated project / git trees in the same environment.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think python has to become uv run everywhere.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You actually can't with editable installs of
numpy
and sayscipy
orscikit-learn
, it won't build the package that depends onnumpy
.That is correct. This is about the same with both methods (ignoring workspaces). In the
pip
/conda
case you have to docd pkg1 && pip install .
thencd pkg2 && pip install . # or similar
, manually keeping track of the two packages both having a dev version installed, then running your tests. If you iteratively work onpkg1
andpkg2
, you just have to manage the state in your head. You can do that withpixi
oruv
as well - it's maybe very slightly more clumsy.In the grand scheme of things, it is uncommon (>95% of users only consume rather than develop packages to start with). It's harder for project-based tools, hence I think it wasn't their first priority. But the workspaces concept was basically copied from Cargo, and seems like a good answer conceptually.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since these methods are so problematic, we could consider only keeping the first sentence (warning) and not giving individual instructions for Debian, Fedora, and macos here.
Optionally, we could then show the route I mentioned elsewhere, which works perfectly well: install Python via system and then just create a virtual env using
python -m venv
. Since this page is no longer the beginner guide, I think that should be acceptable?The suggestion is not needed for this PR, but if it seems like a sensible suggestion I'm happy to make a follow-up PR in the future.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would be happy with that, let's see what others think.
How does that differ from https://deploy-preview-595--scipy-org.netlify.app/install/#installing-with-pip? Just installing Python with a system package manager rather than python.org? I said in another thread that I would be happy to remove the hyperlink to python.org, but I thought that listing out options for installing Python might be overkill.
👍
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Given the default recommendation to use
uv
for projects, I was wondering whether, in the future, we should adjust the above page accordingly?I presume it can be used the same way pip is for SciPy development, adding a third code tab with
uv run python dev.py
anduv pip
for installing dependencies?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yep! Although for SciPy, given the compiler dependencies, Pixi will probably be our best bet. I already use https://github.com/rgommers/pixi-dev-scipystack for all of my SciPy development. Windows support for PyTorch and JAX on conda-forge is still maturing (though actively - conda-forge/pytorch-cpu-feedstock#231), but we may be able to use PyPI on Windows to fill some gaps if anyone needs it.
A third tab for
uv
is probably not so useful, since it's relatively obvious what to do by following along with thepip
instructions, and you have to do the same setup of system deps.All that to say that we could do something like this. Or we could make a new page instead of adding a new tab, since there is much less to explain when most of the work is already done in writing the Pixi manifest. We may want to wait until somebody wants to try to merge Pixi support into the main repo, though.
Also I plan to add a note about
scipy-stubs
to the building from source page after this is merged.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks, Lucas!