Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update installation page #595

Merged
merged 26 commits into from
Dec 23, 2024
Merged
Show file tree
Hide file tree
Changes from 22 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
75 changes: 75 additions & 0 deletions content/en/beginner-install.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
---
title: Beginner Installation Guide
sidebar: false
---

{{< admonition tip >}}
This is the beginner installation guide.
If you are comfortable with using a terminal and happy to learn
how to use a package manager, check out
[the main installation guide](./install.md)!
{{< /admonition >}}

- [JupyterLite](#jupyterlite)
- [Scientific Python Distributions](#distributions)
- [Installing globally with `pip`](#pip-global)

<a name="jupyterlite"></a>

## JupyterLite

To try out SciPy, you don't even need to install it!
You can use SciPy in your browser at https://jupyter.org/try-jupyter/lab/ -
just write `import scipy` in one of the notebook "cells" and hit play.

For next steps, see [the SciPy user guide][scipy-user-guide].

[scipy-user-guide]: https://docs.scipy.org/doc/scipy/tutorial/

<a name="distributions"></a>

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do they need any warnings here about losing their work if they clear their cache etc?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure - suggestions welcome!

## Scientific Python Distributions

Python distributions provide the language itself, along with the most
commonly used packages and tools. These downloadable files require
little configuration, work on almost all setups, and provide all the
most commonly used scientific Python tools.
[Anaconda](https://www.anaconda.com/download/) works on Windows, Mac,
and Linux, and is best suited to beginning users.
Other options include:

- [WinPython](https://winpython.github.io): Another free distribution
including scientific packages and the Spyder IDE; Windows only, but
more actively maintained and supports the latest Python 3 versions.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

More actively maintained than Anaconda?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it looks like that was originally in relation to Python(x, y) - see #259. I guess we just remove that part now.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@CAM-Gerlach thought I'd ping in case you have any opinions on the Python Distributions section here! Hasn't been updated in a while...

- [Pyzo](https://pyzo.org): A free distribution based on Anaconda
and the IEP interactive development environment; Supports Linux,
Windows, and Mac.

{{< admonition note >}}
Users in large, non-university institutions may want to read Anaconda's
helpful blog on ["when is Anaconda free to use?"](https://www.anaconda.com/blog/update-on-anacondas-terms-of-service-for-academia-and-research)
{{< /admonition >}}

After installing a scientific Python distribution,
see next steps in [the SciPy user guide][scipy-user-guide].

<a name="pip-global"></a>

## Installing globally with `pip`

If you already have Python installed, you can install SciPy globally
with `pip` by executing the following in a terminal/shell:

python -m pip install scipy

{{< admonition warning >}}
You may see this recommended in tutorials or classes, but it is not
recommended to install packages globally on your system.
The recommended way to install SciPy with `pip` is to use a virtual
environment - see [Installing with `pip`](./install.md#installing-with-pip).
{{< /admonition >}}

{{< admonition note >}}
For more information on why this is not a recommended installation method,
read about [virtual environments in the Python Packaging User Guide](https://packaging.python.org/en/latest/tutorials/installing-packages/#creating-virtual-environments).
{{< /admonition >}}
231 changes: 176 additions & 55 deletions content/en/install.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,101 +3,222 @@ title: Installation
sidebar: false
---

Installations methods include:
{{< admonition tip >}}
This page assumes that you are comfortable with using a terminal and happy to learn
how to use a package manager. If you are a beginner and just want to get started
with SciPy as quickly as possible, check out
[the beginner installation guide](./beginner-install.md)!
{{< /admonition >}}

- [Distributions](#distributions)
- [pip](#pip-install)
- [conda](#pip-install)
- [Package Manager](#package_manager)
- [Source](#source)
The recommended method of installing SciPy depends on your preferred workflow.
The common workflows can roughly be broken down into the following
categories:

Methods differ in ease of use, coverage, maintenance of old versions,
system-wide versus local environment use, and control. With pip or
Anaconda\'s conda, you can control the package versions for a specific
project to prevent conflicts. Conda also controls non-Python packages,
like MKL or HDF5. System package managers, like `apt-get`, install
across the entire computer, often have older versions, and don\'t have
as many available versions. Source compilation is much more difficult
but is necessary for debugging and development. If you don\'t know which
installation method you need or prefer, we recommend the Scientific
Python Distribution [Anaconda](https://www.anaconda.com/download/).
- [Project-based (e.g. `uv`, `pixi`)](#project-based) (recommended for new users)
- [Environment-based (e.g. `pip`, `conda`)](#environment-based) (the traditional workflow)
- [System package managers](#system-package-managers) (not recommended)
- [Building from source](#building-from-source) (for debugging and development)

<a name="distributions"></a>
To install SciPy with [static type stubs],
see [Installing with type stubs](#type-stubs).

# Scientific Python Distributions (recommended)
[static type stubs]: https://typing.readthedocs.io/en/latest/guides/libraries.html

Python distributions provide the language itself, along with the most
commonly used packages and tools. These downloadable files require
little configuration, work on almost all setups, and provide all the
commonly used scientific Python tools.
<a name="project-based"></a>

[Anaconda](https://www.anaconda.com/download/) works on Windows, Mac,
and Linux, provides over 1,500 Python packages, and is used by over 15
million people. Anaconda is best suited to beginning users; it provides
a large collection of libraries all in one.
## Project-based workflows

For more advanced users who will need to install or upgrade regularly,
[Mambaforge](https://github.com/conda-forge/miniforge#mambaforge) is a more
suitable way to install the `conda` (and `mamba`, a faster `conda` alternative)
package manager.
### Installing with `uv`

Other options include:
Here is a step-by-step guide to setting up a project to use SciPy, with [`uv`],
a Python package manager.

- [WinPython](https://winpython.github.io): Another free distribution
including scientific packages and the Spyder IDE; Windows only, but
more actively maintained and supports the latest Python 3 versions.
- [Pyzo](https://pyzo.org): A free distribution based on Anaconda
and the IEP interactive development environment; Supports Linux,
Windows, and Mac.
[`uv`]: https://docs.astral.sh/uv/

<a name="pip-install"></a>
1. Install `uv`, following [the instructions in the `uv` documentation][install-uv].

# Installing with Pip
[install-uv]: https://docs.astral.sh/uv/getting-started/installation/

You can install SciPy from PyPI with `pip`:
2. Create a new project in a new subdirectory, by executing the following in a terminal:

python -m pip install scipy
uv init try-scipy
cd try-scipy

<a name="conda-install"></a>
{{< admonition hint >}}
The second command changes directory into the directory of your project.
{{< /admonition >}}
Comment on lines +49 to +51
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jarrodmillman, I believe the linter is suggesting changes which would break the formatting.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could try something like

<!-- prettier-ignore-start -->
...
<!-- prettier-ignore-end -->


# Installing via Conda
3. Add SciPy to your project:

You can install SciPy from the `defaults` or `conda-forge` channels with `conda`:
uv add scipy

conda install scipy
{{< admonition note >}}
This will automatically install Python if you don't already have it installed!
{{< /admonition >}}

<a name="package_manager"></a>
{{< admonition tip >}}
You can install other Python libraries in the same way, e.g.

# Install system-wide via a package manager
uv add matplotlib

{{< /admonition >}}

4. Try out SciPy!

uv run python

This will launch a Python interpreter session, from which you can `import scipy`.

See next steps in [the SciPy user guide][scipy-user-guide].

[scipy-user-guide]: https://docs.scipy.org/doc/scipy/tutorial/

{{< admonition note >}}

After rebooting your computer, you'll want to navigate to your `try-scipy`
project directory and execute `uv run python` to drop back into a Python interpreter
with SciPy importable.
To execute a Python script, you can use `uv run myscript.py`.

Read more at [the uv guide to working on projects][uv-projects].

[uv-projects]: https://docs.astral.sh/uv/guides/projects/

{{< /admonition >}}

### Installing with `pixi`

If you work with non-Python packages, you may prefer to install SciPy as
a [Conda] package, so that you can use the same workflow for packages which
are not available on [PyPI](https://pypi.org/), the Python Package Index.
Conda can manage packages in any language, so you can use it to install
Python itself, compilers, and other languages.

[Conda]: https://docs.conda.io/projects/conda/en/latest/index.html

The steps to install SciPy from [conda-forge] using the package management
tool [`pixi`] are very similar to the steps for `uv`:

[conda-forge]: https://conda-forge.org/
[`pixi`]: https://pixi.sh/latest/

1. Install `pixi`, following [the instructions in the `pixi` documentation][install-pixi].

[install-pixi]: https://pixi.sh/latest/

2. Create a new project in a new subdirectory:

pixi init try-scipy
cd try-scipy

3. Add SciPy to your project:

pixi add scipy

4. Try out SciPy!

pixi run python

See next steps in [the SciPy user guide][scipy-user-guide].

<a name="environment-based"></a>

## Environment-based workflows

In project-based workflows, a project is a directory containing a manifest
file describing the project, a lock-file describing the exact dependencies
of the project, and the project's (potentially multiple) environments.

In contrast,
in environment-based workflows you install packages into an environment,
which you can activate and deactivate from any directory.
These workflows are well-established,
but lack some reproducibility benefits of project-based workflows.

### Installing with `pip`

1. [Install Python](https://www.python.org/downloads/).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Of course, you can use uv to do this as well and then switch back to being a simple venv dinosaur like me. But there are indeed already some comments about choice paralysis I think.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I considered leaving out the hyperlink and letting users google "how to install Python" - could do that if people prefer. I don't think we should be laying out different options for installing Python.

Copy link
Member

@stefanv stefanv Dec 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like the uv folks are urging people to switch away from activating virtual environments. And they have good technical reasons for it.

That's all fine if you've decided to switch everything to UV, but it feels problematic from a higher level UI perspective. python myscript.py now becomes some-tool-name run myscript.py, so it (a) introduces a layer on top of Python (not ideal for newcomers) and (b) binds you into a specific, non-general invocation pattern.

As I mentioned before, I suspect this pattern might break a lot of existing tutorial advice, scripts with #!/usr/bin/env python, etc. In fact, that's a question I'm very curious about: how does uv intend for developers to use it if they require uv run for everything? (I'm speaking from complete ignorance here, of course, since I haven't studied uv at all.)

I also don't like that users now need to know about uv and pip and uv pip. It's often possible to plot some narrow trajectory with these tools where you hide much of the complexity (and we should have a community guide from that perspective, perhaps via PyOpenSci, if they don't already). But then things break and you have a lot of concepts to understand before you can debug, and only uv-specific resources are helpful.

Anyway, these are just my high-level concerns; I'm sure they can all be addressed in one way for another, and I appreciate you having patience with me as I iterate them.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding to what Stefan said --- and I also never used uv --- how does the workflow with a test script in a separate repository work with auto-activation?

$ mamba activate scipy-dev
$ python dev.py shell
$ cd ~/tmp/
$ python failing_script_with a breakpoint.py

Same question about working on a project (scipy) and its dependency (scipy-doctest):

$ mamba activate scipy-dev
$ cd scipy-doctest
$ hack hack hack
# ..... in a second terminal window
$ mamba activate scipy-dev
$ cd scipy
$ python dev.py test 

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same question about working on a project (scipy) and its dependency (scipy-doctest)

Can you clarify the situation and expected behaviour here?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same question about working on a project (scipy) and its dependency (scipy-doctest)

The one word answer is "workspaces" (implemented in uv, see https://docs.astral.sh/uv/concepts/projects/workspaces/, work in progress in pixi). In pixi right now you can do what you must also do with pip/conda: you manually install one then develop the other, without tracking the dependency between these packages. There is no good way to do it today outside of workspaces (and no, editable installs don't count since those are inherently incomplete).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Uhm, it can't be that uncommon to have to fix a dependency? The behavior is

  • in the environment I've an editable install of a dependency (scipy-doctest, but it can be numpy or anything)
  • in one terminal I'm in the dependency tree, in the "bugfix" branch where I'm adding fixes
  • in the other terminal I'm in the scipy tree, where I'm running scipy tests; might be also adding fixes to match

That it's an editable install is not very relevant (can be any other way to install the dependency in this specific environment), the choice of the env manager is irrelevant (can be mamba or venv or virtualenvwrapper or ...), the pair of projects can be anything. The main point is the ability to have two unrelated project / git trees in the same environment.

Copy link
Member

@stefanv stefanv Dec 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think python has to become uv run everywhere.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

but it can be numpy or anything

You actually can't with editable installs of numpy and say scipy or scikit-learn, it won't build the package that depends on numpy.

The main point is the ability to have two unrelated project / git trees in the same environment.

That is correct. This is about the same with both methods (ignoring workspaces). In the pip/conda case you have to do cd pkg1 && pip install . then cd pkg2 && pip install . # or similar, manually keeping track of the two packages both having a dev version installed, then running your tests. If you iteratively work on pkg1 and pkg2, you just have to manage the state in your head. You can do that with pixi or uv as well - it's maybe very slightly more clumsy.

Uhm, it can't be that uncommon to have to fix a dependency?

In the grand scheme of things, it is uncommon (>95% of users only consume rather than develop packages to start with). It's harder for project-based tools, hence I think it wasn't their first priority. But the workspaces concept was basically copied from Cargo, and seems like a good answer conceptually.


2. Create and activate a virtual environment with `venv`.

{{< admonition hint >}}
See [the tutorial in the Python Packaging User Guide](https://packaging.python.org/en/latest/tutorials/installing-packages/#creating-virtual-environments).
{{< /admonition >}}

3. Install SciPy, using [`pip`]:

python -m pip install scipy

[`pip`]: https://pip.pypa.io/en/stable/getting-started/

### Installing with `conda`

[Miniforge] is the recommended way to install `conda` and [`mamba`],
two Conda-based environment managers.
After creating an environment, you can install SciPy from conda-forge as follows:

conda install scipy # or
mamba install scipy

[Miniforge]: https://conda-forge.org/download/
[`mamba`]: https://mamba.readthedocs.io/en/latest/

<a name="system-package-managers"></a>

## Installing system-wide via a system package manager
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since these methods are so problematic, we could consider only keeping the first sentence (warning) and not giving individual instructions for Debian, Fedora, and macos here.

Optionally, we could then show the route I mentioned elsewhere, which works perfectly well: install Python via system and then just create a virtual env using python -m venv. Since this page is no longer the beginner guide, I think that should be acceptable?

The suggestion is not needed for this PR, but if it seems like a sensible suggestion I'm happy to make a follow-up PR in the future.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since these methods are so problematic, we could consider only keeping the first sentence (warning) and not giving individual instructions for Debian, Fedora, and macos here.

I would be happy with that, let's see what others think.

Optionally, we could then show the route I mentioned elsewhere, which works perfectly well: install Python via system and then just create a virtual env using python -m venv. Since this page is no longer the beginner guide, I think that should be acceptable?

How does that differ from https://deploy-preview-595--scipy-org.netlify.app/install/#installing-with-pip? Just installing Python with a system package manager rather than python.org? I said in another thread that I would be happy to remove the hyperlink to python.org, but I thought that listing out options for installing Python might be overkill.

The suggestion is not needed for this PR, but if it seems like a sensible suggestion I'm happy to make a follow-up PR in the future.

👍


System package managers can install the most common Python packages.
They install packages for the entire computer, often use older versions,
and don't have as many available versions.
and don't have as many available versions. They are not the recommended
installation method.

## Ubuntu and Debian
### Ubuntu and Debian

Using `apt-get`:

sudo apt-get install python3-scipy

## Fedora
### Fedora

Using `dnf`:

sudo dnf install python3-scipy

## macOS
### macOS

macOS doesn't have a preinstalled package manager, but you can install
[Homebrew](https://brew.sh/) and use it to install SciPy (and Python itself):

brew install scipy

<a name="source"></a>
<a name="building-from-source"></a>

# Source packages
## Building from source

A word of warning: building SciPy from source can be a nontrivial exercise. We
recommend using binaries instead if those are available for your platform.
recommend using binaries instead if those are available for your platform
via one of the above methods.
For details on how to build from source, see
[this guide in the SciPy docs](https://scipy.github.io/devdocs/building/index.html).
[the building from source guide in the SciPy docs][building-docs].

[building-docs]: https://scipy.github.io/devdocs/building/index.html
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given the default recommendation to use uv for projects, I was wondering whether, in the future, we should adjust the above page accordingly?

I presume it can be used the same way pip is for SciPy development, adding a third code tab with uv run python dev.py and uv pip for installing dependencies?

Copy link
Member Author

@lucascolley lucascolley Dec 21, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given the default recommendation to use uv for projects, I was wondering whether, in the future, we should adjust the above page accordingly?

Yep! Although for SciPy, given the compiler dependencies, Pixi will probably be our best bet. I already use https://github.com/rgommers/pixi-dev-scipystack for all of my SciPy development. Windows support for PyTorch and JAX on conda-forge is still maturing (though actively - conda-forge/pytorch-cpu-feedstock#231), but we may be able to use PyPI on Windows to fill some gaps if anyone needs it.

A third tab for uv is probably not so useful, since it's relatively obvious what to do by following along with the pip instructions, and you have to do the same setup of system deps.

I presume it can be used the same way pip is for SciPy development, adding a third code tab with uv run python dev.py and uv pip for installing dependencies?

All that to say that we could do something like this. Or we could make a new page instead of adding a new tab, since there is much less to explain when most of the work is already done in writing the Pixi manifest. We may want to wait until somebody wants to try to merge Pixi support into the main repo, though.


Also I plan to add a note about scipy-stubs to the building from source page after this is merged.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, Lucas!


<a name="type-stubs"></a>

## Installing with type stubs

Static type stubs are available via a separate package, `scipy-stubs`, on
PyPI and conda-forge.
You can also install SciPy and `scipy-stubs` as a single package,
via the `scipy-stubs[scipy]` extra on PyPI, or the `scipy-typed`
package on conda-forge.
To get a specific version `x.y.z` of SciPy (such as `1.14.1`),
you should install version `x.y.z.*`, for example:

uv add "scipy-stubs[scipy]==1.14.1.*" # or
pixi add "scipy-typed=1.15.0.*" # or
python -m pip install "scipy-stubs[scipy]" # or
conda install "scipy-typed>=1.14"

Please direct questions about static typing support to
[the `scipy-stubs` GitHub repository](https://github.com/jorenham/scipy-stubs).
Loading