Skip to content

Commit

Permalink
add more intro content to index.rst (#104)
Browse files Browse the repository at this point in the history
* add more intro content to index.rst

- Add info on how to get help, request features
- Add example CLI to the homepage
- Add links to installation and user guide.

* enh apptainer docs

Improve the apptainer docs. Unset TORCH_HOME in the command line example
and add a note of why that is necessary.

Also add a note to the geojson converter that one can drag and drop the
json to load it into qupath.

* sleep in test_issue_97 so timestamp is different
  • Loading branch information
kaczmarj authored Feb 24, 2023
1 parent 96c359b commit a937f29
Show file tree
Hide file tree
Showing 4 changed files with 71 additions and 13 deletions.
31 changes: 27 additions & 4 deletions docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,6 @@
WSInfer: blazingly fast inference on whole slide images
=======================================================

🔥 🚀 **WSInfer** is a blazingly fast pipeline to run patch-based classification models
on whole slide images.

.. image:: https://github.com/kaczmarj/wsinfer/actions/workflows/ci.yml/badge.svg
:alt: GitHub workflow status
Expand All @@ -21,6 +19,29 @@ on whole slide images.

|
🔥 🚀 **WSInfer** is a blazingly fast pipeline to run patch-based classification models
on whole slide images. It includes several built-in models for tumor and lymphocyte
detection, and it can be used with any PyTorch model as well. The built-in models
:ref:`are listed below <available-models>`.

Running inference on whole slide images is done with a single command line. For example,
this is the command used to generate the heatmap on this page.

::

CUDA_VISIBLE_DEVICES=0 wsinfer run \
--wsi-dir slides/ \
--results-dir results/ \
--model resnet34 \
--weights TCGA-BRCA-v1 \
--num-workers 8

To get started, please :ref:`install WSInfer<installing>` and check out the :ref:`User Guide`.
To get help, report issues or request features, please
`submit a new issue <https://github.com/SBU-BMI/wsinfer/issues/new>`_ on our GitHub
repository. If you would like to make your patch classification model available in WSInfer, please
get in touch with us! You can `submit a new GitHub issue <https://github.com/SBU-BMI/wsinfer/issues/new>`_.

.. toctree::
:maxdepth: 1
:caption: Contents:
Expand All @@ -41,6 +62,8 @@ on whole slide images.
| |img-tissue| | |img-heatmap| |
+----------------+------------------------------+

.. _available-models:

Available models
----------------

Expand All @@ -55,7 +78,7 @@ Available models
- Reference
* - Breast adenocarcinoma detection
- no-tumor, tumor
- inceptionv4
- inception_v4
- TCGA-BRCA-v1
- 350 @ 0.25
- `Ref <https://doi.org/10.1016%2Fj.ajpath.2020.03.012>`_
Expand Down Expand Up @@ -91,7 +114,7 @@ Available models
- `Ref <https://github.com/SBU-BMI/quip_prad_cancer_detection>`_
* - Tumor-infiltrating lymphocyte detection
- til-negative, til-positive
- inceptionv4nobn
- inception_v4nobn
- TCGA-TILs-v1
- 100 @ 0.5
- `Ref <https://doi.org/10.3389/fonc.2021.806603>`_
Expand Down
2 changes: 2 additions & 0 deletions docs/installing.rst
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
.. _installing:

Installing and getting started
==============================

Expand Down
48 changes: 39 additions & 9 deletions docs/user_guide.rst
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
.. _User Guide:

User Guide
==========

Expand Down Expand Up @@ -37,8 +39,8 @@ The option :code:`--num-workers` controls how many processes to use for data loa
Using more workers will typically speed up processing at the expense of more RAM and
processor use.

The results directory includes a file named :code:`run_metadata.json` with provenance
information.
The results directory includes a file named :code:`run_metadata_TIMESTAMP.json` with
provenance information.

Run model inference in containers
---------------------------------
Expand All @@ -57,19 +59,44 @@ addition of downloaded weights.
The base image :code:`kaczmarj/wsinfer` does not include downloaded models. The models
will be downloaded automatically but will be lost when the container is stopped. Use
versioned, application-specific containers like
:code:`kaczmarj/wsinfer:0.3.2-tumor-brca` as they already include weights.
:code:`kaczmarj/wsinfer:0.3.5-tumor-brca` as they already include weights.

Apptainer/Singularity
^^^^^^^^^^^^^^^^^^^^^

We use :code:`apptainer` in this example. You can replace that name with
:code:`singularity` if you do not have :code:`apptainer`.

::
Pull the container: ::

apptainer pull docker://kaczmarj/wsinfer:0.3.2-tumor-paad
CUDA_VISIBLE_DEVICES=0 apptainer run --nv --bind $(pwd) wsinfer_0.3.2-tumor-paad.sif \
run --wsi-dir slides/ --results-dir results/
apptainer pull docker://kaczmarj/wsinfer:latest

Run inference: ::

apptainer run \
--nv \
--bind $(pwd) \
--env TORCH_HOME="" \
--env CUDA_VISIBLE_DEVICES=0 \
wsinfer_latest.sif run \
--wsi-dir slides/ \
--results-dir results/ \
--model resnet34 \
--weights TCGA-BRCA-v1 \
--num-workers 8

.. note::

If you get an error like :code:`Read-only file system: '/var/lib/wsinfer/hub/checkpoints/tmp4rs7aaz9'`,
then unset the environment variable :code:`TORCH_HOME`.

::

apptainer run --nv --bind $(pwd) --env TORCH_HOME="" wsinfer_latest.sif

This happens when model weights are downloaded. Apptainer/Singularity containers are
not writable, so writing the weights file to the container fails. Unsetting
:code:`TORCH_HOME` will cause the models to be downloaded to the current directory.

Docker
^^^^^^
Expand All @@ -93,7 +120,7 @@ When mounting data, keep in mind that the workdir in the Docker container is :co

Pull the Docker image: ::

docker pull kaczmarj/wsinfer:0.3.2-tumor-paad
docker pull kaczmarj/wsinfer:latest

Run inference: ::

Expand All @@ -103,7 +130,7 @@ Run inference: ::
--env CUDA_VISIBLE_DEVICES=0 \
--user $(id -u):$(id -g) \
--mount type=bind,source=$(pwd),target=/work/ \
kaczmarj/wsinfer:0.3.2-tumor-paad run \
kaczmarj/wsinfer:latest run \
--wsi-dir slides/ \
--results-dir results/ \
--model resnet34 \
Expand Down Expand Up @@ -167,6 +194,9 @@ GeoJSON is a JSON format compatible with whole slide image viewers like QuPath.

wsirun togeojson results/ geojson-results/

If you open one of your slides in QuPath, you can drag and drop the corresponding
JSON file into the QuPath window to load the model outputs.

Convert model outputs to Stony Brook format (QuIP)
--------------------------------------------------

Expand Down
3 changes: 3 additions & 0 deletions tests/test_all.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
import platform
import subprocess
import sys
import time
from typing import List

from click.testing import CliRunner
Expand Down Expand Up @@ -1047,6 +1048,8 @@ def test_issue_97(tmp_path: Path, tiff_image: Path):
metas = list(results_dir.glob("run_metadata_*.json"))
assert len(metas) == 1

time.sleep(2) # make sure some time has passed so the timestamp is different

# Run again...
result = runner.invoke(
cli,
Expand Down

0 comments on commit a937f29

Please sign in to comment.