(feat): Update notebooks for new leiden defaults #77

ilan-gold · 2024-01-17T20:03:24Z

See scverse/scanpy#2815 for implementation prompting the changes. The changes here generally reflect the reordering of labels. It does not look like results changed in a meaningful way, so changes should be minimal in that sense.

review-notebook-app · 2024-01-17T20:03:30Z

Check out this pull request on

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB

ilan-gold · 2024-01-17T20:46:21Z

Current issues:

pbmc3k.ipynb
- pca on highly_variable (as was the default evidently) or not in cell 22
spatial/basic-analysis.ipynb
- Why did the data change (3rd cell)?
- Why doesn't sc.pl.spatial not work in cell 14?
- SpatialDE.run is not working for no apparent reason UPDATE: it's hasn't been updated in two years, the issue is pandas related
spatial/integration-scanorama.ipynb
- number of categories for plotting the 2 image clusters is 18, but this is less than the total number (warning was present before)
- Link for adata_processed not working? Should we regenerate?
- scanroma result changed?

Overall issues are the number of warnings, which is quite high without suppression. Is this expected, and if so, how were they repressed in the past for building the docs?

ilan-gold · 2024-01-22T11:17:25Z

So there are a few different strains here:

We need to get this stuff running in a reproducible CI env (Automate tutorial building in CI #79).
We need to fix the current outputs so that the code runs (currently, some notebooks outright do not run)
We need to update the outputs so they match the scanpy outputs even if it is not totally reproducible (i.e., this current PR)

Points 2. and 3. will be the blockers here for 1.10. 1. can come later. The focus for now will just be on these notebooks.

…py-tutorials into ig/leiden_changes

flying-sheep · 2024-01-25T09:15:55Z

Overall issues are the number of warnings, which is quite high without suppression. Is this expected, and if so, how were they repressed in the past for building the docs?

I have no idea why they were displayed without a stack trace in old renderings of the notebooks. Maybe some manual setting somewhere?

By the way, those are not the warnings that will make a sphinx build fail when running sphinx in CI or with -W error. The warnings thrown during notebook creation will simply be displayed unless somehow silenced/removed

ilan-gold · 2024-01-25T09:28:53Z

@flying-sheep I understand, I was just curious. I would be in favor of silencing them if you'd be ok with that.

flying-sheep · 2024-01-25T10:50:11Z

Depends:

If there’s instances of warnings coming

only out of library code, like all these plotting warnings, we should file corresponding scanpy bug and silence them per-notebook in a hidden code cell at the start
both in library code and user code, we can silence the ones coming from library code on a case by case basis by adding hidden code cells before and after the offending cell.

Otherwise:

UserWarning: Received a view of an AnnData. Making a copy.

Should be avoided with adata[...].copy().

ConvergenceWarning: lbfgs failed to converge (status=1): STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.

Should probably fixed?

I don‘t think this is a good idea basically anywhere, one can almost always be more specific

warnings.filterwarnings('ignore')

…py-tutorials into ig/leiden_changes

review-notebook-app · 2024-02-26T16:08:02Z

View / edit / reply to this conversation on ReviewNB

flying-sheep commented on 2024-02-26T16:08:01Z
----------------------------------------------------------------

So the cluster order changed because you did flavor='igraph'? If so this is fine!

ilan-gold commented on 2024-02-26T16:21:36Z
----------------------------------------------------------------

Yes!

review-notebook-app · 2024-02-26T16:08:02Z

View / edit / reply to this conversation on ReviewNB

flying-sheep commented on 2024-02-26T16:08:02Z
----------------------------------------------------------------

The former “B” cluster is now labelled “CD8 T”, plus some other changes … this should be fixed!

ilan-gold commented on 2024-02-26T16:23:15Z
----------------------------------------------------------------

Huh, I am not sure what happened here. The local diff was correct. It's possible something didn't push.

review-notebook-app · 2024-02-26T16:08:04Z

View / edit / reply to this conversation on ReviewNB

flying-sheep commented on 2024-02-26T16:08:03Z
----------------------------------------------------------------

Line #4.    warnings.filterwarnings('ignore')

This seems to be left from when we thought the display version still had them. Please remove this line and rerun.

review-notebook-app · 2024-02-26T16:08:05Z

View / edit / reply to this conversation on ReviewNB

flying-sheep commented on 2024-02-26T16:08:05Z
----------------------------------------------------------------

very weird that this warning is still there, is that an anndata bug?

ilan-gold commented on 2024-02-26T17:30:49Z
----------------------------------------------------------------

Nope, this cell was not run

flying-sheep

The filterwarnings needs to be removed and the cluster order fixed.

Regarding the weird “obsm setting made view into actual” thing I don‘t know what’s going on.

ilan-gold · 2024-02-26T16:21:37Z

Yes!

View entire conversation on ReviewNB

ilan-gold · 2024-02-26T16:23:17Z

Huh, I am not sure what happened here. The local diff was correct. It's possible something didn't push.

View entire conversation on ReviewNB

ilan-gold · 2024-02-26T17:30:51Z

Nope, this cell was not run

View entire conversation on ReviewNB

(feat): first pass at notebook updates

759070a

(fix): correct cluster labelings

668f158

ilan-gold added 7 commits January 22, 2024 12:17

Merge branch 'master' into ig/leiden_changes

f8f098b

(wip)

1e8e899

(feat): update pbmc for correct labels

6ba1866

Merge branch 'ig/leiden_changes' of https://github.com/ilan-gold/scan…

3124271

…py-tutorials into ig/leiden_changes

Merge branch 'master' into ig/leiden_changes

124154f

(chore): remove old changes

ca6f5b7

(fix): update other notebooks

088e82f

ilan-gold marked this pull request as ready for review January 24, 2024 15:09

(fix): same groups for plotting/core.ipynb

84d8d43

ilan-gold and others added 8 commits January 29, 2024 10:28

(fix): correct resolution

84b753f

Fix ImplicitModificationWarning

9d691cb

(chore): update using new leiden settings

063e030

(feat): add pearson notebook

55ba8ef

Merge branch 'ig/leiden_changes' of https://github.com/ilan-gold/scan…

17d0fbe

…py-tutorials into ig/leiden_changes

foo

6a20775

(fix): increase max_iters to remove warning

bf0e341

(chore): copy to remove warning

19ddcb5

ilan-gold requested a review from flying-sheep February 26, 2024 13:31

flying-sheep requested changes Feb 26, 2024

View reviewed changes

ilan-gold added 4 commits February 26, 2024 17:31

(fix): pbmc3k notebook ordering

3ee070d

(chore): no warnings

f5664ed

(fix): run basic-analysis through

741ea9e

(fix): basic-analysis leiden implementation

23dc8f9

ilan-gold requested a review from flying-sheep February 26, 2024 17:31

ilan-gold enabled auto-merge (squash) February 26, 2024 17:31

flying-sheep approved these changes Feb 27, 2024

View reviewed changes

ilan-gold merged commit d42d8e6 into scverse:master Feb 27, 2024
1 check passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

(feat): Update notebooks for new leiden defaults #77

(feat): Update notebooks for new leiden defaults #77

ilan-gold commented Jan 17, 2024 •

edited

Loading

review-notebook-app bot commented Jan 17, 2024

ilan-gold commented Jan 17, 2024 •

edited by flying-sheep

Loading

ilan-gold commented Jan 22, 2024 •

edited

Loading

flying-sheep commented Jan 25, 2024 •

edited

Loading

ilan-gold commented Jan 25, 2024

flying-sheep commented Jan 25, 2024 •

edited

Loading

review-notebook-app bot commented Feb 26, 2024 •

edited

Loading

review-notebook-app bot commented Feb 26, 2024 •

edited

Loading

review-notebook-app bot commented Feb 26, 2024 •

edited

Loading

review-notebook-app bot commented Feb 26, 2024 •

edited

Loading

flying-sheep left a comment

ilan-gold commented Feb 26, 2024

ilan-gold commented Feb 26, 2024

ilan-gold commented Feb 26, 2024

(feat): Update notebooks for new leiden defaults #77

(feat): Update notebooks for new leiden defaults #77

Conversation

ilan-gold commented Jan 17, 2024 • edited Loading

review-notebook-app bot commented Jan 17, 2024

ilan-gold commented Jan 17, 2024 • edited by flying-sheep Loading

ilan-gold commented Jan 22, 2024 • edited Loading

flying-sheep commented Jan 25, 2024 • edited Loading

ilan-gold commented Jan 25, 2024

flying-sheep commented Jan 25, 2024 • edited Loading

review-notebook-app bot commented Feb 26, 2024 • edited Loading

review-notebook-app bot commented Feb 26, 2024 • edited Loading

review-notebook-app bot commented Feb 26, 2024 • edited Loading

review-notebook-app bot commented Feb 26, 2024 • edited Loading

flying-sheep left a comment

Choose a reason for hiding this comment

ilan-gold commented Feb 26, 2024

ilan-gold commented Feb 26, 2024

ilan-gold commented Feb 26, 2024

ilan-gold commented Jan 17, 2024 •

edited

Loading

ilan-gold commented Jan 17, 2024 •

edited by flying-sheep

Loading

ilan-gold commented Jan 22, 2024 •

edited

Loading

flying-sheep commented Jan 25, 2024 •

edited

Loading

flying-sheep commented Jan 25, 2024 •

edited

Loading

review-notebook-app bot commented Feb 26, 2024 •

edited

Loading

review-notebook-app bot commented Feb 26, 2024 •

edited

Loading

review-notebook-app bot commented Feb 26, 2024 •

edited

Loading

review-notebook-app bot commented Feb 26, 2024 •

edited

Loading