Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[PR]: Add Parallel Computing with Dask Jupyter Notebook #489

Merged
merged 45 commits into from
Jun 10, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
0d0ff3a
Add Parallel Computing with Dask Jupyrer Notebook
tomvothecoder May 23, 2023
8e6cf04
Add script for performance metrics
tomvothecoder Jun 7, 2023
6aff17e
Update perf script
tomvothecoder Jun 9, 2023
88c9e88
Clean up script for perf metrics
tomvothecoder Oct 17, 2023
e87faee
Update module docstring for script
tomvothecoder Oct 17, 2023
1d155a9
Refactor perf scripts
tomvothecoder Oct 19, 2023
2b955b1
Add csv and plot results
tomvothecoder Oct 19, 2023
c83345c
Update output comparison script for validation
tomvothecoder Oct 19, 2023
659dc71
Update perf metric scripts to run all apis
tomvothecoder Oct 26, 2023
4f523ae
Delete performance metric scripts
tomvothecoder Oct 26, 2023
5ef0851
Update notebook with scheduler info
tomvothecoder Feb 27, 2024
dd00b60
Remove outdated horizontal regridding notebook
tomvothecoder Feb 28, 2024
94c060c
Update formatting in notebook
tomvothecoder Mar 7, 2024
2af90ff
Fix notebook formatting
tomvothecoder Mar 7, 2024
f93474a
Update notebook
tomvothecoder Mar 7, 2024
de534ec
Add dask logo
tomvothecoder Mar 7, 2024
c6edeec
Update variable names
tomvothecoder Mar 7, 2024
f2e920a
Update var names
tomvothecoder Mar 7, 2024
00ee763
Add latest changes
tomvothecoder Apr 16, 2024
e3f8906
Update notebook
tomvothecoder Apr 16, 2024
e11824f
Update notebook
tomvothecoder Apr 16, 2024
4e3e8cf
Fix order of notebooks
tomvothecoder Apr 16, 2024
d5ef246
Update notebook
tomvothecoder Apr 17, 2024
c6f8765
Update notebook
tomvothecoder Apr 17, 2024
3cbcce4
Clean up sections and add `memory_limit`
tomvothecoder Apr 30, 2024
ed5dc83
Update overview section
tomvothecoder Apr 30, 2024
c5c1391
Update setup
tomvothecoder Apr 30, 2024
afe4371
Run dask notebook cells
tomvothecoder Apr 30, 2024
c6e3241
Fix code markdown formatting
tomvothecoder Apr 30, 2024
8431483
Add Dask to gallery
tomvothecoder Apr 30, 2024
cf96e8b
Fix dask logo ref link
tomvothecoder Apr 30, 2024
f24a9fb
Fix dask logo in gallery
tomvothecoder Apr 30, 2024
dda5f5b
Fix dask logo ref link
tomvothecoder Apr 30, 2024
855cccc
Address Steve's comments
tomvothecoder Jun 7, 2024
0d61eb5
Fix mdash
tomvothecoder Jun 7, 2024
19e23a5
Add latest changes
tomvothecoder Jun 7, 2024
040c043
Fix notebook
tomvothecoder Jun 7, 2024
61a40d1
Fix title level
tomvothecoder Jun 7, 2024
d582c00
Add links xCDAT example notebook
tomvothecoder Jun 7, 2024
9ec25f3
Fix introduction to xcdat notebook (attempt)
tomvothecoder Jun 7, 2024
1302acb
Replace svg with png in index.rst to fix rtd build
tomvothecoder Jun 10, 2024
e88c23d
Revert some sections of overview.rst to test
tomvothecoder Jun 10, 2024
3ab2b89
Add projects using xCDAT
tomvothecoder Jun 10, 2024
3f0ca5f
Fix contributors section
tomvothecoder Jun 10, 2024
b3ed34c
Remove contributors graph
tomvothecoder Jun 10, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file added docs/_static/dask-array.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/_static/dask-dashboard-example.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
4 changes: 4 additions & 0 deletions docs/_static/dask-overview-schedulers.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/_static/icons/book.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
4 changes: 0 additions & 4 deletions docs/_static/icons/book.svg

This file was deleted.

Binary file added docs/_static/icons/play.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
4 changes: 0 additions & 4 deletions docs/_static/icons/play.svg

This file was deleted.

Binary file added docs/_static/icons/plot.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
23 changes: 0 additions & 23 deletions docs/_static/icons/plot.svg

This file was deleted.

4 changes: 0 additions & 4 deletions docs/_static/icons/wrench.svg

This file was deleted.

Binary file added docs/_static/thumbnails/dask-logo.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
10 changes: 3 additions & 7 deletions docs/demos/1-25-23-cwss-seminar/introduction-to-xcdat.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -4770,13 +4770,9 @@
"\n",
"Visit these pages for more guidance (e.g., when to parallelize):\n",
"\n",
"- Ongoing xCDAT Dask Investigation: https://github.com/xCDAT/xcdat/discussions/376\n",
" - Performance metrics, best practices, and possibly a guide\n",
"- Parallel computing with Dask: https://docs.xarray.dev/en/stable/user-guide/dask.html\n",
"- Xarray with Dask Arrays: https://examples.dask.org/xarray.html\n",
"\n",
"\n",
"[//]: # \"TODO: Add link to xCDAT Dask guidance\""
"- Parallel computing with Dask (xCDAT): https://xcdat.readthedocs.io/en/latest/examples/parallel-computing-with-dask.html\n",
"- Parallel computing with Dask (Xarray): https://docs.xarray.dev/en/stable/user-guide/dask.html\n",
"- Xarray with Dask Arrays: https://examples.dask.org/xarray.html\n"
]
},
{
Expand Down
128 changes: 63 additions & 65 deletions docs/examples/introduction-to-xcdat.ipynb

Large diffs are not rendered by default.

Loading
Loading