Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Jet paper benchmarking samples and submission scripts #35

Merged
merged 87 commits into from
Jul 20, 2021
Merged

Conversation

mlxd
Copy link
Member

@mlxd mlxd commented Jun 24, 2021

Context: All data included in the Jet paper must be reproducible by the repo main branch. This PR will include all necessary scripts and data to reproduce all such benchmarks in the paper.

Description of the Change: An examples directory is added, housing the different scripts and benchmarking examples for the Niagara HPC system for m=10 with and without slicing, and m=12 with a fixed number of slices.

Benefits: All data are directly reproducible by any user or researcher.

Possible Drawbacks: The submission scripts for Niagara are guaranteed to work while the system is online. Minimal medications may be required to run on additional systems.

Related GitHub Issues: None.

@github-actions
Copy link

github-actions bot commented Jun 24, 2021

Test Report (C++) on Ubuntu

    1 files  ±0      1 suites  ±0   0s ⏱️ ±0s
521 tests ±0  521 ✔️ ±0  0 💤 ±0  0 ❌ ±0 
870 runs  ±0  870 ✔️ ±0  0 💤 ±0  0 ❌ ±0 

Results for commit d74502b. ± Comparison against base commit d74502b.

♻️ This comment has been updated with latest results.

@github-actions
Copy link

github-actions bot commented Jun 24, 2021

Test Report (C++) on MacOS

    1 files  ±0      1 suites  ±0   0s ⏱️ ±0s
521 tests ±0  521 ✔️ ±0  0 💤 ±0  0 ❌ ±0 
870 runs  ±0  870 ✔️ ±0  0 💤 ±0  0 ❌ ±0 

Results for commit 225a667. ± Comparison against base commit 29805e4.

♻️ This comment has been updated with latest results.

Copy link
Collaborator

@Mandrenkov Mandrenkov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should be OK to merge once we've finalized the benchmarks in the paper; the important thing is to enable others to reproduce the results in the paper.

As a side note, is examples/paper_benchmarks/CPU/cot_cpu_m10/niagara_submit.slurm meant to be empty?

@github-actions
Copy link

github-actions bot commented Jul 6, 2021

Test Report (Python) on Ubuntu

    1 files  ±0      1 suites  ±0   8s ⏱️ ±0s
529 tests ±0  529 ✔️ ±0  0 💤 ±0  0 ❌ ±0 

Results for commit d74502b. ± Comparison against base commit d74502b.

♻️ This comment has been updated with latest results.

Copy link
Collaborator

@Mandrenkov Mandrenkov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good - thanks again for putting this together Lee!

examples/paper_benchmarks/README.md Outdated Show resolved Hide resolved
examples/paper_benchmarks/README.md Outdated Show resolved Hide resolved
examples/paper_benchmarks/README.md Outdated Show resolved Hide resolved
examples/paper_benchmarks/README.md Outdated Show resolved Hide resolved
@trevor-vincent trevor-vincent self-requested a review July 19, 2021 15:30
Copy link
Contributor

@trevor-vincent trevor-vincent left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

@mlxd
Copy link
Member Author

mlxd commented Jul 20, 2021

Benchmarks now will pull down the 0.2.0 tag when ready. I'll merge this shortly, and we can tag when pushed out.

@mlxd mlxd merged commit d74502b into main Jul 20, 2021
@mlxd mlxd deleted the paper_bm branch July 20, 2021 07:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants