Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore/ci: Print if stamp is missing #124

Draft
wants to merge 101 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
101 commits
Select commit Hold shift + click to select a range
79d1834
wip
ZuseZ4 Dec 26, 2023
12292ca
got autodiff macro to do something
ZuseZ4 Dec 26, 2023
175d236
cleanup, first helper
ZuseZ4 Dec 26, 2023
095eabd
adding Enzyme submodule
ZuseZ4 Jan 4, 2024
c3b73ce
adding extra error message
ZuseZ4 Jan 4, 2024
e4e3430
adding ffi, not building enzyme yet
ZuseZ4 Jan 5, 2024
51e0b65
Build and link Enzyme durin bootstrap
ZuseZ4 Jan 5, 2024
f253e67
update rustc_middle, rustc_monomorphize, and (partly) cg_ssa
ZuseZ4 Jan 8, 2024
b394066
update the backend, cg_ssa and cg_llvm
ZuseZ4 Jan 8, 2024
6700542
add minimal ast based macro parser
ZuseZ4 Jan 15, 2024
bf72f16
yeet
ZuseZ4 Jan 17, 2024
001bc8a
compiles and runs binaries, but without Enzyme yet
ZuseZ4 Jan 24, 2024
e3ba2d2
I will clean this up, I promise!
ZuseZ4 Jan 25, 2024
3e89575
remove leftovers
ZuseZ4 Jan 25, 2024
f17e81d
mark autodiff_into as builtin and permanently unstable
ZuseZ4 Jan 25, 2024
c97f625
follow rustc_ naming convention
ZuseZ4 Jan 25, 2024
494b102
It works (somewhat)
ZuseZ4 Jan 25, 2024
66f86a6
Fwd: Rename Duplicated -> Dual
ZuseZ4 Feb 2, 2024
59bc30f
properly allow interior instability
ZuseZ4 Feb 4, 2024
12b180e
make d_fnc decl a bit more precise
ZuseZ4 Feb 4, 2024
47d6d3c
running x.py fmt
ZuseZ4 Feb 5, 2024
e4b7ef1
various fn_decl fixes
ZuseZ4 Feb 5, 2024
9cbdf68
More precise Enzyme settings
ZuseZ4 Feb 5, 2024
bdc8e5d
cleanup, reduce diff
ZuseZ4 Feb 5, 2024
c8c4ea3
cleanups
ZuseZ4 Feb 9, 2024
7b0d0f1
cleanups2
ZuseZ4 Feb 9, 2024
a06caa7
make Enzyme build optional
ZuseZ4 Feb 9, 2024
2de98a2
Implement fallback to optionally not link and build Enzyme
ZuseZ4 Feb 9, 2024
293ab97
adding CI and updating README
ZuseZ4 Feb 10, 2024
4889ad3
fix mem leak, fix logic bug, cleanup
ZuseZ4 Feb 11, 2024
769622d
try to make enzyme configurable, not working yet
ZuseZ4 Feb 11, 2024
e394546
small cleanups, make Enzyme config nicer
ZuseZ4 Feb 12, 2024
84c418c
Update submodule 'enzyme'
ZuseZ4 Feb 12, 2024
4c2a59f
fix reverse_mode setting, various cleanups
ZuseZ4 Feb 17, 2024
e54da7b
allow differentiating a fnc multiple times
ZuseZ4 Feb 17, 2024
c90ab5a
improve inlining prevention
ZuseZ4 Feb 17, 2024
f8c263b
use propper rustc error handler for mode/act check
ZuseZ4 Feb 18, 2024
273c2b5
use propper rustc error handler for type/act check
ZuseZ4 Feb 18, 2024
bdb9449
add CI back
ZuseZ4 Feb 18, 2024
9445d76
better error, fix activity check
ZuseZ4 Feb 19, 2024
3e0ba90
add parser support for const
ZuseZ4 Feb 19, 2024
a74948f
fix for opaque ptrs
ZuseZ4 Feb 27, 2024
0ec88ee
imprv placeholder towards removing indirection
ZuseZ4 Mar 8, 2024
691d2f9
simplify wrapper
ZuseZ4 Mar 10, 2024
1cc0216
cmake improvement
ZuseZ4 Mar 10, 2024
618bcef
replace loop {} by actually correct return, use default for floats
ZuseZ4 Mar 11, 2024
a46e667
adjust cleanup to new return handling
ZuseZ4 Mar 11, 2024
4698879
improve error handling for unrecognized mode
ZuseZ4 Mar 13, 2024
7242970
fixing forward mode with Dual return
ZuseZ4 Mar 14, 2024
78bd09a
add handling for ->()
ZuseZ4 Mar 18, 2024
6933f08
enforce fat-lto when using autodiff
ZuseZ4 Mar 19, 2024
7352472
handle more () ret cases
ZuseZ4 Mar 19, 2024
daad666
Update README.md
ZuseZ4 Mar 19, 2024
294f137
updating enzyme to latest master
ZuseZ4 Mar 20, 2024
215089d
rust-alloc testing
ZuseZ4 Mar 20, 2024
1eff53c
div bugfixes
ZuseZ4 Mar 26, 2024
15f7389
ci: ensure nightly and rustup are available
jedbrown Mar 28, 2024
cf35163
ci: cache LLVM build and use rustbook for testing
jedbrown Mar 29, 2024
fceee05
build: fix stamp logic for rebuilding enzyme
jedbrown Mar 29, 2024
ea3d213
ci: rustbook tests workspace, not examples
jedbrown Mar 29, 2024
518390f
ci: cache Enzyme and use rustbook@main
jedbrown Mar 29, 2024
e5f35f4
add tt to mem calls
ZuseZ4 Mar 29, 2024
ccb9fab
handle slice abi. Wohooo. Also other fixes
ZuseZ4 Mar 29, 2024
5c5de7c
Merge pull request #82 from EnzymeAD/morett
jedbrown Mar 29, 2024
7ac8be7
simplify binder handling (#83)
ZuseZ4 Mar 30, 2024
3e116e5
update enzyme (#85)
ZuseZ4 Mar 31, 2024
1f65049
Upgr enzyme2 (#87)
ZuseZ4 Mar 31, 2024
ca1aa97
No dbg call (#88)
ZuseZ4 Mar 31, 2024
3a582e4
Update enzyme (#89)
ZuseZ4 Mar 31, 2024
415c714
make post opts optional (#90)
ZuseZ4 Mar 31, 2024
fca0b58
prevent ice by first checking that macro is applicable (#91)
ZuseZ4 Mar 31, 2024
e0e8180
catch wrong spelled activities (#92)
ZuseZ4 Mar 31, 2024
eafa142
safety check for slices! (#93)
ZuseZ4 Apr 1, 2024
b288111
update enzyme (#94)
ZuseZ4 Apr 1, 2024
672f84a
default to C++-Enzyme opt pipeline (#95)
ZuseZ4 Apr 1, 2024
af8a8a6
fix wrong variable usage
ZuseZ4 Apr 1, 2024
7468cf8
fix panic calling code and iteration order (#96)
ZuseZ4 Apr 2, 2024
26c676b
enable runtime activity (#98)
ZuseZ4 Apr 2, 2024
683aa31
printing (#99)
ZuseZ4 Apr 2, 2024
8e61557
new flag (#101)
ZuseZ4 Apr 2, 2024
eab555d
updatenzyme (#102)
ZuseZ4 Apr 2, 2024
1218cb2
Warn if the shadow will have internal immutability (#103)
ZuseZ4 Apr 3, 2024
1e32009
impl new modes for higher order ad (#106)
ZuseZ4 Apr 3, 2024
b5a3130
adding two more flags (#109)
ZuseZ4 Apr 6, 2024
af4d766
chore: Use correct commit hash as github cache key if submodule is no…
I-Al-Istannen Apr 7, 2024
948848d
support self ty (#110)
ZuseZ4 Apr 11, 2024
ffdbffb
add rust error for activity to arg count missmatch (#111)
ZuseZ4 Apr 11, 2024
17c772f
unbreak ActiveOnly return (#112)
ZuseZ4 Apr 12, 2024
9a411dc
handle DualOnly ret more reliably (#113)
ZuseZ4 Apr 12, 2024
db2a069
last commit broke the right order of updating attributes (#114)
ZuseZ4 Apr 13, 2024
5747648
fix higher-order { float } ret case (#115)
ZuseZ4 Apr 14, 2024
15bfd28
fix fwd test case (#116)
ZuseZ4 Apr 14, 2024
b445b6c
ci: update rustbook tests to use ENYZME_LOOSE_TYPES (#117)
jedbrown Apr 19, 2024
4cae481
Update enzyme-ci.yml (#84)
ZuseZ4 Apr 19, 2024
c9b28e7
chore: Cache random stuff (#118)
I-Al-Istannen Apr 22, 2024
c00250e
fix ci (#119)
ZuseZ4 Apr 23, 2024
75b31f9
Update README.md (#120)
ZuseZ4 May 1, 2024
fc4ab22
ci: Print if stamp is missing
I-Al-Istannen May 8, 2024
9e4c000
ci: Add tmate
I-Al-Istannen May 10, 2024
bd475eb
ci: Do not run further
I-Al-Istannen May 10, 2024
0ebffb1
ci: create cache folders manually?
I-Al-Istannen May 10, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
103 changes: 103 additions & 0 deletions .github/workflows/enzyme-ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
name: Rust CI

on:
push:
branches:
- master
pull_request:
branches:
- master
merge_group:

jobs:
build:
name: Rust Integration CI LLVM ${{ matrix.llvm }} ${{ matrix.build }} ${{ matrix.os }}
runs-on: ${{ matrix.os }}

strategy:
fail-fast: false
matrix:
os: [openstack22]

timeout-minutes: 600
steps:
- name: Checkout Rust source
uses: actions/checkout@v4
with:
submodules: true # check out all submodules so the cache can work correctly
fetch-depth: 2
- uses: dtolnay/rust-toolchain@nightly
- name: Create cache folders
run: |
mkdir -p build/build/x86_64-unknown-linux-gnu/llvm
mkdir -p build/build/x86_64-unknown-linux-gnu/enzyme
mkdir -p build/build/bootstrap/
mkdir -p build/build/x86_64-unknown-linux-gnu/stage0-rustc/
mkdir -p build/build/x86_64-unknown-linux-gnu/stage0-std/
mkdir -p build/build/x86_64-unknown-linux-gnu/stage0-tools/
mkdir -p build/build/x86_64-unknown-linux-gnu/stage1-std/
- name: Get LLVM commit hash
id: llvm-commit
run: echo "HEAD=$(git rev-parse HEAD:src/llvm-project)" >> $GITHUB_OUTPUT
- name: Cache LLVM
id: cache-llvm
uses: actions/cache@v4
with:
path: build/build/x86_64-unknown-linux-gnu/llvm
key: ${{ matrix.os }}-llvm-${{ steps.llvm-commit.outputs.HEAD }}
- name: Get Enzyme commit hash
id: enzyme-commit
run: echo "HEAD=$(git rev-parse HEAD:src/tools/enzyme)" >> $GITHUB_OUTPUT
- name: Cache Enzyme
id: cache-enzyme
uses: actions/cache@v4
with:
path: build/build/x86_64-unknown-linux-gnu/enzyme
key: ${{ matrix.os }}-enzyme-${{ steps.enzyme-commit.outputs.HEAD }}
- name: Cache bootstrap/stage0 artifacts for incremental builds
uses: actions/cache@v4
with:
path: |
build/build/bootstrap/
build/build/x86_64-unknown-linux-gnu/stage0-rustc/
build/build/x86_64-unknown-linux-gnu/stage0-std/
build/build/x86_64-unknown-linux-gnu/stage0-tools/
build/build/x86_64-unknown-linux-gnu/stage1-std/
# Approximate stable hash. It doesn't matter too much when this goes out of sync as it just caches
# some stage0/stage1 dependencies and stdlibs which *hopefully* are hash-keyed.
key: enzyme-rust-incremental-${{ runner.os }}-${{ hashFiles('src/**/Cargo.lock', 'Cargo.lock') }}
restore-keys: |
enzyme-rust-incremental-${{ runner.os }}
- name: Setup tmate session
uses: mxschmitt/action-tmate@v3
- name: Fail
run: 'false'
# - name: Build
# run: |
# wget -O - https://apt.llvm.org/llvm-snapshot.gpg.key|sudo apt-key add -
# sudo apt-add-repository "deb http://apt.llvm.org/`lsb_release -c | cut -f2`/ llvm-toolchain-`lsb_release -c | cut -f2`-17 main" || true
# sudo apt-get -y update
# sudo apt-get install -y lld-17
# mkdir lld-path-manipulation
# ln -s "$(which lld-17)" lld-path-manipulation/lld
# ln -s "$(which lld-17)" lld-path-manipulation/ld
# ln -s "$(which lld-17)" lld-path-manipulation/ld.lld
# ln -s "$(which lld-17)" lld-path-manipulation/lld-17
# export PATH="$PWD/lld-path-manipulation:$PATH"
# mkdir -p build
# cd build
# rm -f config.toml
# ../configure --enable-llvm-link-shared --enable-llvm-plugins --enable-llvm-enzyme --set=rust.use-lld=true --release-channel=nightly --enable-llvm-assertions --enable-option-checking --enable-ninja --disable-docs
# ../x.py build --stage 1 library/std library/proc_macro library/test tools/rustdoc
# rustup toolchain link enzyme build/host/stage1
# - name: checkout Enzyme/rustbook
# uses: actions/checkout@v4
# with:
# repository: EnzymeAD/rustbook
# ref: main
# path: rustbook
# - name: test Enzyme/rustbook
# working-directory: rustbook
# run: |
# cargo +enzyme test
# ENZYME_LOOSE_TYPES=1 cargo +enzyme test -p samples-loose-types
4 changes: 4 additions & 0 deletions .gitmodules
Original file line number Diff line number Diff line change
Expand Up @@ -43,3 +43,7 @@
path = library/backtrace
url = https://github.com/rust-lang/backtrace-rs.git
shallow = true
[submodule "src/tools/enzyme"]
path = src/tools/enzyme
url = https://github.com/enzymeAD/enzyme
shallow = true
2 changes: 2 additions & 0 deletions Cargo.lock
Original file line number Diff line number Diff line change
Expand Up @@ -4261,6 +4261,7 @@ dependencies = [
name = "rustc_monomorphize"
version = "0.0.0"
dependencies = [
"rustc_ast",
"rustc_data_structures",
"rustc_errors",
"rustc_fluent_macro",
Expand All @@ -4269,6 +4270,7 @@ dependencies = [
"rustc_middle",
"rustc_session",
"rustc_span",
"rustc_symbol_mangling",
"rustc_target",
"serde",
"serde_json",
Expand Down
59 changes: 57 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,64 @@
# The Rust Programming Language
# The Rust Programming Language + AutoDiff

[![Rust Community](https://img.shields.io/badge/Rust_Community%20-Join_us-brightgreen?style=plastic&logo=rust)](https://www.rust-lang.org/community)

This is the main source code repository for [Rust]. It contains the compiler,
standard library, and documentation.
standard library, and documentation. It is modified to use Enzyme for AutoDiff.


Please configure this fork using the following command:

```
mkdir build
cd build
../configure --enable-llvm-link-shared --enable-llvm-plugins --enable-llvm-enzyme --release-channel=nightly --enable-llvm-assertions --enable-clang --enable-lld --enable-option-checking --enable-ninja --disable-docs
```

Afterwards you can build rustc using:
```
../x.py build --stage 1 library
```

Afterwards rustc toolchain link will allow you to use it through cargo:
```
rustup toolchain link enzyme build/host/stage1
rustup toolchain install nightly # enables -Z unstable-options
```

You can then run an examples from our [docs](https://enzyme.mit.edu/index.fcgi/rust/usage/usage.html) using

```bash
cargo +enzyme run --release
```

## Enzyme Config
To help with debugging, Enzyme can be configured using environment variables.
```bash
export ENZYME_PRINT_TA=1
export ENZYME_PRINT_AA=1
export ENZYME_PRINT=1
export ENZYME_PRINT_MOD=1
export ENZYME_PRINT_MOD_AFTER=1
```
The first three will print TypeAnalysis, ActivityAnalysis and the llvm-ir on a function basis, respectively.
The last two variables will print the whole module directly before and after Enzyme differented the functions.

When experimenting with flags please make sure that EnzymeStrictAliasing=0
is not changed, since it is required for Enzyme to handle enums correctly.

## Bug reporting
Bugs are pretty much expected at this point of the development process.
In order to help us please minimize the Rust code as far as possible.
This tool might be a nicer helper: https://github.com/Nilstrieb/cargo-minimize
If you have some knowledge of LLVM-IR we also greatly appreciate it if you could help
us by compiling your minimized Rust code to LLVM-IR and reducing it further.

The only exception to this strategy is error based on "Can not deduce type of X",
where reducing your example will make it harder for us to understand the origin of the bug.
In this case please just try to inline all dependencies into a single crate or even file,
without deleting used code.



[Rust]: https://www.rust-lang.org/

Expand Down
6 changes: 6 additions & 0 deletions compiler/rustc_ast/src/ast.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2574,6 +2574,12 @@ impl FnRetTy {
FnRetTy::Ty(ty) => ty.span,
}
}
pub fn has_ret(&self) -> bool {
match self {
FnRetTy::Default(_) => false,
FnRetTy::Ty(_) => true,
}
}
}

#[derive(Clone, Copy, PartialEq, Encodable, Decodable, Debug)]
Expand Down
Loading
Loading