feat(docs): add half-precision training section in using_simulator docs #678
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
PabloCarmona
added
documentation
jubueche
previously approved these changes
Aug 12, 2024
PabloCarmona
force-pushed
the
feat-677-half-precision-docs
branch
from
e017345 to
0877c23
Compare
|
@coreylammie @anu-pub Take a look at it and let me know what you think, thanks! |
|
Compilation is currently not working for the half precision type. To reproduce: Error: |
|
@maljoras I think this is something that you could maybe easily fix. Or you could tell me how I could resolve it. |
maljoras
reviewed
Aug 25, 2024
| dataset = datasets.MNIST("data", train=True, download=True, transform=transform) | ||
| train_loader = torch.utils.data.DataLoader(dataset, batch_size=32) | ||
|
|
||
| model = model.to(device=device, dtype=torch.bfloat16) |
There was a problem hiding this comment.
Does this also work with other tiles that TorchInference? I had added a HALF type, like in the above: rpu_config.runtime.data_type = RPUDataType.HALF
because the HALF could either be bfloat16 of float16 depending on the compilation options.
There was a problem hiding this comment.
There also a runtime.data_type.as_torch() or something function to convert it to the corresponding torch type, see https://github.com/IBM/aihwkit/blob/master/src/aihwkit/simulator/parameters/enums.py#L23
|
Yes, one has to enable the FP16 auto-conversion during the compilation if I remember correctly. You have to make sure that |
|
You could try to set additionally |
|
You might also want to use bfloat as default FP16 type (with |
|
Actually, could also be related to the rather old compiler you are using? One way to go about it might to change each conversion |
|
I suspect the compiler version to be the problem. Do you know what compiler version worked? On the system I using to check this it is not working. Regarding the other things you mentioned: I tried |
|
@jubueche have you tried a different compiler version? what is the status on this testing? |
|
Tested now in my local env with enabled CUDA GPU and CPU and both making the example working correctly. Please check if everything seems correct on the example and on the docs are clear, thanks! |
Signed-off-by: Pablo Carmona Gonzalez <[email protected]> Signed-off-by: Pablo Carmona González <[email protected]>
Signed-off-by: Pablo Carmona Gonzalez <[email protected]> Signed-off-by: Pablo Carmona González <[email protected]>
* Draft fix * Remove DS Store * Ran mypy and pylint and passing * add ignore on unused ctx params temporarly for linting pass --------- Co-authored-by: Pablo Carmona Gonzalez <[email protected]> Signed-off-by: Pablo Carmona González <[email protected]>
Signed-off-by: Pablo Carmona González <[email protected]>
* add release content for 0.9.2 Signed-off-by: Pablo Carmona <[email protected]> * remove added patch text Signed-off-by: Pablo Carmona <[email protected]> * modify changelog.md and version Signed-off-by: Pablo Carmona <[email protected]> * ufix(examples/31_custom_drift_models.py): change way rpu config is setup to a cleaner way Signed-off-by: Pablo Carmona <[email protected]> * fix(examples/31_custom_drift_models.py): fix errors with pycodestyle Signed-off-by: Pablo Carmona <[email protected]> * feat(notebooks): clean up and enhacenments on notebook contents for clarification and typos fixings * feat(converter/conductance): new conductance converters development * updates to custom drift model suggested by Malte (#690) Co-authored-by: Charles Mackin <[email protected]> * Update hermes.py (#691) * fix(hermes.py): fix linting errors related to whitespacing * fix(requirements.txt): solve problem with numpy versions in some cases * feat(CHANGELOG.md): add PR number for new related developments and modify the date of release to a proper one * feat(CHANGELOG.md): add PR number for new related developments and modify the date of release to a proper one * feat(travis): update travis build with latest pytorch version --------- Signed-off-by: Pablo Carmona <[email protected]> Co-authored-by: charlesmackin <[email protected]> Co-authored-by: Charles Mackin <[email protected]> Co-authored-by: Athanasios Vasilopoulos <[email protected]> Signed-off-by: Pablo Carmona González <[email protected]>
* add release content for 0.9.2 * feat(notebooks/tutorial): add notebook outputs for reference * fix(pcm.py): add a check to not get custom_drift_model as None * fix(travis/build): fix proper version for pytorch on mac osx wheel build * fix(travis/build): fix torch version to match compatibility with osx version Signed-off-by: Pablo Carmona González <[email protected]>
* add release content for 0.9.2 * feat(notebooks/tutorial): add notebook outputs for reference * fix(pcm.py): add a check to not get custom_drift_model as None * fix(travis/build): fix proper version for pytorch on mac osx wheel build * fix(travis/build): fix torch version to match compatibility with osx version * feat(docs): update advance install docs to match different GPU versions Signed-off-by: Pablo Carmona González <[email protected]>
* add release content for 0.9.2 * feat(notebooks/tutorial): add notebook outputs for reference * fix(pcm.py): add a check to not get custom_drift_model as None * fix(travis/build): fix proper version for pytorch on mac osx wheel build * fix(travis/build): fix torch version to match compatibility with osx version * feat(license): change LICENSE.txt and license references to MIT License * fix(array.py): add temporal ignore to bypass pylint * fix: pycodestyle errors Signed-off-by: Pablo Carmona González <[email protected]>
* Initital changes. Signed-off-by: Julian Buechel <[email protected]> * Executed notebook Signed-off-by: Julian Buechel <[email protected]> --------- Signed-off-by: Julian Buechel <[email protected]> Signed-off-by: Pablo Carmona González <[email protected]>
* passing code checkers Signed-off-by: Charles Mackin <[email protected]> * minor updates Signed-off-by: Charles Mackin <[email protected]> --------- Signed-off-by: Charles Mackin <[email protected]> Co-authored-by: Charles Mackin <[email protected]> Signed-off-by: Pablo Carmona González <[email protected]>
* passing code checkers Signed-off-by: Charles Mackin <[email protected]> * minor updates Signed-off-by: Charles Mackin <[email protected]> * updated examples README.md Signed-off-by: Charles Mackin <[email protected]> --------- Signed-off-by: Charles Mackin <[email protected]> Co-authored-by: Charles Mackin <[email protected]> Signed-off-by: Pablo Carmona González <[email protected]>
Signed-off-by: Julian Buechel <[email protected]> Signed-off-by: Pablo Carmona González <[email protected]>
* Added Notebook on Device Non_Idealities Signed-off-by: Ghazi Sarwat Syed <[email protected]> * Added the updated tutorial Signed-off-by: Ghazi Sarwat Syed <[email protected]> * Add files via upload Signed-off-by: Ghazi Sarwat Syed <[email protected]> * Cleanup of the notebooks directory. (#444) Signed-off-by: Ghazi Sarwat Syed <[email protected]> * fix typo (#445) Signed-off-by: Ghazi Sarwat Syed <[email protected]> * [ImgBot] Optimize images (#448) *Total -- 450.13kb -> 41.19kb (90.85%) /examples/img/replay_fake_images_gan.gif -- 423.50kb -> 24.00kb (94.33%) /notebooks/examples/imgs/xbar.png -- 26.62kb -> 17.19kb (35.44%) Signed-off-by: ImgBotApp <[email protected]> Signed-off-by: ImgBotApp <[email protected]> Co-authored-by: ImgBotApp <[email protected]> Signed-off-by: Ghazi Sarwat Syed <[email protected]> * for pr 437 * ghs-tutorial * changed to non executable code * change code to markdown * trying again with markdown changes --------- Signed-off-by: Ghazi Sarwat Syed <[email protected]> Signed-off-by: ImgBotApp <[email protected]> Co-authored-by: Kaoutar El Maghraoui <[email protected]> Co-authored-by: Malte J. Rasch <[email protected]> Co-authored-by: imgbot[bot] <31301654+imgbot[bot]@users.noreply.github.com> Co-authored-by: ImgBotApp <[email protected]> Co-authored-by: Borja Godoy Gago <[email protected]> Co-authored-by: pablocarmona <[email protected]> Signed-off-by: Pablo Carmona González <[email protected]>
Signed-off-by: Pablo Carmona González <[email protected]>
PabloCarmona
force-pushed
the
feat-677-half-precision-docs
branch
from
1b0197d to
fd3e438
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Related issues
#623
#677
Description