[WIP][tests] add precomputation tests #234
Conversation
| try: | ||
| tokenizer = T5Tokenizer.from_pretrained(model_id, subfolder="tokenizer", revision=revision, cache_dir=cache_dir) | ||
| except: | ||
| tokenizer = AutoTokenizer.from_pretrained(model_id, subfolder="tokenizer", revision=revision, cache_dir=cache_dir) |
There was a problem hiding this comment.
Not super proud of this but we cannot do T5Tokenizer on the dummy T5 tokenizer checkpoint. Some sentencepiece error.
| try: | ||
| tokenizer = T5Tokenizer.from_pretrained( | ||
| model_id, subfolder="tokenizer", revision=revision, cache_dir=cache_dir | ||
| ) | ||
| except: # noqa | ||
| tokenizer = AutoTokenizer.from_pretrained( | ||
| model_id, subfolder="tokenizer", revision=revision, cache_dir=cache_dir | ||
| ) |
There was a problem hiding this comment.
Not proud of the change but T5Tokenizer cannot be used on a dummy T5 tokenizer ckpt.
|
@a-r-r-o-w LMK what you think of the latest changes. |
Co-authored-by: a-r-r-o-w <[email protected]>
|
@a-r-r-o-w LMK if I can apply the tests to the rest of the models. I have taken care of addressing the rest of your feedback. |
|
Yes please, lgtm |
|
@a-r-r-o-w I have now completed an initial test suite for precomputation for all supported models for T2V, an essential part of The three tests are:
I believe this should be sufficient for now but LMK. I haven't yet configured the runner for CI. I want to do that after another round of review and it should be good to go. |
|
@a-r-r-o-w a gentle ping. |
|
Missed the email update on previous message, so apologies for delay. Thanks for working on improving this a lot. Will test and review very soon. The test suite will be super helpful for #245 as well 🤗 |
Adds precomputation tests.
Currently, I have changed the bare-minimum to show the approach taken for the tests. After I have some reviews, I will propagate the changes to the rest of the supported models and make the PR ready for further reviews.
Some further comments in-line.
To run the tests from DGX or any other internal CUDA machines without using CUDA, run:
CUDA_VISIBLE_DEVICES="" pytest tests/trainers/Just LMK if you want something change before proceeding to review at this stage of the PR. I will make it happen.
TODOs