Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reorganize pipeline tests #963

Merged
merged 2 commits into from
Oct 24, 2022
Merged

Reorganize pipeline tests #963

merged 2 commits into from
Oct 24, 2022

Conversation

anton-l
Copy link
Member

@anton-l anton-l commented Oct 24, 2022

This PR sorts the pipeline-specific tests from test_pipelines.py into their individual directories.
Partially solves #913

Note: this doesn't touch the flax tests yet

Comment on lines +4 to +12
@require_torch
class PipelineTesterMixin:
"""
This mixin is designed to be used with unittest.TestCase classes.
It provides a set of common tests for each PyTorch pipeline, e.g. saving and loading the pipeline,
equivalence of dict and tuple outputs, etc.
"""

pass
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a dummy mixin for now (only @require_torch is useful), but the purpose is described in the docstring.

Comment on lines +4 to +12
@require_onnxruntime
class OnnxPipelineTesterMixin:
"""
This mixin is designed to be used with unittest.TestCase classes.
It provides a set of common tests for each ONNXRuntime pipeline, e.g. saving and loading the pipeline,
equivalence of dict and tuple outputs, etc.
"""

pass
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a dummy mixin for now (only @require_onnxruntime is useful), but the purpose is described in the docstring.

@HuggingFaceDocBuilderDev
Copy link

HuggingFaceDocBuilderDev commented Oct 24, 2022

The documentation is not available anymore as the PR was closed or merged.

Comment on lines +124 to +128
def require_onnxruntime(test_case):
"""
Decorator marking a test that requires onnxruntime. These tests are skipped when onnxruntime isn't installed.
"""
return unittest.skipUnless(is_onnx_available(), "test requires onnxruntime")(test_case)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now the ONNX tests can be skipped locally too, when onnxruntime is not installed.


torch.backends.cuda.matmul.allow_tf32 = False


class PipelineFastTests(unittest.TestCase):
class StableDiffusionPipelineFastTests(PipelineTesterMixin, unittest.TestCase):
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Making the class name a bit more verbose here, for easier search withing the test logs

@anton-l
Copy link
Member Author

anton-l commented Oct 24, 2022

Note to self: we'll need a cleaner dummy pipeline creation method, similar to prepare_config_and_inputs() in transformers. The dummy setup part duplicates a lot of code, especially in SD tests:

        unet = self.dummy_cond_unet
        scheduler = DDIMScheduler(
            beta_start=0.00085,
            beta_end=0.012,
            beta_schedule="scaled_linear",
            clip_sample=False,
            set_alpha_to_one=False,
        )

        vae = self.dummy_vae
        bert = self.dummy_text_encoder
        tokenizer = CLIPTokenizer.from_pretrained("hf-internal-testing/tiny-random-clip")

        # make sure here that pndm scheduler skips prk
        sd_pipe = StableDiffusionPipeline(
            unet=unet,
            scheduler=scheduler,
            vae=vae,
            text_encoder=bert,
            tokenizer=tokenizer,
            safety_checker=None,
            feature_extractor=self.dummy_extractor,
        )

Something like self.get_dummy_pipeline(scheduler_class=DDIMScheduler) might make sense to add.

Copy link
Contributor

@patrickvonplaten patrickvonplaten left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool, let's merge this maybe quickly to prevent merge conflicts :-)

@anton-l anton-l merged commit 2c82e0c into main Oct 24, 2022
@anton-l anton-l deleted the split-pipeline-tests branch October 24, 2022 16:10
PhaneeshB pushed a commit to nod-ai/diffusers that referenced this pull request Mar 1, 2023
exe should now default to dark theme too
yoonseokjin pushed a commit to yoonseokjin/diffusers that referenced this pull request Dec 25, 2023
* Reorganize pipeline tests

* fix vq
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants