[Cog] some minor fixes and nits

[sayakpaul]

What does this PR do?

• Cleans up check_inputs()
• Moves the generator check in prepare_latents() at the beginning of the method so that we can error as early as possible.
• Documentation nits.

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[a-r-r-o-w]

Oops 😬 So, we don't have a test that passes both prompt embeds and negative prompt embeds? I believe that would have caught this, no? Surprising and very lucky that it was working for the normal prompt inputs so far.
Apologies for missing this on my part

[sayakpaul]

So, we don't have a test that passes both prompt embeds and negative prompt embeds? I believe that would have caught this, no?

I believe that could be added but the test would be about checking if an error is thrown if prompt and prompt_embeds are both passed and other adjacent areas.

[sayakpaul]

Sorry, I have additional updates to make. Will let you all know here after that.

[sayakpaul]

We don't need it because height and width are already at their default values.

[sayakpaul]

Is beneficial for scenarios where we want to run the pipeline without the VAE.

[a-r-r-o-w]

Looks good! Thanks for the fixes

[sayakpaul]

Adding this so that a pipeline can operate with the vae.

[yiyixuxu]

thanks!

[FurkanGozukara]

thank you for amazing work

i am using the code that is running here : https://huggingface.co/spaces/THUDM/CogVideoX-5B-Space

however I am getting below errors both in text to video

and also video to video

could there have been something broken the pipe?

pipe = CogVideoXPipeline.from_pretrained("THUDM/CogVideoX-5b", torch_dtype=default_dtype).to("cpu")
pipe.scheduler = CogVideoXDPMScheduler.from_config(pipe.scheduler.config, timestep_spacing="trailing")

    else:
        pipe.to(device)
        pipe.transformer = transformer
        pipe.vae = vae
        pipe.text_encoder = text_encoder

        if use_cpu_offload:
            pipe.enable_sequential_cpu_offload()
        if use_slicing:
            pipe.vae.enable_slicing()
        if use_tiling:
            pipe.vae.enable_tiling()

        video_pt = pipe(
            prompt=prompt,
            num_videos_per_prompt=1,
            num_inference_steps=num_inference_steps,
            num_frames=49,
            use_dynamic_cfg=True,
            output_type="pt",
            guidance_scale=guidance_scale,
            generator=torch.Generator(device="cpu").manual_seed(seed),
        ).frames
        gc.collect()
    return (video_pt, seed)

[sayakpaul]

Open a separate issue with a minimal fully reproducible code snippet. https://github.com/user-attachments/files/17161626/app.txt is quite long and is not minimal and the snippet you provided is incomplete.

[FurkanGozukara]

@sayakpaul current diffusers doesn't have this?
CogVideoXLoraLoaderMixin

it can't find it installed latest one on Windows

0.30.3

[a-r-r-o-w]

v0.30.3 only released the image-to-video and video-to-video pipelines. LoRA support will be added in a future release. For now, you can use the lora loader mixin via the main branch