StableDiffusionXLPipeline does not work when COMPEL=1

[ljcleo]

LocalAI version:
quay.io/go-skynet/local-ai:latest

Environment, CPU architecture, OS, and Version:
OS: Ubuntu 16.04 LTS
CPU architecture : x86_64
Linux dbcloud 4.4.0-142-generic #168-Ubuntu SMP Wed Jan 16 21:00:45 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Describe the bug
StableDiffusionXLPipeline for the Diffusers backend does not work when the environment variable COMPEL=1 (or not set). Instead, it returns:

{"error":{"code":500,"message":"rpc error: code = Unknown desc = Exception calling application: If `prompt_embeds` are provided, `pooled_prompt_embeds` also have to be passed. Make sure to generate `pooled_prompt_embeds` from the same text encoder that was used to generate `prompt_embeds`.","type":""}}

To Reproduce
Simply follow the Linaqruf/animagine-xl example in the doc (https://localai.io/model-compatibility/diffusers/#model-setup).

Expected behavior

Logs

Additional context
Looks like the problem comes from #904 where Compel is introduced to handle long prompts. The Diffusers API for SDXL requires both prompt_embeds and pooled_prompt_embeds (maybe they'll change that in the future, see huggingface/diffusers#4341), yet the code doesn't cover this scenario.

By the way, it would be better if the .env file contains an option for the environment variable COMPEL. The above issues could really confuse someone new to LocalAI (like me) when they found that the official example does not work🤔

[lunamidori5]

@mudler I am also having this bug is there a fix?

[lianee]

same error trying to follow the StableDiffusionXLPipeline example

[lunamidori5]

same error trying to follow the StableDiffusionXLPipeline example

@lianee Welp, thats a oops, so in your env you need to add COMPEL=0 then it will work, Ill push a update to the how to tonight to make that more clear

[lianee]

@lunamidori5 I use the docker image, and with -e COMPEL=0 in the command line this works
I had tried this before, but had dumbly put this parameter at the end of the command line. Thanks for making me try again.

[mudler]

I've updated usage of Compel within #1746 with XL models