Hunyuan Lora in Comfy

[hashu786]

is there any plans to release some sort of a conversion script to make a Hunyuan lora trained with finetrainers compatible with comfy? Right now I just get a bunch of "lora key not loaded" errors.

[a-r-r-o-w]

Currently, Comfy does not support loading diffusers-format loras (conversion script to diffusers format from original is available here).

So, the only way to be able to load it is by writing a reverse script of above to go from diffusers format to original. I don't think I have the bandwidth to do so myself atm, so would be super cool if someone could help out with the script. I will try and find the time if no one picks it up

[neph1]

I've one for ltx loras, and I could probably adapt it to hunyuan as well (and make a PR for it in comfy). I've however not been able to run hunyuan locally yet, so I don't know what the issue is. Can you post some of the keys it complains about and I can try to make a blind fix. Alt, if there is a small lora I can test with.

[cseti007]

Hi @a-r-r-o-w and @neph1 !
As I see Kohya has a convert script from diffusers to default. Could this do it? https://github.com/kohya-ss/musubi-tuner/blob/main/convert_lora.py

[hashu786]

not my area of expertise but I created a quick convert script to work with comfy and a test example lora trained on all target modules. Seems to be working for me for now but I am sure lots of the script can be improved to be made better.

https://huggingface.co/hashu786/cine/tree/main

[a-r-r-o-w]

@cseti007 The Kohya one is not really a full conversion script, and I'm about 95% sure it shouldn't work OOTB with comfy that way.

FWIW, I wrote a partial conversion script here: huggingface/diffusers#10376 (comment)

It should account for most loras being trained but still requires handling a complete layer to make compatible with original format. I'll try to finish it up soon. Thanks for offering to help @neph1 - would be awesome if you can bring it to completion but I suspect it might require some debugging so please do not spend time on this unless purely for educational purposes or wanting to continue improving this 🤗