Dreambooth Prior Preservation Loss Example

[ghpkishore]

What API design would you like to have changed or added to the library? Why?
What happens when a not safe for work image is generated by Stable Diffusion when I am trying to use it to create class images? It returns a black image. I want to know how does this Image affect the training process. Are the black images used for training purposes or are they discarded?
What use case would this enable or better enable? Can you give us a code example?
If a NSFW image is generated by SD during Dreambooth training, auto discard the image and generate more images to ensure that the total number of images are kept constant. I would not want to see "Potential NSFW content was detected in one or more images. A black image will be returned instead. Try again with a different prompt and/or seed." when I am 94% in generating class images.

Also, I do not see why safety feature needs to be active for dreambooth. Shouldn't safety feature be default OFF?

[patrickvonplaten]

Hey @ghpkishore,

Regarding whether it should be turned off or on there was a longer discussion here #254 (comment)

In general, we never use the safety checker for training, but we might very well use it for evaluation (when evaluating the model in between training epochs).

Does this make sense?

Also cc @patil-suraj in case you have more/better information :-)

[ghpkishore]

I get your point on not using the safety checker for training, but generation of images is part of the training process in dreambooth. So is this ( the safety checker being active) a bug in the code right now? Will read on the mentioned issue as well.

[patil-suraj]

As Patrick said, it's not necessary to use safety checker during training for generating prior images.
This is now fixed in #922

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.