It’s probably Stable diffusion. I use comfyui since you can watch the sausage get made but there’s also other UIs like automatic1111. Originally for a qr pattern beautifier, there is a controlnet that takes a two tone black and white “guide” image. but you can guide it to follow any image you feed it. Such as a meme edited to be black and white, or text like “GAY SEX.”
This is done by combining a Diffusion model with ControlNet interface. As long as you have a decently modern Nvidia GPU and familiarity with Python and Pytorch it’s relatively simple to create your own model.
I implemented this paper back in March. It’s as simple as it is brilliant. By using methods originally intended to adapt large pre-trained language models to a specific application, the author’s created a new model architecture that can better control the output of a diffusion model.
any link to the site you using
Would like to know that as well. I just stole the meme from non-fediverse meme site
It’s probably Stable diffusion. I use comfyui since you can watch the sausage get made but there’s also other UIs like automatic1111. Originally for a qr pattern beautifier, there is a controlnet that takes a two tone black and white “guide” image. but you can guide it to follow any image you feed it. Such as a meme edited to be black and white, or text like “GAY SEX.”
Looks like somebody created an outline mask and then used that in img2img with a prompt for the particular scenery.
I remember seeing somebody use that technique to generate the same pose and model but different colored outfits with that technique
I’ve used InvokeAI (stable diffusion models) with ControlNets for it too, it’s a bit easier to use than comfy/a1111 but not as powerful IMHO.
There’s also SD.Next with some enhancements to automatic1111.
This is done by combining a Diffusion model with ControlNet interface. As long as you have a decently modern Nvidia GPU and familiarity with Python and Pytorch it’s relatively simple to create your own model.
The ControlNet paper is here: https://arxiv.org/pdf/2302.05543.pdf
I implemented this paper back in March. It’s as simple as it is brilliant. By using methods originally intended to adapt large pre-trained language models to a specific application, the author’s created a new model architecture that can better control the output of a diffusion model.
Check this out:
https://github.com/camenduru/controlnet-colab
I started looking after you responded. Havnt gotten it to work yet.