ComfyUI_MagicClothing

★ 599

服装生成服饰合成人脸识别(IPAdapter)OpenPose/AnimateDiff

在ComfyUI中实现MagicClothing，基于服饰与文本提示生成并合成服装图像，支持人脸识别(IPAdapter)、OpenPose控制、分部位模型与AnimateDiff动画与修补。

💡 将服装模板根据姿态与面部信息合成到人物图像或制作服装动画

🍴 45 Forks💻 Python🔄 2024-09-04

🔗 GitHub 原文

📦

网盘下载

复制链接后前往夸克网盘下载

https://pan.quark.cn/s/b6135d9bd930

📦 requirements.txt

insightface==0.7.3
onnxruntime-gpu==1.17.1
--extra-index-url
https://download.pytorch.org/whl/cu118
torch==2.1.1+cu118
torchvision==0.16.1+cu118
numpy==1.25.1
diffusers==0.26.2
opencv-python==4.9.0.80
transformers==4.31.0
safetensors==0.3.1
controlnet-aux==0.0.6
accelerate==0.21.0

📄 README

Updates:

✅ [2024/04/17] only cloth image with prompt generation

✅ [2024/04/18] IPAdapter FaceID with human face detection and synthesize with cloth image generation

✅ [2024/04/18] IPAdapter FaceID with controlnet openpose and synthesize with cloth image generation

✅ [2024/04/19] lower-body and full-body models for preliminary experiment

✅ [2024/04/26] AnimateDiff and cloth inpaiting have been supported

U can contact me thr twitter Weixin：GalaticKing

the main workflow

IPAdapater FaceID workflow

IPAdapater FaceID chained with controlnet openpose workflow

lower-body full-body workflow

full-body workflow with IPadapterFaceid

cloth inpainting workflow

AnimateDiff workflow

you should run under custom_nodes directory of ComfyUI

git clone https://github.com/frankchieng/ComfyUI_MagicClothing.git

then run

pip install -r requirements.txt

download the models of cloth_segm.pth , magic_clothing_768_vitonhd_joint.safetensors(upper-body model),OMS_1024_VTHD+DressCode_200000.safetensors(lower-body and full-body model) from

🤗Huggingface and place them at the checkpoints directory, If u wanna to run AnimateDiff you should place garment_extractor.safetensors and ip_layer.pth in checkpoints/stable_ckpt directory

you should try the combination of miscellaneous hyperparameters especially when you inference with the lower and full body model,just for experiment now

install the ComfyUI_IPAdapter_plus custom node at first if you wanna to experience the ipadapterfaceid.Then download the IPAdapter FaceID models from IP-Adapter-FaceID and place them as the following placement structure

For cloth inpainting, i just installed the Segment anything node,you can utilize other SOTA model to seg out the cloth from background.

##### tips:If you wanna to run the controlnet openpose part,you have to install the comfyui_controlnet_aux custome code as well as download the body_pose_model.pth, facenet.pth and hand_pose_model.pth at openpose models and place them in custom_nodes/comfyui_controlnet_aux/ckpts/lllyasviel/Annotators

ComfyUI
|-- models
|   |-- ipadapter
|   |   |-- ip-adapter-faceid-plus_sd15.bin
|   |   |-- ip-adapter-faceid-plusv2_sd15.bin
|   |   |-- ip-adapter-faceid_sd15.bin
|   |-- loras
|   |   |-- ip-adapter-faceid-plus_sd15_lora.safetensors
|   |   |-- ip-adapter-faceid-plusv2_sd15_lora.safetensors
|   |   |-- ip-adapter-faceid_sd15_lora.safetensors
|-- custom_nodes
|   |-- ComfyUI_MagicClothing
|   |   |-- checkpoints
|   |   |   |-- cloth_segm.pth
|   |   |   |-- magic_clothing_768_vitonhd_joint.safetensors
|   |   |   |-- OMS_1024_VTHD+DressCode_200000.safetensors
|   |   |   |-- stable_ckpt
|   |   |   |   |-- garment_extractor.safetensors
|   |   |   |   |-- ip_layer.pth