ComfyUI_CSGO_Wrapper

★ 17

风格迁移CSGO 集成SDXL 解码ControlNet 支持

在 ComfyUI 中封装 InstantX 的 CSGO，方便调用其风格模型，简化 SDXL 图像解码与 ControlNet 集成，修复兼容性问题。

💡 在 ComfyUI 中应用 CSGO 风格并集成 SDXL/ControlNet

🍴 2 Forks💻 Python🔄 2024-09-07

🔗 GitHub 原文

📦

网盘下载

复制链接后前往夸克网盘下载

https://pan.quark.cn/s/c1eafc754fbb

📦 requirements.txt

#diffusers==0.25.1
#torch==2.0.1
#torchaudio==2.0.2
#torchvision==0.15.2
#transformers==4.40.2
#accelerate
#safetensors
#einops
#spaces==0.19.4
#omegaconf
#peft
#huggingface-hub==0.24.5
#opencv-python
#insightface
#controlnet_aux
#gdown

📄 README

ComfyUI_CSGO_Wrapper

You can using InstantX’s CSGO in comfyUI

CSGO From: link

Update

2024/09/07

fix runway loader error, using single clip_vision /controlnet weight now；

修复runway跑路导致的diffuser加载报错，现在直接使用IP adapter的SDXL 图片解码单体模型（请放在clip_vision目录下）和单体controlnet SDLX模型（请放在controlnet目录下）；

vae is not necessary,if you get black img ,please using vae

vae并不是必须的，但是如果出现黑图或者解码不正确，才使用；

1.Installation

In the ./ComfyUI /custom_node directory, run the following:

git clone https://github.com/smthemex/ComfyUI_CSGO_Wrapper.git

2.requirements

For ComfyUI users, all libraries in the requirements file should be available. If not, please uncomment the # and reinstall

pip install -r requirements.txt

3 Need model

3.1 base SDXl ckpt and vae and clip_vision and controlnet

h94/IP-Adapter/sdxl_models link

TTPlanet/TTPLanet_SDXL_Controlnet_Tile_Realistic link

no need config

├── ComfyUI/models/checkpoints/
|      ├──any SDXL weights  # for example: Jumpernaut XL_v9-RunDiffusionPhoto_v2.safetensors
├── ComfyUI/models/vae/
|      ├──any SDXL vae weights  # sdxl.vae.safetensors 
├── ComfyUI/models/clip_vision/
|      ├──model.safetensors  # h94/IP-Adapter/sdxl_models/model.safetensors
├── ComfyUI/models/controlnet /
|      ├──TTPLANET_Controlnet_Tile_realistic_v2_fp16.safetensors  # TTPlanet/TTPLanet_SDXL_Controlnet_Tile_Realistic

3.2 main ckpt

CSGO models link

├── ComfyUI/models/checkpoints/
|      ├──acsgo_4_32.bin  #need token 4/32
|      ├──acsgo.bin   #need token 4/16

3.3 if using LLM (unnecessary)

Salesforce/blip-image-captioning-large link

├── any path dir/
|             ├── model.safetensors
|             ├── config.json
|             ├── preprocessor_config.json
|             ├── special_tokens_map.json
|             ├── tokenizer.json
|             ├── tokenizer_config.json
|             ├── vocab.txt

or change prompt input using any other LLM.

4 Example

conternt+style img

style img + prompt

conternt+style img +llm new 最新

5 Function Description of Nodes

content_tokens: keep in 4;

style_tokens: if using csgo.bin keep num “16”,if using csgo_4_32.bin keep num “32” ;

text_only: if not using content img ;

Blip is not necessary;

6 Citation

CSGO

@article{xing2024csgo,
       title={CSGO: Content-Style Composition in Text-to-Image Generation}, 
       author={Peng Xing and Haofan Wang and Yanpeng Sun and Qixun Wang and Xu Bai and Hao Ai and Renyuan Huang and Zechao Li},
       year={2024},
       journal = {arXiv 2408.16766},
}
}