ComfyUI-DiffSynth-Studio

★ 100

ComfyUI扩展DiffSynth-Studio音视频合成环境依赖

将 DiffSynth-Studio 集成到 ComfyUI，能在节点流程中直接调用其合成能力，便于生成音视频并快速复现实验。

💡 在 ComfyUI 流程中调用 DiffSynth-Studio 做音视频合成与批量生成

🍴 15 Forks💻 Python🔄 2024-08-05

🔗 GitHub 原文

📦

网盘下载

复制链接后前往夸克网盘下载

https://pan.quark.cn/s/b6135d9bd930

📦 requirements.txt

cupy-cuda12x
pip
transformers
controlnet-aux==0.0.7
streamlit
streamlit-drawable-canvas
imageio
imageio[ffmpeg]
safetensors
einops
sentencepiece

📄 README

ComfyUI-DiffSynth-Studio

make DiffSynth-Studio avialbe in ComfyUI

how to use

test on py3.10，2080ti 11gb，torch==2.3.0+cu121

make sure ffmpeg is worked in your commandline

for Linux

apt update
apt install ffmpeg

for Windows,you can install ffmpeg by WingetUI automatically

then!

# in ComfyUI/custom_nodes
git clone https://github.com/AIFSH/ComfyUI-DiffSynth-Studio.git
cd ComfyUI-DiffSynth-Studio
pip install -r requirements.txt

weights will be downloaded from huggingface or model scope

Tutorial

Demo for Diffutoon DiffSynth-Studio!ComfyUI插件之Diffutoon节点-哔哩哔哩

Run on 4090

wechat：aifsh_98

Nodes Detail and Workflow

ExVideo Node

ExVideo workflow

"image":("IMAGE",),
"svd_base_model":("SD_MODEL_PATH",),
"exvideo_model":("SD_MODEL_PATH",),
"num_frames":("INT",{ 
    "default": 128
}),
"fps":("INT",{
    "default": 30
}),
"num_inference_steps":("INT",{
    "default": 50
}),
"if_upscale":("BOOLEAN",{
    "default": True,
}),
"seed": ("INT",{
    "default": 1
})

image synthesis

comming soon

Diffutoon Node

Diffutoon workflow

"required":{
    "source_video_path": ("VIDEO",),
    "sd_model_path":("SD_MODEL_PATH",),
    "postive_prompt":("TEXT",),
    "negative_prompt":("TEXT",),
    "start":("INT",{
        "default": 0 ## from which second of your video to be shaded
    }),
    "length":("INT",{
        "default": -1 ## how long you want to shade, -1 name the whole video frames
    }),
    "seed":("INT",{
        "default": 42 
    }),
    "cfg_scale":("INT",{
        "default": 3
    }),
    "num_inference_steps":("INT",{
        "default": 10
    }),
    "animatediff_batch_size":("INT",{
        "default": 32 ## lower it till you can run
    }),
    "animatediff_stride":("INT",{
        "default": 16 ## lower it till you can run
    }),
    "vram_limit_level":("INT",{
        "default": 0 ## meet killed? try to 1
    }),
},
"optional":{
    "controlnet1":("ControlNetConfigUnit",),
    "controlnet2":("ControlNetConfigUnit",),
    "controlnet3":("ControlNetConfigUnit",),
}

Video Stylization

Chinese Models

ask for answer as soon as you want

wechat: aifsh_98

need donate if you mand it,

but please feel free to new issue for answering

Windows环境配置太难？可以添加微信：aifsh_98，赞赏获取Windows一键包，当然你也可以提issue等待大佬为你答疑解惑。