ComfyUI_Dreamtalk

ComfyUI_Dreamtalk
★ 12

语音合成口型同步视频处理ComfyUI插件
ComfyUI_Dreamtalk 将 a/dreamtalk 非官方实现整合到 ComfyUI，可在节点流程中生成与处理语音/口型同步的视频或音频，方便一键集成与批量处理。
💡 在 ComfyUI 中使用 ComfyUI_Dreamtalk 生成语音/口型同步视频
🍴 6 Forks💻 Python🔄 2024-08-15
📦
网盘下载
复制链接后前往夸克网盘下载
https://pan.quark.cn/s/b6135d9bd930
📦 requirements.txt
yacs
scipy
scikit-image
scikit-learn
PyYAML
Pillow
numpy
opencv-python
imageio
ffmpeg-python
av
torch
torchvision
torchaudio
urllib3
transformers
dlib
huggingface_hub
📄 README

ComfyUI Dreamtalk (Unofficial Support)

Unofficial Dreamtalk support for ComfyUI.

Important Updates

  • 2024/03/29: Added installation from ComfyUI Manager
  • 2024/03/28: Added ComfyUI nodes and workflow examples

    • Basic Workflow

    This workflow shows the basic usage on making an image into a talking face video.

    Advanced Workflow

    This workflow shows the advanced usage on making the whole image talking, instead of just a cropped face.

    Installation

  • Install from ComfyUI Manager (search for dreamtalk, make sure ffmpeg is installed)

  • Download or git clone this repository into the ComfyUI/custom_nodes/ directory and run:
    • sudo apt install ffmpeg
pip install -r requirements.txt

    Download Checkpoints

    Put the downloaded denoising_network.pth and renderer.pt into checkpoints folder.

    Above two files can be requested from the original author as follows:

    In light of the social impact, we have ceased public download access to checkpoints. If you want to obtain the checkpoints, please request it by emailing mayf18@mails.tsinghua.edu.cn . It is important to note that sending this email implies your consent to use the provided method solely for academic research purposes.