ComfyUI-MotionCtrl-SVD

ComfyUI-MotionCtrl-SVD
★ 85

运动控制视频一致性模型权重ComfyUI节点
在ComfyUI中实现MotionCtrl-SVD，加载指定权重以提供基于SVD的运动控制，提升多帧/视频生成的一致性与时序控制。
💡 在ComfyUI中用SVD MotionCtrl提升多帧/视频生成一致性
🍴 5 Forks💻 Python🔄 2024-06-14
📦
网盘下载
复制链接后前往夸克网盘下载
https://pan.quark.cn/s/2df45d172dc1
📦 requirements.txt
Pytorch-Lightning==1.9.0
decord
kornia
timm
open_clip_torch
av
omegaconf
transformers
einops
scikit-learn
taming-transformers-rom1504
pandas
fairscale
psutil==5.9.5
annotated-types==0.5.0
plotly
imageio==2.14.1
imageio-ffmpeg==0.4.7
opencv-python==4.8.0.74
moviepy
Pillow
tqdm
webdataset
Fire
natsort
wandb
clip
@
git+https://github.com/openai/CLIP.git
📄 README

This is an implementation of ComfyUI MotionCtrl for SVD

MotionCtrl for SVD

Install

  • Clone this repo into custom_nodes directory of ComfyUI location

  • Run pip install -r requirements.txt

  • Download the weights of MotionCtrl for SVD motionctrl_svd.ckpt and put it to ComfyUI/models/checkpoints

    • Examples

    base workflow

    https://github.com/chaojie/ComfyUI-MotionCtrl-SVD/blob/main/workflow.json

    A little exploration workflow: for videos with relatively static camera angles, applying the same MotionCtrl to each frame, then combining images from corresponding positions to create a new video. The nth frame image is taken from the generated nth frame and merged again.

    https://github.com/chaojie/ComfyUI-MotionCtrl-SVD/blob/main/workflow_video.json

    original video: https://github.com/chaojie/ComfyUI-MotionCtrl-SVD/blob/main/assets/original.mp4

    generate video: https://github.com/chaojie/ComfyUI-MotionCtrl-SVD/blob/main/assets/svd.mp4