ComfyUI-MotionCtrl

ComfyUI-MotionCtrl
★ 140

视频生成动作控制ComfyUI节点MotionCtrl集成
在ComfyUI中集成MotionCtrl节点，提供统一灵活的视频运动控制，便于生成可控动态视频，需加载motionctrl.pth权重。
💡 在ComfyUI流程中为视频生成添加精确运动控制。
🍴 7 Forks💻 Python🔄 2024-06-14
📦
网盘下载
复制链接后前往夸克网盘下载
https://pan.quark.cn/s/2df45d172dc1
📦 requirements.txt
Pytorch-Lightning==1.9.0
decord
kornia
timm
open_clip_torch
av
omegaconf
transformers
einops
scikit-learn
taming-transformers-rom1504
pandas
pandas
fairscale
psutil==5.9.5
annotated-types==0.5.0
plotly
imageio==2.14.1
imageio-ffmpeg==0.4.7
opencv-python==4.8.0.74
moviepy
Pillow
tqdm
gradio==3.37.0
📄 README

This is an implementation of MotionCtrl for ComfyUI

MotionCtrl: A Unified and Flexible Motion Controller for Video Generation

Install

  • Clone this repo into custom_nodes directory of ComfyUI location

  • Run pip install -r requirements.txt

  • Download the weights of MotionCtrl motionctrl.pth and put it to ComfyUI/models/checkpoints

    • Nodes

    Four nodes Load Motionctrl Checkpoint & Motionctrl Cond & Motionctrl Sample Simple & Load Motion Camera Preset & Load Motion Traj Preset & Select Image Indices &Motionctrl Sample

    Tools

    Motion Traj Tool Generate motion trajectories

    Motion Camera Tool Generate motion camera points

    Examples

    base workflow

    https://github.com/chaojie/ComfyUI-MotionCtrl/blob/main/workflow_motionctrl_base.json

    src=”assets/dog.mp4″

    type=”video/mp4″

    />

    unofficial implementation “MotionCtrl deployed on AnimateDiff” workflow:

    https://github.com/chaojie/ComfyUI-MotionCtrl/blob/main/workflow_motionctrl.json

  • Generate LVDM/VideoCrafter Video
  • Select Images->Scribble
  • Use AnimateDiff Scribble SparseCtrl