einops==0.4.1 imageio==2.33.0 imageio-ffmpeg==0.4.9 omegaconf==2.2.3 opencv-contrib-python==4.8.1.78 opencv-python==4.8.1.78 scikit-image==0.21.0 scikit-learn==1.3.2 transformers==4.33.1 moviepy==1.0.3 urllib3==1.26.9 torch torchdiffeq torchmetrics torchsde torchvision accelerate==0.29.3 av==11.0.0 clip @ https://github.com/openai/CLIP/archive/d50d76daa670286dd6cacf3bcd80b5e4823fc8e1.zip#sha256=b5842c25da441d6c581b53a5c60e0c2127ebafe0f746f8e15561a006c6c3be6a decord==0.6.0; sys_platform != 'darwin' and platform_machine != 'arm64' eva-decord==0.6.1; sys_platform == 'darwin' and platform_machine == 'arm64' diffusers>=0.24.0,<=0.27.2 open-clip-torch==2.20.0 xformers
MusePose is an image-to-video generation framework for virtual human under control signal such as pose.
MusePose is the last building block of the Muse opensource serie. Together with MuseV and MuseTalk, we hope the community can join us and march towards the vision where a virtual human can be generated end2end with native ability of full body movement and interaction. Please stay tuned for our next milestone!
If you’re running on Linux, or non-admin account on windows you’ll want to ensure /ComfyUI/custom_nodes and Comfyui-MusePose has write permissions.
Followed ComfyUI’s manual installation steps and do the following:
/ComfyUI/custom_nodes/ foldergit clone https://github.com/TMElyralab/Comfyui-MusePose.git/ComfyUI/custom_nodes/Comfyui-MusePose folder and run“`shell
pip install -r requirements.txt
pip install –no-cache-dir -U openmim
mim install mmengine
mim install “mmcv>=2.0.1”
mim install “mmdet>=3.1.0”
mim install “mmpose>=1.1.0”
“`
You can download weights manually as follows:
yolox_l_8x8_300e_coco.pthFinally, these weights should be organized in pretrained_weights as follows:
./pretrained_weights/
|-- MusePose
| |-- denoising_unet.pth
| |-- motion_module.pth
| |-- pose_guider.pth
| └── reference_unet.pth
|-- dwpose
| |-- dw-ll_ucoco_384.pth
| └── yolox_l_8x8_300e_coco.pth
|-- sd-image-variations-diffusers
| └── unet
| |-- config.json
| └── diffusion_pytorch_model.bin
|-- image_encoder
| |-- config.json
| └── pytorch_model.bin
└── sd-vae-ft-mse
|-- config.json
└── diffusion_pytorch_model.bin
https://github.com/TMElyralab/Comfyui-MusePose/blob/main/musepose-workflow-demo.json
https://github.com/TMElyralab/Comfyui-MusePose/assets/114042542/9cd8b9b8-6876-4281-b7a0-a7fbcb2de7e1