ComfyUI-Flowty-CRM

ComfyUI-Flowty-CRM
★ 156

ComfyUICRM单图像到3D模型集成
在ComfyUI中集成Convolutional Reconstruction Models（CRM），可将单张图片通过前馈高保真模型生成3D重建，便于在工作流中快速部署与可视化。
💡 在ComfyUI工作流中快速把单张图片生成高保真3D模型并导出。
🍴 10 Forks💻 Python🔄 2024-06-14
📦
网盘下载
复制链接后前往夸克网盘下载
https://pan.quark.cn/s/2df45d172dc1
📦 requirements.txt
huggingface-hub
diffusers
Pillow
einops
open-clip-torch
omegaconf
trimesh
ninja
example
📄 README

ComfyUI-Flowty-CRM

This is a custom node that lets you use Convolutional Reconstruction Models right from ComfyUI.

CRM is a high-fidelity feed-forward single image-to-3D generative model.

This node has been adapted from the official implementation with many improvements that make it easier to use and production ready:

  • Added support for cpu generation (initially could only run on cuda)
  • xformers is no longer required
  • separation of the inference pipeline into 3 parts, which enables it to run on low vram gpus and gives users freedom to alter the inputs at any step
  • automatic input image processing
  • high speed and high quality outputs with materials in glb and obj formats
  • closely integrated with comfyui’s internal attention code
  • cleaner code, with less dependencies and less memory usage

    • Installation:

  • Install ComfyUI
  • Clone this repo into “custom_nodes“:

    • “`shell

    $ cd ComfyUI/custom_nodes

    $ git clone https://github.com/flowtyone/ComfyUI-Flowty-CRM.git

    “`

  • Install dependencies:

    • “`shell

    $ cd ComfyUI-Flowty-CRM

    $ pip install -r requirements.txt

    “`

    if using cuda, install cuda dependencies too:

    “`shell

    $ pip install -r requirements-cuda.txt

    “`

  • Download CRM models and place them in “ComfyUI/models/checkpoints“
  • Start ComfyUI (or restart)
  • Use the example workflow (cpu | cuda)
  • Note: To run the example workflows you will also need to install ComfyUI_essentials

    • Acknowledgement:

  • Researchers of CRM: Zhengyi Wang, Yikai Wang, Yifei Chen, Chendong Xiang,

    • Shuo Chen, Dajiang Yu, Chongxuan Li, Hang Su, Jun Zhu

  • three.js comfyui interface: MrForExample
  • snr depth reconstruction: Imagedream

    • This is a community project from flowt.ai. If you like it, check us out!

    Troubleshooting

  • NVdiffrast
  • Windows users with cuda(nvidia) gpus might need some special preparations to make the nvdiffrast module work. Please check the official documentation of nvdiffrast for windows installation instructions.
  • CUDA_HOME environment variable is not set – you will need to set an environment variable that points to your cuda installation path. See this stackoverflow answer.
  • If you’re experiencing too many issues trying to install NVdiffrast, consider using the cpu workflow by restarting comfyui with the cpu-only option (much slower).
  • Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!
  • This usually happens if you tried to run the cpu workflow but have a cuda gpu. Try to restart comfyui and run only the cuda workflow.
  • Portable ComfyUI
  • Users might need to install the dependencies differently, see here.
  • Not enough VRAM/RAM
  • Using these nodes you should be able to run CRM on GPUs with 8GB of VRAM and above, and at least 16GB of RAM.
  • For GPUs with less than 16GB of VRAM, you might need to split the workflow into 4 parts and run them separately. I’ve added workflows here for demonstration.