ComfyUI_Streamv2v_Plus

You can using Streamv2v/StreamDiffusion in comfyui

Streamv2v From: Streamv2v

StreamDiffusion From: StreamDiffusion

My ComfyUI node list：

---

1、ParlerTTS node:ComfyUI_ParlerTTS

2、Llama3_8B node:ComfyUI_Llama3_8B

3、HiDiffusion node：ComfyUI_HiDiffusion_Pro

4、ID_Animator node： ComfyUI_ID_Animator

5、StoryDiffusion node：ComfyUI_StoryDiffusion

6、Pops node：ComfyUI_Pops

7、stable-audio-open-1.0 node ：ComfyUI_StableAudio_Open

8、GLM4 node：ComfyUI_ChatGLM_API

9、CustomNet node：ComfyUI_CustomNet

10、Pipeline_Tool node :ComfyUI_Pipeline_Tool

11、Pic2Story node :ComfyUI_Pic2Story

12、PBR_Maker node:ComfyUI_PBR_Maker

13、ComfyUI_Streamv2v_Plus node:ComfyUI_Streamv2v_Plus

14、ComfyUI_MS_Diffusion node:ComfyUI_MS_Diffusion

15、ComfyUI_AnyDoor node: ComfyUI_AnyDoor

16、ComfyUI_Stable_Makeup node: ComfyUI_Stable_Makeup

17、ComfyUI_EchoMimic node: ComfyUI_EchoMimic

18、ComfyUI_FollowYourEmoji node: ComfyUI_FollowYourEmoji

1.Installation

---

In the ./ComfyUI /custom_node directory, run the following:

git clone https://github.com/smthemex/ComfyUI_Streamv2v_Plus.git

2.requirements

---

pip install -r requirements.txt

（如果peft报错）if get error about ：PEFT backend is required for this method

请按以下代码更新 PEFT和transformers

pip install -U PEFT transformers

tensorrt still has bug，need module below：

tensorrt 暂时无法使用，以下是需求库，可以不装

polygraphy

onnx_graphsurgeon

tensorrt

cuda-python

缺啥装啥。。。

If the module is missing, , pip install missing module.

3 Need model

---

3.1 base model

SDXL or sd1.5 pure单体模型，或者社区模型，部分模型没有vae会导致报错，可以在vae填入常规的vae repo，该地址栏默认是空，也就是使用社区模型内置的vae和encoder。 首次使用会下载config文件，注意连外网。。。

SDXL or sd1.5 pure monolithic model, or community model,

3.2 vae

====注意，使用SDXL或者XL turbo时，vae必须使用 madebyollin/sdxl-vae-fp16-fix

SD1.5可以不使用vae或者使用madebyollin/taesd

3.3 lcm lora lcm loras是必需的。

latent-consistency/lcm-lora-sdv1-5 SD1.5

latent-consistency/lcm-lora-sdxl SDXL or XL turbo

3.4 style lora

choice which you like 选一个你喜欢的风格lora，注意匹配底模。

when you changge a style lora，twigger word need change so. 改变风格lora时，关键词需要跟着变。

4 Function Description

—

4.1 txt2img 文生图

4.2 webcam2img/img2img 摄像头生图

4.3 video2video 视频转绘

5.example 示例

---

sd1.5 txt2im/img2img/webcam2img/video2video 选择菜单使用不同的功能。

SDXL（turbo 1 step，XL 4 step） SDXL示例，选择菜单使用不同的功能。

cam2video or cam2img need more vr 摄像头生图得看配置，低的跑的慢。

6 Citation

---

streamdiffusion

@article{kodaira2023streamdiffusion,
      title={StreamDiffusion: A Pipeline-level Solution for Real-time Interactive Generation},
      author={Akio Kodaira and Chenfeng Xu and Toshiki Hazama and Takanori Yoshimoto and Kohei Ohno and Shogo Mitsuhori and Soichi Sugano and Hanying Cho and Zhijian Liu and Kurt Keutzer},
      year={2023},
      eprint={2312.12491},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}

StreamingV2V

@article{liang2024looking,
  title={Looking Backward: Streaming Video-to-Video Translation with Feature Banks},
  author={Liang, Feng and Kodaira, Akio and Xu, Chenfeng and Tomizuka, Masayoshi and Keutzer, Kurt and Marculescu, Diana},
  journal={arXiv preprint arXiv:2405.15757},
  year={2024}
}