ComfyUI-IF_VideoPrompts

A ComfyUI extension that provides video sequence analysis and prompting using advanced multimodal LLMs. This extension uses the Qwen2.5-VL models from Alibaba to analyze video sequences and generate detailed descriptions.

Important Requirements

This extension requires transformers 4.49.0 or above to work properly. Earlier versions (including 4.48.0) will cause errors.

Features

Video Frame Analysis: Analyze a sequence of video frames loaded via the Video Helper Suite nodes

Direct Video File Processing: Process MP4 and other video files directly without pre-loading frames

Multiple Analysis Types:

Full sequence narratives

Key scene breakdowns

Single summaries

Language Support: English and Chinese output

Customizable Prompting: Define your own system prompts or use provided presets

Negative Prompt Generation: Generate negative prompts for video content

Installation

Method 1: Using the installation script (Recommended)

This method handles dependency conflicts automatically:

Clone this repository into your ComfyUI custom_nodes directory:

cd ComfyUI/custom_nodes

git clone https://github.com/yourusername/ComfyUI-IF_VideoPrompts.git

Run the installation script:

cd ComfyUI-IF_VideoPrompts

python install.py

Restart ComfyUI

Method 2: Manual installation

Clone this repository into your ComfyUI custom_nodes directory:

cd ComfyUI/custom_nodes

git clone https://github.com/yourusername/ComfyUI-IF_VideoPrompts.git

Install required dependencies:

# Install compatible autoawq version WITHOUT dependencies to prevent transformers downgrade

# If you want to use AWQ to save VRAM and up to 3x faster inference

# you need to install triton and autoawq



# Then install other dependencies

pip install -r requirements.txt

pip install --no-deps --no-build-isolation autoawq

I also have precompiled wheels for FA2 sageattention and triton for windows 10 for cu126 and pytorch 2.6.3 and python 12+ https://huggingface.co/impactframes/ComfyUI_desktop_wheels_win_cp12_cu126/tree/main

Restart ComfyUI

Method 3: Direct pip installation

If you want to install the dependencies directly with pip:

pip install transformers>=4.49.0 opencv-python decord huggingface_hub pillow torch numpy tokenizers safetensors accelerate tqdm psutil packaging

pip install --no-deps --no-build-isolation autoawq

Usage

Frame-based Mode

Load a video using the VideoHelperSuite’s LoadVideo node

Connect the output to the VideoSequenceAnalyzer node

Select “Frames” as the input mode

Choose your preferred model, analysis type, and other settings

Run the workflow to get a detailed description of the video sequence

Direct Video File Mode

Upload a video file to your ComfyUI input directory

Add the VideoSequenceAnalyzer node

Select “Video File” as the input mode

Choose your video file from the dropdown

Configure FPS, analysis type, and other settings

Run the workflow to get a detailed description of the video

Models

The extension supports the following Qwen2.5-VL models:

Qwen2.5-VL-3B-Instruct

Qwen2.5-VL-7B-Instruct

Qwen2.5-VL-14B-Instruct

Qwen2.5-VL-72B-Instruct

Qwen2.5-VL-3B-Instruct-AWQ (quantized)

Qwen2.5-VL-7B-Instruct-AWQ (quantized)

Qwen2.5-VL-14B-Instruct-AWQ (quantized)

Qwen2.5-VL-72B-Instruct-AWQ (quantized)

AWQ quantized models are recommended for better performance.

Custom Presets

You can define your own presets by adding them to the presets/profiles.json file.

Troubleshooting

“Image features and image tokens do not match” Error

If you encounter this error, try the following:

Switch to “Video File” input mode to use the native Qwen-VL video processing

Reduce the number of frames in your sequence

Try a different model (AWQ versions often work better)

Transformers Version Conflicts

This extension requires transformers version 4.49.0 or higher. Earlier versions (including 4.48.0) will not work.

If autoawq or other packages downgrade your transformers version, follow these steps:

# Uninstall both packages

pip uninstall -y autoawq transformers



# Install specific transformers version first

pip install transformers==4.49.0



# Then install compatible autoawq WITHOUT dependencies

pip install --no-deps autoawq==0.2.8

if you have some fuckery with the LD_libray whatever do:

pip uninstall bitsandbytes -y

pip install bitsandbytes

Missing Dependencies

If you’re missing dependencies, use the installation script:

python install.py

Credits

This extension uses the following components:

Qwen2.5-VL models from Alibaba

Video Helper Suite for frame extraction

Support

If you find this tool useful, please consider supporting my work by:

Starring the repository on GitHub: ComfyUI-IF_VideoPrompts

Subscribing to my YouTube channel: Impact Frames

Follow me on X: Impact Frames X

Thank You!