ComfyUI LLMs Extension

A ComfyUI extension that supports various Large Language Models (LLMs) and Vision Language Models (VLMs), providing a unified interface and simple configuration.

English | 中文

✨ Features

🤖 Support for multiple LLM chat models

🎯 Unified vision model interface with multiple VLM support

🔄 Dynamic model switching

🌐 Bilingual interface (English/Chinese)

⚙️ Simple configuration

📦 Installation

Navigate to ComfyUI’s custom_nodes directory

cd ComfyUI/custom_nodes

Clone the repository

git clone https://github.com/leoleexh/ComfyUI-LLMs

Install dependencies

cd ComfyUI-LLMs
pip install -r requirements.txt

⚙️ Configuration

Basic Setup

Copy the configuration template

cp settings.yaml.sample settings.yaml

Edit settings.yaml to configure your API keys and model settings

Configuration Validation

After setting up your configuration, you can use the validation tool to check if everything is configured correctly:

python validate_config.py

This tool will:

✅ Check if settings.yaml exists

✅ Validate configuration file structure

✅ Display current API endpoints and model configurations

⚠️ Warn about default API keys that need to be updated

Detailed Configuration

The configuration file structure is as follows:

chatllmleoleexh:
  # OpenAI Compatible Interface Configuration
  openai_compatible:
    default:
      api_base: "http://your-api-endpoint:3200/v1"  # API endpoint
      organisation: "NONE"                          # Organization ID (optional)
      api_key: "your-api-key"                      # API key
      model:                                       # Supported models
        - "gpt-3.5-turbo"
        - "gpt-4"
        # Other supported models...

  # Vision Models Configuration
  vision_models:
    # OpenAI Vision Model
    openai:
      api_key: "your-openai-key"
      api_base: "http://your-api-endpoint:3200/v1"
      model_list:
        - "gpt-4-vision-preview"
    
    # Zhipu GLM4 Vision Model
    glm4:
      api_key: "your-glm4-key"
      model_list: 
        - "glm-4v"
        - "glm-4"
    
    # Alibaba Qwen Vision Model
    ali:
      api_key: "your-ali-key"
      model_list:
        - "qwen-vl-plus"
        - "qwen-vl-max"
    
    # Google Gemini Vision Model
    gemini:
      api_key: "your-gemini-key"
      model_list:
        - "gemini-pro-vision"

  # Prompt Templates Configuration
  prompt_templates:
    default:
      system: "System prompt"
      prefix: "Prefix prompt"
      suffix: "Suffix prompt"

Model Support

OpenAI and Compatible Models

Standard OpenAI API support

Third-party compatible interfaces (e.g., One API)

Multiple models and endpoints configuration

Vision Model Support

OpenAI GPT-4V

Zhipu GLM-4V

Alibaba Qwen VL

Google Gemini

API Key Acquisition

OpenAI: https://platform.openai.com/

Zhipu GLM: https://open.bigmodel.cn/

Alibaba Qwen: https://dashscope.aliyun.com/

Google Gemini: https://makersuite.google.com/

🎯 Usage

Chat Function

Find 🤖 LLMs Chat in the node list

Configure model parameters

Input conversation content

Image Understanding

Find 🎯 LLMs Vision in the node list

Select the vision model to use

Connect image input

Run to get image description

🔄 Changelog

See CHANGELOG.md

📝 Notes

Ensure API keys are configured correctly

Some models may require proxy access

Stable network connection recommended

Be aware of API rate limits

🤝 Contributing

Issues and Pull Requests are welcome!

📄 License

MIT License

---

ComfyUI LLMs 扩展

ComfyUI的LLM扩展，支持多种大语言模型和视觉语言模型，提供统一的接口和简单的配置方式。

✨ 功能特点

🤖 支持多种LLM模型的对话功能

🎯 统一的视觉模型接口，支持多种视觉语言模型

🔄 动态模型切换

🌐 支持中英文双语界面

⚙️ 简单的配置方式

📦 安装方法

进入ComfyUI的custom_nodes目录

cd ComfyUI/custom_nodes

克隆仓库

git clone https://github.com/leoleexh/ComfyUI-LLMs

安装依赖

cd ComfyUI-LLMs
pip install -r requirements.txt

⚙️ 配置说明

基本配置

复制配置文件模板

cp settings.yaml.sample settings.yaml

编辑 settings.yaml 文件，配置您的API密钥和模型设置

Configuration Validation

After setting up your configuration, you can use the validation tool to check if everything is configured correctly:

python validate_config.py

This tool will:

✅ Check if settings.yaml exists

✅ Validate configuration file structure

✅ Display current API endpoints and model configurations

⚠️ Warn about default API keys that need to be updated

详细配置说明

配置文件结构如下：

chatllmleoleexh:
  # OpenAI兼容接口配置
  openai_compatible:
    default:
      api_base: "http://your-api-endpoint:3200/v1"  # API端点
      organisation: "NONE"                          # 组织ID（可选）
      api_key: "your-api-key"                      # API密钥
      model:                                       # 支持的模型列表
        - "gpt-3.5-turbo"
        - "gpt-4"
        # 其他支持的模型...

  # 视觉模型配置
  vision_models:
    # OpenAI视觉模型
    openai:
      api_key: "your-openai-key"
      api_base: "http://your-api-endpoint:3200/v1"
      model_list:
        - "gpt-4-vision-preview"
    
    # 智谱GLM4视觉模型
    glm4:
      api_key: "your-glm4-key"
      model_list: 
        - "glm-4v"
        - "glm-4"
    
    # 阿里通义千问视觉模型
    ali:
      api_key: "your-ali-key"
      model_list:
        - "qwen-vl-plus"
        - "qwen-vl-max"
    
    # Google Gemini视觉模型
    gemini:
      api_key: "your-gemini-key"
      model_list:
        - "gemini-pro-vision"

  # 提示词模板配置
  prompt_templates:
    default:
      system: "系统提示词"
      prefix: "前缀提示词"
      suffix: "后缀提示词"

模型支持说明

OpenAI及兼容模型

支持标准OpenAI接口

支持第三方兼容接口（如One API）

可配置多个模型和接口

视觉模型支持

OpenAI GPT-4V

智谱 GLM-4V

阿里通义千问

Google Gemini

API密钥获取

OpenAI: https://platform.openai.com/

智谱GLM: https://open.bigmodel.cn/

阿里通义千问: https://dashscope.aliyun.com/

Google Gemini: https://makersuite.google.com/

🎯 使用方法

聊天功能

在节点列表中找到 🤖 LLMs Chat | 智能对话

配置模型参数

输入对话内容

图像理解功能

在节点列表中找到 🎯 LLMs Vision | 图像理解

选择使用的视觉模型

连接图像输入

运行获取图像描述

🔄 更新日志

详见 CHANGELOG.md

📝 注意事项

请确保API密钥配置正确

部分模型可能需要代理访问

建议使用稳定的网络环境

注意API调用频率限制

🤝 贡献

欢迎提交Issue和Pull Request！

📄 许可证

MIT License