ComfyUI_HF_Servelress_Inference

ComfyUI_HF_Servelress_Inference
★ 17

Hugging Face 平台无服务器推理大型语言模型提示增强
在 Hugging Face Serverless 端点上对指定 LLM 运行推理，实时增强提示并返回字符串结果，便于在 ComfyUI 中调用远程模型。
💡 在 ComfyUI 中调用 Hugging Face serverless 端点以增强并执行 LLM 提问。
🍴 5 Forks💻 Python🔄 2024-09-01
📦
网盘下载
复制链接后前往夸克网盘下载
https://pan.quark.cn/s/e98a62d17551
📦 requirements.txt
huggingface_hub
📄 README

Huggingface Api Serverless Inference

It’s a node to Enhance any Prompt running the inference of LLM model on the serverless endpoint on Hugginggface.

Inputs:

  • hf_token: String — It’s the token string
  • endpoint: String — It’s the repo id hosted on HF (Ex: meta-llama/Meta-Llama-3-8B-Instruct)
  • question: String — Type here the question
  • context: String — LLM Instruction for the model

    • Output:

  • STRING: String — A string

    • Installation

    Clone and install dependencies

    git clone https://github.com/alexgenovese/ComfyUI_HF_Servelress_Inference custom_nodes/ComfyUI_HF_Servelress_Inference
cd custom_nodes/ComfyUI_HF_Servelress_Inference
pip install -r requirements.txt

    Go into your account and create a new TOKEN: Hugging Face tokens

    Limitations

    [!WARNING]

    Inference API (serverless) requires a model 10GB or below and fails for random reasons on different models.

    Sponsor

    If you find this ComfyUI helpful node and would like to support its development, consider becoming a sponsor. Your sponsorship will help ensure the continued maintenance and improvement of this Comfy nodes and other projects.

    You can sponsor me on GitHub Sponsors by clicking this link.

    Every contribution is highly appreciated!