ComfyUI-Lumina-Next-SFT-DiffusersWrapper

ComfyUI-Lumina-Next-SFT-DiffusersWrapper
★ 17

文本到图像高质量生成图像到图像时间感知采样
ComfyUI-Lumina-Next-SFT-DiffusersWrapper 将 Lumina-Next-SFT 集成到 ComfyUI，提供高质量文本到图像生成、时间感知缩放、可选 ODE 采样与高分辨率支持，支持输入 latent 与 strength 控制，输出 latent 表示。
💡 在 ComfyUI 中使用 Lumina-Next-SFT 生成或修复高分辨率图像。
🍴 2 Forks💻 Python🔄 2024-07-30
📦
网盘下载
复制链接后前往夸克网盘下载
https://pan.quark.cn/s/f414772aa5c3
📦 requirements.txt
git+https://github.com/huggingface/diffusers
transformers
accelerate
torchdiffeq
Screenshot 2024-07-28 195044
ComfyUI_temp_qqfjt_00016_(1)
Screenshot 2024-07-22 103940
image
Screenshot 2024-07-22 131142
Screenshot 2024-07-22 104629
image
image
ComfyUI_temp_mhdzy_00001_
ComfyUI_temp_mhdzy_00004_
ComfyUI_temp_ntirq_00004_
ComfyUI_temp_ntirq_00003_
image
ComfyUI_temp_kbsgn_00011_
📄 README

ComfyUI-Lumina-Next-SFT-DiffusersWrapper

Lumina Diffusers Node for ComfyUI

This custom node seamlessly integrates the Lumina-Next-SFT model into ComfyUI, enabling high-quality image generation using the advanced Lumina text-to-image pipeline. While still under active development, it offers a robust and functional implementation with advanced features.

Features

  • Harnesses the power of the Lumina-Next-SFT model for state-of-the-art image generation
  • Offers a wide range of generation parameters for fine-tuned control
  • Implements Lumina-specific features including scaling watershed and proportional attention
  • Supports input latents and strength parameter for image-to-image capabilities
  • Automatic model downloading for seamless setup
  • Outputs generated latent representations

    • Installation

    Now in ComfyUI Manager!

    For manual installation:

  • Ensure you have ComfyUI installed and properly set up.
  • Clone this repository into your ComfyUI custom nodes directory:

    • “`

    git clone https://github.com/Excidos/ComfyUI-Lumina-Diffusers.git

    “`

  • The required dependencies will be automatically installed.

    • NOTE: This installation includes a development branch of diffusers, which may conflict with some existing nodes.

    Usage

    Use with the standard SDXL_VAE or SDXL_Fixed_FP16-VAE

  • Launch ComfyUI.
  • Locate the “Lumina-Next-SFT Diffusers” node in the node selection menu.
  • Add the node to your workflow.
  • Connect the necessary inputs and outputs.
  • Configure the node parameters as desired.
  • Execute your workflow to generate images.

    • Parameters

  • model_path: Path to the Lumina model (default: “Alpha-VLLM/Lumina-Next-SFT-diffusers”)
  • prompt: Text prompt for image generation
  • negative_prompt: Negative text prompt
  • num_inference_steps: Number of denoising steps (default: 30)
  • guidance_scale: Classifier-free guidance scale (default: 4.0)
  • seed: Random seed for generation (-1 for random)
  • batch_size: Number of images to generate in one batch (default: 1)
  • scaling_watershed: Scaling watershed parameter (default: 0.3)
  • proportional_attn: Enable proportional attention (default: True)
  • clean_caption: Clean input captions (default: True)
  • max_sequence_length: Maximum sequence length for text input (default: 256)
  • use_time_shift: Enable time shift feature (default: False)
  • t_shift: Time shift factor (default: 4)
  • strength: Strength for image-to-image generation (default: 1.0, range: 0.0 to 1.0)

    • Inputs

  • latents (optional): Input latents for image-to-image generation

    • Outputs

  • LATENT: Latent representation of the generated image(s)

    • Known Features and Limitations

  • Supports input latents for image-to-image generation
  • Implements strength parameter for controlling the influence of input latents
  • Time shift feature for advanced control over the generation process
  • Output is currently limited to latent representations; use a VAE decode node to obtain images

    • Example Outputs

    Troubleshooting

    If you encounter any issues, please check the console output for error messages. Common issues include:

  • Insufficient GPU memory
  • Missing dependencies
  • Incorrect model path

    • For further assistance, please open an issue on the GitHub repository.

    Contributing

    Contributions are welcome! Please feel free to submit a Pull Request.

    Acknowledgements

  • Lumina-Next-SFT-Diffusers for the Lumina-Next-SFT model
  • The ComfyUI community for their continuous support and inspiration