ComfyUI_SemanticSAM

★ 6

语义分割细粒度掩码候选掩码生成依赖Detectron2

基于Semantic-SAM的一键细粒度语义分割节点，生成更多候选掩码，适合作为inpainting等任务的预处理，Linux+CUDA环境下可用。

💡 在inpainting前自动生成细粒度掩码

🍴 2 Forks💻 Python🔄 2024-08-13

🔗 GitHub 原文

📦

网盘下载

复制链接后前往夸克网盘下载

https://pan.quark.cn/s/a1f1f564f19c

📦 requirements.txt

torch
torchvision
opencv-python
pyyaml
json_tricks
yacs
scikit-learn
pandas
timm==0.4.12
numpy==1.23.5
einops
fvcore
transformers
#==4.19.2
sentencepiece
ftfy
regex
nltk
vision-datasets==0.2.2
pycocotools
diffdist
pyarrow
cityscapesscripts
shapely
scikit-image
mup
#gradio==3.35.2
scann
kornia==0.6.4
torchmetrics==0.6.0
progressbar
pillow==9.4.0

📄 README

ComfyUI_SemanticSAM

This is a ComfyUI node based-on Semantic-SAM official implementation. We provide a workflow node for one-click segment. Compared with SAM, Semantic-SAM has better fine-grained capabilities and more candidate masks. Users can take this node as the pre-node for inpainting to obtain the mask region.

This node have been valided on Ubuntu-20.04 & CUDA-11.8. Due to detectron2, this node is currently not supported on Windows (unless the compatibility of Detectron2 on Windows is addressed)

Get Start

1. Prepare repo

cd ComfyUI/custom
git clone https://github.com/eastoc/ComfyUI_SemanticSAM
cd ComfyUI_SemanticSAM
git clone https://github.com/facebookresearch/detectron2
git clone https://github.com/facebookresearch/Mask2Former

Because detectron2 has not been updated, if the CUDA=11.8, you can

git clone https://github.com/johnnynunez/detectron2

or you can try other 3rd detectron2 implementation.

2. Download models

download Semantic-SAM model to “ComfyUI_SemanticSAM/ckpt”

Name	Training Dataset	Backbone	1-IoU@Multi-Granularity	1-IoU@COCO(Max\|Oracle)	download
Semantic-SAM \| config	SA-1B	SwinT	88.1	54.5\|73.8	model
Semantic-SAM \| config	SA-1B	SwinL	89.0	55.1\|74.1	model

3. Install environment

Install PyTorch & torchvision through the official method. We have been valid that from pytorch 1.13.0 to 2.3.0.

Install Semantic-SAM dependencies

pip install -r requirements.txt

Install detectron2

cd detectron2
pip install -e .
cd ..

Install Mask2Former

cd Mask2Former/mask2former/modeling/pixel_decoder/ops
sh make.sh

Workflow

one-click segment workflow is in “./workflow/workflow.json”