Update README.md
Browse files
README.md
CHANGED
|
@@ -10,12 +10,12 @@ library_name: diffusers
|
|
| 10 |
---
|
| 11 |
|
| 12 |
|
| 13 |
-
Kolors-ControlNet-Canny weights and inference code
|
| 14 |
|
| 15 |
|
| 16 |
## <a name="Introduction"></a>📖 Introduction
|
| 17 |
|
| 18 |
-
We provide two ControlNet weights and inference code based on Kolors-Basemodel: Canny and Depth. You can find some example images
|
| 19 |
|
| 20 |
|
| 21 |
**1、ControlNet Demos**
|
|
@@ -40,14 +40,14 @@ To evaluate the performance of models, we compiled a test set of more than 200 i
|
|
| 40 |
| **Kolors-ControlNet-Canny** | **4.06** | **4.64** | **4.45** | **3.52** |
|
| 41 |
|
| 42 |
|
| 43 |
-
|
| 44 |
**2、Depth**
|
| 45 |
|
| 46 |
| Model | Average Overall Satisfaction | Average Visual Appeal | Average Text Faithfulness | Average Conditional Controllability |
|
| 47 |
| :--------------: | :--------: | :--------: | :--------: | :--------: |
|
| 48 |
-
| SDXL-ControlNet-
|
| 49 |
| **Kolors-ControlNet-Depth** | **4.12** | **4.12** | **4.62** | **4.6** |
|
| 50 |
|
|
|
|
| 51 |
<img src="compare_demo.png">
|
| 52 |
|
| 53 |
<font color=gray style="font-size:12px">*The [SDXL-ControlNet-Canny](https://huggingface.co/diffusers/controlnet-canny-sdxl-1.0) and [SDXL-ControlNet-Depth](https://huggingface.co/diffusers/controlnet-depth-sdxl-1.0) load [DreamShaper-XL](https://civitai.com/models/112902?modelVersionId=351306) as backbone model.*</font>
|
|
@@ -60,10 +60,8 @@ To evaluate the performance of models, we compiled a test set of more than 200 i
|
|
| 60 |
|
| 61 |
The dependencies and installation are basically the same as the [Kolors-BaseModel](https://huggingface.co/Kwai-Kolors/Kolors).
|
| 62 |
|
| 63 |
-
<br>
|
| 64 |
-
|
| 65 |
|
| 66 |
-
|
| 67 |
```bash
|
| 68 |
# Canny - ControlNet
|
| 69 |
huggingface-cli download --resume-download Kwai-Kolors/Kolors-ControlNet-Canny --local-dir weights/Kolors-ControlNet-Canny
|
|
@@ -72,7 +70,7 @@ huggingface-cli download --resume-download Kwai-Kolors/Kolors-ControlNet-Canny -
|
|
| 72 |
huggingface-cli download --resume-download Kwai-Kolors/Kolors-ControlNet-Depth --local-dir weights/Kolors-ControlNet-Depth
|
| 73 |
```
|
| 74 |
|
| 75 |
-
If you intend to utilize the depth estimation network, please
|
| 76 |
```
|
| 77 |
huggingface-cli download lllyasviel/Annotators ./dpt_hybrid-midas-501f0c75.pt --local-dir ./controlnet/annotator/ckpts
|
| 78 |
```
|
|
@@ -104,7 +102,7 @@ python ./controlnet/sample_controlNet.py ./controlnet/assets/bird.png 一只颜
|
|
| 104 |
|
| 105 |
**c. Using depth ControlNet + IP-Adapter-Plus:**
|
| 106 |
|
| 107 |
-
If you intend to utilize the kolors-ip-adapter-plus, please
|
| 108 |
|
| 109 |
```bash
|
| 110 |
python ./controlnet/sample_controlNet_ipadapter.py ./controlnet/assets/woman_2.png ./ipadapter/asset/2.png 一个红色头发的女孩,唯美风景,清新明亮,斑驳的光影,最好的质量,超细节,8K画质 Depth
|
|
|
|
| 10 |
---
|
| 11 |
|
| 12 |
|
| 13 |
+
# Kolors-ControlNet-Canny weights and inference code
|
| 14 |
|
| 15 |
|
| 16 |
## <a name="Introduction"></a>📖 Introduction
|
| 17 |
|
| 18 |
+
We provide two ControlNet weights and inference code based on Kolors-Basemodel: Canny and Depth. You can find some example images below.
|
| 19 |
|
| 20 |
|
| 21 |
**1、ControlNet Demos**
|
|
|
|
| 40 |
| **Kolors-ControlNet-Canny** | **4.06** | **4.64** | **4.45** | **3.52** |
|
| 41 |
|
| 42 |
|
|
|
|
| 43 |
**2、Depth**
|
| 44 |
|
| 45 |
| Model | Average Overall Satisfaction | Average Visual Appeal | Average Text Faithfulness | Average Conditional Controllability |
|
| 46 |
| :--------------: | :--------: | :--------: | :--------: | :--------: |
|
| 47 |
+
| SDXL-ControlNet-Depth | 3.35 | 3.77 | 4.26 | 4.5 |
|
| 48 |
| **Kolors-ControlNet-Depth** | **4.12** | **4.12** | **4.62** | **4.6** |
|
| 49 |
|
| 50 |
+
|
| 51 |
<img src="compare_demo.png">
|
| 52 |
|
| 53 |
<font color=gray style="font-size:12px">*The [SDXL-ControlNet-Canny](https://huggingface.co/diffusers/controlnet-canny-sdxl-1.0) and [SDXL-ControlNet-Depth](https://huggingface.co/diffusers/controlnet-depth-sdxl-1.0) load [DreamShaper-XL](https://civitai.com/models/112902?modelVersionId=351306) as backbone model.*</font>
|
|
|
|
| 60 |
|
| 61 |
The dependencies and installation are basically the same as the [Kolors-BaseModel](https://huggingface.co/Kwai-Kolors/Kolors).
|
| 62 |
|
|
|
|
|
|
|
| 63 |
|
| 64 |
+
### Weights download:
|
| 65 |
```bash
|
| 66 |
# Canny - ControlNet
|
| 67 |
huggingface-cli download --resume-download Kwai-Kolors/Kolors-ControlNet-Canny --local-dir weights/Kolors-ControlNet-Canny
|
|
|
|
| 70 |
huggingface-cli download --resume-download Kwai-Kolors/Kolors-ControlNet-Depth --local-dir weights/Kolors-ControlNet-Depth
|
| 71 |
```
|
| 72 |
|
| 73 |
+
If you intend to utilize the depth estimation network, please make sure to download its corresponding model weights.
|
| 74 |
```
|
| 75 |
huggingface-cli download lllyasviel/Annotators ./dpt_hybrid-midas-501f0c75.pt --local-dir ./controlnet/annotator/ckpts
|
| 76 |
```
|
|
|
|
| 102 |
|
| 103 |
**c. Using depth ControlNet + IP-Adapter-Plus:**
|
| 104 |
|
| 105 |
+
If you intend to utilize the kolors-ip-adapter-plus, please make sure to download its corresponding model weights.
|
| 106 |
|
| 107 |
```bash
|
| 108 |
python ./controlnet/sample_controlNet_ipadapter.py ./controlnet/assets/woman_2.png ./ipadapter/asset/2.png 一个红色头发的女孩,唯美风景,清新明亮,斑驳的光影,最好的质量,超细节,8K画质 Depth
|