|
26 | 26 |
|
27 | 27 | ## :book:Table Of Contents |
28 | 28 |
|
29 | | -- [Visual Results On Real-world Images](#visual_results) |
30 | 29 | - [Update](#update) |
| 30 | +- [Visual Results On Real-world Images](#visual_results) |
31 | 31 | - [TODO](#todo) |
32 | 32 | - [Installation](#installation) |
33 | 33 | - [Pretrained Models](#pretrained_models) |
34 | 34 | - [Inference](#inference) |
35 | 35 | - [Train](#train) |
36 | 36 |
|
| 37 | +## <a name="update"></a>:new:Update |
| 38 | + |
| 39 | +- **2024.04.08**: ✅ Release everything about our [updated manuscript](https://arxiv.org/abs/2308.15070), including (1) a **new model** trained on subset of laion2b-en and (2) a **more readable code base**, etc. DiffBIR is now a general restoration pipeline that could handle different blind image restoration tasks with a unified generation module. |
| 40 | +- **2023.09.19**: ✅ Add support for Apple Silicon! Check [installation_xOS.md](assets/docs/installation_xOS.md) to work with **CPU/CUDA/MPS** device! |
| 41 | +- **2023.09.14**: ✅ Integrate a patch-based sampling strategy ([mixture-of-diffusers](https://github.com/albarji/mixture-of-diffusers)). [**Try it!**](#patch-based-sampling) Here is an [example](https://imgsli.com/MjA2MDA1) with a resolution of 2396 x 1596. GPU memory usage will continue to be optimized in the future and we are looking forward to your pull requests! |
| 42 | +- **2023.09.14**: ✅ Add support for background upsampler (DiffBIR/[RealESRGAN](https://github.com/xinntao/Real-ESRGAN)) in face enhancement! :rocket: [**Try it!**](#inference_fr) |
| 43 | +- **2023.09.13**: :rocket: Provide online demo (DiffBIR-official) in [OpenXLab](https://openxlab.org.cn/apps/detail/linxinqi/DiffBIR-official), which integrates both general model and face model. Please have a try! [camenduru](https://github.com/camenduru) also implements an online demo, thanks for his work.:hugs: |
| 44 | +- **2023.09.12**: ✅ Upload inference code of latent image guidance and release [real47](inputs/real47) testset. |
| 45 | +- **2023.09.08**: ✅ Add support for restoring unaligned faces. |
| 46 | +- **2023.09.06**: :rocket: Update [colab demo](https://colab.research.google.com/github/camenduru/DiffBIR-colab/blob/main/DiffBIR_colab.ipynb). Thanks to [camenduru](https://github.com/camenduru)!:hugs: |
| 47 | +- **2023.08.30**: This repo is released. |
| 48 | + |
37 | 49 | ## <a name="visual_results"></a>:eyes:Visual Results On Real-world Images |
38 | 50 |
|
39 | 51 | ### Blind Image Super-Resolution |
|
44 | 56 |
|
45 | 57 | [<img src="assets/visual_results/bsr3.png" height="223px"/>](https://imgsli.com/MTk5ODI0) [<img src="assets/visual_results/bsr5.png" height="223px"/>](https://imgsli.com/MjAxMjM0) --> |
46 | 58 |
|
47 | | -[<img src="assets/visual_results/bsr1.png" height="223px"/>](https://imgsli.com/MTk5ODIy) [<img src="assets/visual_results/bsr5.png" height="223px"/>](https://imgsli.com/MjAxMjM0) |
| 59 | +<!-- [<img src="assets/visual_results/bsr1.png" height="223px"/>](https://imgsli.com/MTk5ODIy) [<img src="assets/visual_results/bsr5.png" height="223px"/>](https://imgsli.com/MjAxMjM0) --> |
48 | 60 |
|
49 | 61 | ### Blind Face Restoration |
50 | 62 |
|
51 | | -[<img src="assets/visual_results/bfr1.png" height="223px"/>](https://imgsli.com/MTk5ODI5) [<img src="assets/visual_results/bfr2.png" height="223px"/>](https://imgsli.com/MTk5ODMw) [<img src="assets/visual_results/bfr4.png" height="223px"/>](https://imgsli.com/MTk5ODM0) |
| 63 | +<!-- [<img src="assets/visual_results/bfr1.png" height="223px"/>](https://imgsli.com/MTk5ODI5) [<img src="assets/visual_results/bfr2.png" height="223px"/>](https://imgsli.com/MTk5ODMw) [<img src="assets/visual_results/bfr4.png" height="223px"/>](https://imgsli.com/MTk5ODM0) --> |
52 | 64 |
|
53 | 65 | [<img src="assets/visual_results/whole_image1.png" height="370"/>](https://imgsli.com/MjA2MTU0) |
54 | 66 | [<img src="assets/visual_results/whole_image2.png" height="370"/>](https://imgsli.com/MjA2MTQ4) |
|
66 | 78 |
|
67 | 79 | [<img src="assets/visual_results/tiled_sampling.png" height="480px"/>](https://imgsli.com/MjUzODE4) |
68 | 80 |
|
69 | | -## <a name="update"></a>:new:Update |
70 | | - |
71 | | -- **2024.04.08**: ✅ Release everything about our [updated manuscript](https://arxiv.org/abs/2308.15070), including (1) a **new model** trained on subset of laion2b-en and (2) a **more readable code base**, etc. DiffBIR is now a general restoration pipeline that could handle different blind image restoration tasks with a unified generation module. |
72 | | -- **2023.09.19**: ✅ Add support for Apple Silicon! Check [installation_xOS.md](assets/docs/installation_xOS.md) to work with **CPU/CUDA/MPS** device! |
73 | | -- **2023.09.14**: ✅ Integrate a patch-based sampling strategy ([mixture-of-diffusers](https://github.com/albarji/mixture-of-diffusers)). [**Try it!**](#patch-based-sampling) Here is an [example](https://imgsli.com/MjA2MDA1) with a resolution of 2396 x 1596. GPU memory usage will continue to be optimized in the future and we are looking forward to your pull requests! |
74 | | -- **2023.09.14**: ✅ Add support for background upsampler (DiffBIR/[RealESRGAN](https://github.com/xinntao/Real-ESRGAN)) in face enhancement! :rocket: [**Try it!**](#inference_fr) |
75 | | -- **2023.09.13**: :rocket: Provide online demo (DiffBIR-official) in [OpenXLab](https://openxlab.org.cn/apps/detail/linxinqi/DiffBIR-official), which integrates both general model and face model. Please have a try! [camenduru](https://github.com/camenduru) also implements an online demo, thanks for his work.:hugs: |
76 | | -- **2023.09.12**: ✅ Upload inference code of latent image guidance and release [real47](inputs/real47) testset. |
77 | | -- **2023.09.08**: ✅ Add support for restoring unaligned faces. |
78 | | -- **2023.09.06**: :rocket: Update [colab demo](https://colab.research.google.com/github/camenduru/DiffBIR-colab/blob/main/DiffBIR_colab.ipynb). Thanks to [camenduru](https://github.com/camenduru)!:hugs: |
79 | | -- **2023.08.30**: This repo is released. |
80 | | - |
81 | 81 | ## <a name="todo"></a>:climbing:TODO |
82 | 82 |
|
83 | 83 | - [x] Release code and pretrained models:computer:. |
@@ -115,10 +115,10 @@ Here we list pretrained weight of stage 2 model (IRControlNet) and our trained S |
115 | 115 |
|
116 | 116 | | Model Name | Description | HuggingFace | BaiduNetdisk | OpenXLab | |
117 | 117 | | :---------: | :----------: | :----------: | :----------: | :----------: | |
118 | | -| v2.pth | IRControlNet trained on filtered laion2b-en | [download](https://huggingface.co/lxq007/DiffBIR-v2/resolve/main/v2.pth) | [download](https://pan.baidu.com/s/1uTAFl13xgGAzrnznAApyng?pwd=xiu3)<br>(pwd: xiu3) | [download](https://download.openxlab.org.cn/repos/file/linxinqi/DiffBIR/main?filepath=v2.pth&sign=e71fbc40d2a1c1c4a45e860701954e6b&nonce=1712581884856) | |
119 | | -| v1_general.pth | IRControlNet trained on ImageNet-1k | [download](https://huggingface.co/lxq007/DiffBIR-v2/resolve/main/v1_general.pth) | [download](https://pan.baidu.com/s/1PhXHAQSTOUX4Gy3MOc2t2Q?pwd=79n9)<br>(pwd: 79n9) | [download](https://download.openxlab.org.cn/repos/file/linxinqi/DiffBIR/main?filepath=v1_general.pth&sign=6dacb066be8339cd2814f162c0447535&nonce=1712581928407) | |
120 | | -| v1_face.pth | IRControlNet trained on FFHQ | [download](https://huggingface.co/lxq007/DiffBIR-v2/resolve/main/v1_face.pth) | [download](https://pan.baidu.com/s/1kvM_SB1VbXjbipLxdzlI3Q?pwd=n7dx)<br>(pwd: n7dx) | [download](https://download.openxlab.org.cn/repos/file/linxinqi/DiffBIR/main?filepath=v1_face.pth&sign=45d329c9007a7d3c7d44b54cc9660c51&nonce=1712581953073) | |
121 | | -| codeformer_swinir.ckpt | SwinIR trained on ImageNet-1k | [download](https://huggingface.co/lxq007/DiffBIR-v2/resolve/main/codeformer_swinir.ckpt) | [download](https://pan.baidu.com/s/176fARg2ySYtDgX2vQOeRbA?pwd=vfif)<br>(pwd: vfif) | [download](https://download.openxlab.org.cn/repos/file/linxinqi/DiffBIR/main?filepath=codeformer_swinir.ckpt&sign=d7f783f73e50bf3ac9e7d5c13612bc63&nonce=1712581967673) | |
| 118 | +| v2.pth | IRControlNet trained on filtered laion2b-en | [download](https://huggingface.co/lxq007/DiffBIR-v2/resolve/main/v2.pth) | [download](https://pan.baidu.com/s/1uTAFl13xgGAzrnznAApyng?pwd=xiu3)<br>(pwd: xiu3) | [download](https://openxlab.org.cn/models/detail/linxinqi/DiffBIR/tree/main) | |
| 119 | +| v1_general.pth | IRControlNet trained on ImageNet-1k | [download](https://huggingface.co/lxq007/DiffBIR-v2/resolve/main/v1_general.pth) | [download](https://pan.baidu.com/s/1PhXHAQSTOUX4Gy3MOc2t2Q?pwd=79n9)<br>(pwd: 79n9) | [download](https://openxlab.org.cn/models/detail/linxinqi/DiffBIR/tree/main) | |
| 120 | +| v1_face.pth | IRControlNet trained on FFHQ | [download](https://huggingface.co/lxq007/DiffBIR-v2/resolve/main/v1_face.pth) | [download](https://pan.baidu.com/s/1kvM_SB1VbXjbipLxdzlI3Q?pwd=n7dx)<br>(pwd: n7dx) | [download](https://openxlab.org.cn/models/detail/linxinqi/DiffBIR/tree/main) | |
| 121 | +| codeformer_swinir.ckpt | SwinIR trained on ImageNet-1k | [download](https://huggingface.co/lxq007/DiffBIR-v2/resolve/main/codeformer_swinir.ckpt) | [download](https://pan.baidu.com/s/176fARg2ySYtDgX2vQOeRbA?pwd=vfif)<br>(pwd: vfif) | [download](https://openxlab.org.cn/models/detail/linxinqi/DiffBIR/tree/main) | |
122 | 122 |
|
123 | 123 | During inference, we use off-the-shelf models from other papers as the stage 1 model: [BSRNet](https://github.com/cszn/BSRGAN) for BSR, [SwinIR-Face](https://github.com/zsyOAOA/DifFace) used in DifFace for BFR, and [SCUNet-PSNR](https://github.com/cszn/SCUNet) for BID, while the trained IRControlNet remains **unchanged** for all tasks. Please check [code](utils/inference.py) for more details. Thanks for their work! |
124 | 124 |
|
|
0 commit comments