jaychempan commited on
Commit
d9c7212
·
verified ·
1 Parent(s): 04fffe5

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +47 -4
README.md CHANGED
@@ -44,15 +44,59 @@ tags:
44
  <a href="#statement">Statement</a>
45
  </p>
46
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
47
  ## TODO
48
 
49
  - [ ] Release EarthSynth Models to 🤗 HuggingFace
50
  - [x] Release EarthSynth-180K Dataset to 🤗 HuggingFace
51
 
52
  ## News
53
- - [2025/8/7] EarthSynth-180K dataset is uploaded to 🤗 [HuggingFace](jaychempan/EarthSynth-180K).
54
  - [2025/5/20] Our paper of "EarthSynth: Generating Informative Earth Observation with Diffusion Models" is up on [arXiv](https://arxiv.org/abs/2505.12108).
55
 
 
56
  ## Abstract
57
 
58
  Remote sensing image (RSI) interpretation typically faces challenges due to the scarcity of labeled data, which limits the performance of RSI interpretation tasks. To tackle this challenge, we propose **EarthSynth**, a diffusion-based generative foundation model that enables synthesizing multi-category, cross-satellite labeled Earth observation for downstream RSI interpretation tasks. To the best of our knowledge, EarthSynth is the first to explore multi-task generation for remote sensing, tackling the challenge of limited generalization in task-oriented synthesis for RSI interpretation. EarthSynth, trained on the EarthSynth-180K dataset, employs the Counterfactual Composition training strategy with a three-dimensional batch-sample selection mechanism to improve training data diversity and enhance category control. Furthermore, a rule-based method of R-Filter is proposed to filter more informative synthetic data for downstream tasks. We evaluate our EarthSynth on scene classification, object detection, and semantic segmentation in open-world scenarios. There are significant improvements in open-vocabulary understanding tasks, offering a practical solution for advancing RSI interpretation.
@@ -160,10 +204,8 @@ python test.py --base_model path/to/stable-diffusion/ --controlnet_path path/to/
160
  ```
161
  ### Training Data Generation
162
 
163
- [TODO]
164
-
165
  <p align="center">
166
- <img src="assets/Vis.png" alt="Image" width="500">
167
  </p>
168
 
169
 
@@ -202,3 +244,4 @@ primaryClass={cs.CV},
202
  url={https://arxiv.org/abs/2505.12108},
203
  }
204
  ```
 
 
44
  <a href="#statement">Statement</a>
45
  </p>
46
 
47
+ ## Examples
48
+
49
+ <div align="center">
50
+ <p>A satellite image of <span style="color:inherit;">road</span>.</p>
51
+ <img src="./assets/gifs/DG_629683_0_road_1.gif" width="10%">
52
+ <img src="./assets/gifs/DG_629683_0_road_2.gif" width="10%">
53
+ <img src="./assets/gifs/DG_629683_0_road_3.gif" width="10%">
54
+ <img src="./assets/gifs/DG_629683_0_road_4.gif" width="10%">
55
+ <img src="./assets/gifs/DG_629683_0_road_5.gif" width="10%">
56
+ </div>
57
+ <div align="center">
58
+ <p>A satellite image of <span style="color:inherit;">small vehicle</span>.</p>
59
+ <img src="./assets/gifs/FAST_train_5093_0001_0_small_vehicle_1.gif" width="10%">
60
+ <img src="./assets/gifs/FAST_train_5093_0001_0_small_vehicle_2.gif" width="10%">
61
+ <img src="./assets/gifs/FAST_train_5093_0001_0_small_vehicle_3.gif" width="10%">
62
+ <img src="./assets/gifs/FAST_train_5093_0001_0_small_vehicle_4.gif" width="10%">
63
+ <img src="./assets/gifs/FAST_train_5093_0001_0_small_vehicle_5.gif" width="10%">
64
+ </div>
65
+ <div align="center">
66
+ <p>A satellite image of <span style="color:inherit;">tree</span>. (Flood)</p>
67
+ <img src="./assets/gifs/OEMxBD_coxsbazar_56_2_tree_1.gif" width="10%">
68
+ <img src="./assets/gifs/OEMxBD_coxsbazar_56_2_tree_2.gif" width="10%">
69
+ <img src="./assets/gifs/OEMxBD_coxsbazar_56_2_tree_3.gif" width="10%">
70
+ <img src="./assets/gifs/OEMxBD_coxsbazar_56_2_tree_4.gif" width="10%">
71
+ <img src="./assets/gifs/OEMxBD_coxsbazar_56_2_tree_5.gif" width="10%">
72
+ </div>
73
+ <div align="center">
74
+ <p>A satellite image of <span style="color:inherit;">water</span>.</p>
75
+ <img src="./assets/gifs/LD_300_0_water_1.gif" width="10%">
76
+ <img src="./assets/gifs/LD_300_0_water_2.gif" width="10%">
77
+ <img src="./assets/gifs/LD_300_0_water_3.gif" width="10%">
78
+ <img src="./assets/gifs/LD_300_0_water_4.gif" width="10%">
79
+ <img src="./assets/gifs/LD_300_0_water_5.gif" width="10%">
80
+ </div>
81
+ <div align="center">
82
+ <p>A satellite image of <span style="color:inherit;">baseball diamond, vehicle</span>.</p>
83
+ <img src="./assets/gifs/SIOR_07349_0_1.gif" width="10%">
84
+ <img src="./assets/gifs/SIOR_07349_0_2.gif" width="10%">
85
+ <img src="./assets/gifs/SIOR_07349_0_3.gif" width="10%">
86
+ <img src="./assets/gifs/SIOR_07349_0_4.gif" width="10%">
87
+ <img src="./assets/gifs/SIOR_07349_0_5.gif" width="10%">
88
+ </div>
89
+
90
  ## TODO
91
 
92
  - [ ] Release EarthSynth Models to 🤗 HuggingFace
93
  - [x] Release EarthSynth-180K Dataset to 🤗 HuggingFace
94
 
95
  ## News
96
+ - [2025/8/7] EarthSynth-180K dataset is uploaded to 🤗 [HuggingFace](https://huggingface.co/datasets/jaychempan/EarthSynth-180K).
97
  - [2025/5/20] Our paper of "EarthSynth: Generating Informative Earth Observation with Diffusion Models" is up on [arXiv](https://arxiv.org/abs/2505.12108).
98
 
99
+
100
  ## Abstract
101
 
102
  Remote sensing image (RSI) interpretation typically faces challenges due to the scarcity of labeled data, which limits the performance of RSI interpretation tasks. To tackle this challenge, we propose **EarthSynth**, a diffusion-based generative foundation model that enables synthesizing multi-category, cross-satellite labeled Earth observation for downstream RSI interpretation tasks. To the best of our knowledge, EarthSynth is the first to explore multi-task generation for remote sensing, tackling the challenge of limited generalization in task-oriented synthesis for RSI interpretation. EarthSynth, trained on the EarthSynth-180K dataset, employs the Counterfactual Composition training strategy with a three-dimensional batch-sample selection mechanism to improve training data diversity and enhance category control. Furthermore, a rule-based method of R-Filter is proposed to filter more informative synthetic data for downstream tasks. We evaluate our EarthSynth on scene classification, object detection, and semantic segmentation in open-world scenarios. There are significant improvements in open-vocabulary understanding tasks, offering a practical solution for advancing RSI interpretation.
 
204
  ```
205
  ### Training Data Generation
206
 
 
 
207
  <p align="center">
208
+ <img src="assets/Vis.png" alt="Image" width="300">
209
  </p>
210
 
211
 
 
244
  url={https://arxiv.org/abs/2505.12108},
245
  }
246
  ```
247
+