PsSAR-v1.0-Unlocking-SAR-Panoptic-Segmentation

Liang Zhang, Ziyu Lin, Chenyue Liu, Sai Ma, Zizhuo Teng, Enling Wu, Xingyu Jiao, Jiayi Song, Lihan Wang, Pingling Fang, Xin Zhang* (*Corresponding author)

Paper

Contents

• Introduction
• Install
• Dataset
• Train
• Model config
• Inference
• Evaluation

Introduction

Contributions

We propose the first public SAR panoptic segmentation dataset, named PsSAR-v1.0. Derived from HRSID, this dataset contains 5604 high resolution SAR images and their corresponding annotationsfor panoptic segmentation tasks. Based on PsSAR-v1.0, we constructs a bench-mark and conducts evaluations on SOTA panoptic segmentation models.

PsSAR-v1.0 Performance

Benchmark Method	Backbone Network	Category	PQ(%)	SQ(%)	RQ(%)	Model Size	Latency
Maskformer	ResNet-50	Overall	63.96	86.368	72.267	523MB	0.2437s
		Ship	54.117	83.077	65.141
		Land and Sea	68.881	88.014	75.83
Panoptic FPN	ResNet-50+FPN	Overall	66.427	85.415	75.749	350MB	0.1622s
		Ship	66.482	83.440	79.677
		Land and Sea	66.400	86.403	73.786
	ResNet-101+FPN	Overall	67.864	85.847	77.160	495MB	0.1205s
		Ship	68.573	83.974	81.660
		Land and Sea	67.510	86.784	74.910
Mask2former	ResNet-50	Overall	68.783	88.235	76.866	595MB	0.2256s
		Ship	62.692	82.694	75.813
		Land and Sea	71.829	91.006	77.392
	ResNet-101	Overall	68.814	88.405	76.790	798MB	0.2592s
		Ship	62.797	82.844	75.801
		Land and Sea	71.822	91.186	77.285
Panoptic-DeepLab	ResNet-52-DC5	Overall	61.055	84.874	70.288	699MB	0.0359s
		Ship	47.416	74.703	63.473
		Land and Sea	67.874	89.959	73.695

Install

Please refer to Installation for the official installation guide of MMdetection.

Recommended Environment

• CUDA 12.1
• PyTorch 2.1.0

# Create a new conda environment
conda create -n pansar 
conda activate pansar

Dataset

To download PsSAR-v1.0, , please refer to PsSAR-v1.0.

The PsSAR-v1.0 dataset is derived from HRSID: A High-Resolution SAR Images Dataset for Ship Detection and Instance Segmentation (GPL-3.0 license). Modifications include the proposal of panoptic segmentation annotations to support panoptic segmentation tasks. The dataset is licensed under GPL-3.0. （See README_DATASET.md）

Directory Structure of the Dataset

PsSAR-v1.0 follows the COCO panoptic segmentation standard.

Expected directory structure:

├── ps_sarship
	├── train2017
	├── val2017
	└── annotations: 
		├── instances_train2017.json		
		├── instances_val2017.json			
		├── panoptic_train2017.json			
		├── panoptic_val2017.json			
		├── panoptic_val2017	
		└── panoptic_train2017

The pipeline of annotation

PsSAR-v1.0 dataset is implemented with the Labelme tool. First, a polygon is constructed by taking the four corners of the image as key points to generate an initial mask for the sea area. Next, refined annotation is conducted on the land area to produce a land mask. The existing ship instance annotations from the HRSID dataset are directly adopted to form a set of ship masks. Finally, the land mask and all ship masks are subtracted from the initial sea mask to obtain the final sea mask.

The original JSON annotation files generated by Labelme are converted into a format compliant with the COCO panoptic segmentation annotation standard via the labelme_to_pan1.py.

Train

Dataset Preparation and Preprocessing

Ensure that the PsSAR-v1.0 is fully downloaded and correctly organized as described in the previous section. Once prepared, you can proceed with the training instructions below.

conda activate pansar
cd mmdetection
# Start training
python tools/train.py configs/panoptic_fpn/panoptic-fpn_r50_fpn_1x_ship_1.py
# python tools/train.py ${CONFIG_FILE}

Model config

model	backbone network	config
Maskformer	resnet-50	config
Panoptic FPN	resnet-50+FPN	config
	resnet101+FPN	config
Mask2former	resnet-50	config
	resnet-101	config

Inference

cd mmdetection
python tools/test.py \
    ${CONFIG_FILE} \         
    ${CHECKPOINT_FILE} \     
    --eval ${EVAL_METRIC}     

Evaluation

We used the CocoPanopticMetric evaluation metric class built into the MMDetection framework, for evaluating the performance of PsSAR-v1.0.

# for evaluation
bash mmdet/evaluation/metrics/coco_panoptic_metric.py

Acknowledgement

Sincere acknowledgment to the amazing open-source community for their great contributions:

• MMDetection: for the excellent open source object detection toolbox.
• detectron2: for the state-of-the-art detection and segmentation algorithms.
• HRSID: PsSAR-v1.0 is built upon the HRSID dataset