Hyperspectral-Image-Segmentation

Semantic Segmentation of HyperSpectral Images using a U-Net with Separable Convolutions.

IMPORTANT UPDATE

Please do not use this codebase for your experiments.

In my opinion the results presented in this document is not a fair analysis for two reasons.

Reason one is that there is data leakage between the train and val sets. Since patches are created using stride 1, it means that some patches have partial overlap with few other patches. And also since synthetic patches are created using smaller patches, some common information maybe present across various patches. Since the general and synthetic patches are first created, then concatenated and then finally split into train and val splits, there is data leakage.

Reason two is that I think the final performance mentioned in the document is over all pixels, which naturally would include the ones used for training as well. Hence the numbers may not be representative of the effectiveness of the models.

In retrospect, I also think the idea of synthetic patches is not good. This is because the smaller patches that make up a synthetic patch belong to different parts of the full image and hence the spatial context within a synthetic patch may not be useful.

Due to the above reasons, please do not use this codebase. Since this repo was created long ago I do not remember how/why things ended up this way. I wanted to post this update for quite a while now but I kept forgetting and/or postponing. I am truly sorry for the inconvenience.

Currently there are no plans to resolve the aforementioned issues and hence the repository will be made read-only. Hence, once again, please do not use this codebase and I am truly sorry for the inconvenience. If you have any questions, please send me an email (email ID will be available in my profile).

Features

• HyperSpectral Images (HSI) are semantically segmented using two variants of U-Nets and their performance is comparaed.
• Model A uses Depthwise Separable Convolutions in the downsampling arm of the U-Net, and Model B uses Convolutions in the downsampling arm of the U-Net.
• Due to the lack of multiple HSI image and ground truth pairs, we train the models by extracting patches of the image. Here, patches are smaller square regions of the image. After training the model, we make predictions patch-wise.
• Patches are extracted using a stride of 1 for training. We used patches of size patch_size = 4 (4x4 square regions) for our experiment.
• We use the sample-weight feature offered by Keras to weight the classes in the loss function by their log frequency. We use this as there is a skew in the number of examples per class.
• Some classes do not have patches of size patch_size = 4. For these classes, we create synthetic patches of size patch_size = 4 by using patches of size 1.
• Experimental results are tabulated below.

Requirements

• Keras
• TensorFlow (>1.4)
• Scikit-Learn
• Numpy
• Scipy

Instructions

1. Clone the repository and change working directory using:

git clone https://github.com/thatbrguy/Hyperspectral-Image-Segmentation.git \
&& cd Hyperspectral-Image-Segmentation

2. Download the dataset from here or by using the following commands:

wget http://www.ehu.eus/ccwintco/uploads/6/67/Indian_pines_corrected.mat
wget http://www.ehu.eus/ccwintco/uploads/c/c4/Indian_pines_gt.mat

3. Train the model using:

python main.py \
--model A \
--mode train

The file main.py supports a few options, which are listed below:

• --model: (required) Choose between models A and B.
• --mode: (required) Choose between training (train) and inference (infer) modes.
• --weights: (required for inference only) Path of the weights file for inference mode. Pretrained weights for both models (A and B) are available in the weights directory.
• --epochs: Set the number of epochs. Default value is 100.
• --batch_size: Set the batch size. Default value is 200.
• --lr: Set the learning rate. Default value is 0.001.

Results

Indian Pines Dataset

Output Maps

Ground Truth	Model A	Model B

Epoch vs Accuracy

Tabulatation

References:

1. Dataset