We don learn before about Object Detection, wey dey help us find objects for image by predicting dia bounding boxes. But for some tasks, we no only need bounding boxes, we go need more accurate way to locate objects. Dis task na segmentation.
Segmentation fit be like pixel classification, wey mean say for each pixel for image, we go predict im class (background na one of di classes). Two main segmentation algorithms dey:
- Semantic segmentation dey only tell di pixel class, e no dey separate different objects wey dey di same class.
- Instance segmentation dey divide di classes into different instances.
For instance segmentation, di sheep dem na different objects, but for semantic segmentation, all di sheep go dey under one class.
Image from this blog post
Different neural architectures dey for segmentation, but all of dem get di same structure. E be like di autoencoder wey you don learn before, but instead of breaking down di original image, our goal na to break down mask. So, segmentation network get di following parts:
- Encoder dey extract features from di input image.
- Decoder dey change di features into di mask image, wey get di same size and number of channels wey match di number of classes.
Image from this publication
We go talk about di loss function wey dem dey use for segmentation. For classical autoencoders, we dey measure di similarity between two images, and we fit use mean square error (MSE) for dat. But for segmentation, each pixel for di target mask image dey represent di class number (one-hot-encoded along di third dimension), so we go need loss functions wey dey specific for classification - cross-entropy loss, wey dem dey average over all di pixels. If di mask na binary - binary cross-entropy loss (BCE) go dey used.
✅ One-hot encoding na way to encode class label into vector wey get length wey match di number of classes. Check this article to learn about dis technique.
For dis lesson, we go see segmentation in action by training di network to recognize human nevi (wey people dey call moles) for medical images. We go use PH2 Database of dermoscopy images as di image source. Dis dataset get 200 images of three classes: typical nevus, atypical nevus, and melanoma. All di images get corresponding mask wey dey outline di nevus.
✅ Dis technique dey very good for dis type of medical imaging, but which other real-world applications you fit think of?
Image from di PH2 Database
We go train model to segment any nevus from di background.
Open di notebooks below to learn more about different semantic segmentation architectures, practice how to work with dem, and see how dem dey work.
Segmentation na strong technique for image classification, e dey go beyond bounding boxes to pixel-level classification. E dey used for medical imaging and other applications.
Body segmentation na one of di common tasks wey we fit do with images of people. Another important tasks na skeleton detection and pose detection. Try OpenPose library to see how pose detection fit work.
Dis wikipedia article dey give good overview of di different applications of dis technique. Learn more on your own about di subdomains of Instance segmentation and Panoptic segmentation for dis field.
For dis lab, try human body segmentation using Segmentation Full Body MADS Dataset from Kaggle.
Disclaimer:
Dis document don use AI translation service Co-op Translator take translate am. Even though we dey try make sure say e correct, abeg no forget say automatic translation fit get mistake or no dey accurate well. Di original document for di language wey dem write am first na di main correct one. For important information, e go better make professional human translator check am. We no go fit take blame for any misunderstanding or wrong interpretation wey fit happen because of dis translation.