Marine Debris Detection Using Planet Data
TLDRThe transcript discusses a study on marine debris detection using high-resolution satellite imagery from Planet Data. The research aimed to understand the impact of shape and spectral characteristics on feature selection for detecting marine debris, which is highly dynamic due to ocean currents. The study primarily used RGB and IR bands from Planet Scope, given the limitations of its four-band imagery. A dataset was created through a literature review and manual labeling of debris patches in the Planet Explorer tool. The chosen deep learning architecture, RetinaNet, favored speed and accuracy, utilizing a single-stage object detection approach with a feature pyramid network. The model was trained for 50 epochs, achieving an F1 score of 0.74. The research also explored the use of spectral indices but faced challenges with geographical offsets between RGB and NIR bands. The model was tested on various areas, revealing a need for greater geodiversity in the training dataset. Future work includes improving dataset diversity, developing a spectral mixing workflow, scaling model deployment, and potentially transitioning from object detection to semantic or instance segmentation for more precise area measurements and to address the challenge of long debris aggregates spanning multiple tiles.
Takeaways
- 📡 The research aimed to use small satellite imagery with high spatial and temporal resolution to monitor marine debris, which is highly dynamic due to ocean currents and weather conditions.
- 🔍 The study focused on the contributions of shape and spectral characteristics in feature selection for identifying marine debris, using Planet Scope imagery with only four bands (RGB and IR).
- 🌐 A dataset was created by conducting a literature review, locating marine debris on Planet's explorer tool, and validating through manual time series analysis.
- 🏗️ The marine debris definition was broad, including any floating material on the ocean, not solely plastics, to account for the limitations of RGB and IR imagery.
- 📈 The choice of deep learning architecture, RetinaNet, was driven by a balance between accuracy and speed, with a focus on single-stage object detection.
- 🤖 The model training process included tiling and serialization of the large Planet Scope images to fit into a machine learning workflow.
- 🔢 The data was split into 80% for training, 10% for validation, and 10% for testing, with subsampling to avoid bias towards specific conditions.
- 📉 The final model achieved an F1 score of 0.74, indicating reasonable performance for the task, with further scope for improvement.
- 🌍 Efforts were made to test the model's generalizability to other areas of interest, revealing that performance dropped in new areas, highlighting the need for more geodiversity in the training dataset.
- 🔴 The team explored the use of spectral indices like NDVI for better accuracy but faced challenges with sensor-derived offsets between RGB and NIR bands.
- 🔬 Future initiatives include improving dataset geodiversity, developing a spectral mixing workflow, scaling model deployment for real-time monitoring, and transitioning from object detection to semantic or instance segmentation for more precise area measurements.
Q & A
What is the main advantage of using small satellite imagery for monitoring marine debris?
-Small satellite imagery offers high spatial and temporal resolution, which is advantageous for monitoring marine debris due to its highly dynamic nature and constant movement influenced by ocean currents and weather conditions.
What was the primary objective of the research in investigating marine debris?
-The primary objective was to investigate the contributions of shape versus spectral characteristics in the feature selection process for detecting marine debris using small satellite imagery.
Why were the researchers limited in their choice of spectral indices for the PlanetScope imagery?
-The researchers were limited because PlanetScope imagery only has four bands (red, green, blue, and IR), and the main spectral index used for detecting marine debris, the Floating Debris Index, requires a shortwave infrared band which is not present in PlanetScope.
How did the researchers create the dataset for training the model?
-The researchers conducted a literature review, identified marine debris locations using Planet's explorer tool by co-locating dates and locations from previous studies, verified the temporal nature of marine debris through manual time series analysis, and hand annotated the debris patches using NASA's Image Labor tool.
What is the definition of marine debris in the context of this project?
-In this project, marine debris is defined as any aggregate of material floating on the ocean, which includes not only plastics but also vegetation, wood, algae, or any other material that is dynamically floating and could potentially carry plastics.
Why did the researchers choose to use object detection instead of segmentation for the model?
-The researchers chose object detection over segmentation because creating bounding boxes for the features was significantly faster than performing a granular segmentation, especially since they had to create the dataset themselves due to the lack of existing geospatial ground truth datasets for marine debris.
What is the significance of the Feature Pyramid Network in the Retina architecture?
-The Feature Pyramid Network is significant as it produces feature maps at multiple scales, allowing the network to capture different levels of information at various scales. This enables the model to detect objects of different sizes within an image.
How does the Retina architecture address the class imbalance problem between foreground and background?
-Retina uses focal loss, which is an extension of cross-entropy loss with a tuning parameter that helps the model focus more on difficult examples (foreground objects) and less on easy examples (background), thus addressing the class imbalance problem.
What was the F1 score achieved by the final model?
-The final model produced an F1 score of 0.74, which is considered acceptable for the task at hand.
Why was it challenging to use the near-infrared band for detecting marine debris in the study?
-It was challenging to use the near-infrared band because of sensor-derived and spatial registration offsets between the NIR band and RGB data. These offsets led to misalignments that could not be corrected over the open ocean, thus complicating learning and accuracy.
What are the future initiatives the researchers are considering to improve the model?
-The future initiatives include improving dataset geodiversity, developing a spectral mixing workflow for validating the presence of plastic, scaling model deployment and inference for real-time monitoring, and transitioning from object detection to semantic or instance segmentation to refine area measurements and address the problem of long debris aggregates spanning multiple tiles.
Outlines
📡 Marine Debris Monitoring with Small Satellite Imagery
The video discusses the use of small satellite imagery to monitor marine debris due to its high spatial and temporal resolution. The research aims to understand the impact of shape and spectral characteristics in feature selection for marine debris detection. The study also explores the use of Planet Scope imagery, which has limitations in spectral indices due to only having four bands (RGB and IR), thus focusing on indices like NDVI. The choice of imagery was crucial due to the dynamic nature of marine debris, and the research leveraged deep learning for rapid deployment.
🌐 Creating a Dataset for Marine Debris Detection
The process of creating a dataset for marine debris detection involved a literature review, identification of relevant studies, and manual location of marine debris using Planet's explorer tool. The dataset included various materials floating on the ocean, not limited to plastics. The labeling process ensured each debris patch was covered diagonally to fit within a rectangle for object detection. The study also collected river discharge images for potential use, labeling them with NASA's Image Labor tool.
🔍 Tiling and Serialization for Machine Learning
To prepare the data for machine learning, the high-resolution Planet Scope images and their associated labels were tiled to manage memory constraints and enable ingestion into the machine learning workflow. The output was a set of tiled images with map coordinates and labels in a compressed array format, ready for serialization into TensorFlow records. The data was also partitioned into training, validation, and testing sets with an 80-10-10 split, ensuring no bias towards specific conditions.
🛠️ Object Detection Architecture and Training
The video introduces the choice of the RetinaNet architecture for object detection, a single-stage approach that balances accuracy and speed. The feature pyramid network captures features at multiple scales, and the model uses a default set of bounding boxes that are adjusted to fit the ground truth during training. Focal loss is employed to focus on difficult examples and ignore easy ones, which is particularly useful for single-class problems like marine debris detection. The model was trained for 50 epochs with various augmentations and achieved an F1 score of 0.74.
🌌 Addressing Geospatial Challenges in Detection
The output of the model consists of geospatially naive bounding boxes that require conversion back to a geospatial context using slippy map information. The process involves converting the coordinates into a standard projection system like WGS 84. The video also discusses the attempt to use spectral indices for detecting marine debris with Planet Scope imagery but encountered issues due to sensor-derived and spatial registration offsets between the NIR band and RGB.
🌍 Generalizing the Model for Global Applications
The model was trained on imagery from diverse locations to ensure geodiversity and tested on various areas of interest, including Jakarta Bay and Manila Bay. The results showed that the model's performance varied across different locations, indicating the need for further training and data collection to improve its generalizability. The video also highlights the challenge of differentiating marine debris from other linear features like vessels, which appear problematic due to their color and could be addressed by including additional classes for false positives in future iterations.
🛑 Future Initiatives and Model Improvements
The video outlines future initiatives, including improving dataset geodiversity, developing a spectral mixing workflow to validate the presence of plastic, scaling model deployment for real-time monitoring, and transitioning from object detection to semantic or instance segmentation for more accurate area measurements and to address the challenge of long debris aggregates spanning multiple tiles. The researchers are also collaborating with others to increase diversity and are open to contributions, as all their work is open source.
Mindmap
Keywords
💡Marine Debris
💡Planet Data
💡Spectral Characteristics
💡Feature Pyramid Network
💡Object Detection
💡Focal Loss
💡Geospatial Data
💡Temporal Resolution
💡Spectral Indices
💡Semantic Segmentation
💡Instance Segmentation
Highlights
The study aims to monitor marine debris using high spatial and temporal resolution imagery from small satellites, which is advantageous due to the dynamic nature of marine debris movement.
The research investigates the contribution of shape versus spectral characteristics in feature selection for detecting marine debris aggregates influenced by ocean currents and fronts.
The study explores the use of Planet Scope imagery with only four bands (RGB and IR), limiting the choice of spectral indices due to the absence of a short wave infrared band.
The choice of Planet Scope imagery was crucial for the research due to its high spatial and temporal resolution, offering competitive advantages for dynamic environment monitoring.
A deep learning approach coupled with high-resolution imagery allows for rapid deployment in monitoring marine debris.
The dataset was created by conducting a literature review and using data from boat expeditions, focusing on areas like the Bay Islands of Honduras.
Marine debris is defined broadly in this study, including any floating aggregates such as vegetation, wood, algae, and plastics.
The study identified 65 PlanetScope scene IDs and approximately 1300 debris patches, using manual time series analysis to confirm the dynamic nature of marine debris.
The labeling process ensured that each debris patch was fully covered by the bounding box to cater to the object detection model's requirements.
The study used the Retina object detection architecture, a single-stage detection network that balances accuracy and speed.
The feature pyramid network captures features at multiple scales, allowing the model to detect objects of various sizes.
The model training utilized focal loss to address class imbalance and improve the focus on difficult examples.
The final model achieved an F1 score of 0.74, indicating reasonable performance for the detection of marine debris.
The study faced challenges with geographical offsets between the RGB and near-infrared bands of Planet Scope imagery, limiting the use of certain spectral indices.
The model was tested on new areas of interest, such as the Citarum River in Indonesia and the Pasig River in the Philippines, to evaluate its generalizability.
The output of the model includes geospatially aware bounding boxes that can be converted back into a standard geographic projection for further analysis.
Future initiatives include improving dataset geodiversity, developing a spectral mixing workflow, scaling model deployment, and transitioning from object detection to semantic segmentation.
The study highlights the potential of using satellite imagery and deep learning for rapid monitoring and alerting of marine debris, with applications in environmental conservation and plastic pollution mitigation.