東北大学 大学院情報科学研究科 情報基礎科学専攻 計算機構論分野
(東北大学 工学部 電気情報物理工学科 情報工学コース)
青木・伊藤(康)研究室

PlankFormer: Robust Plankton Instance Segmentation via MAE-Pretrained Vision Transformers and Pseudo Community Image Generation

Masaharu Miyazaki   (Tohoku University),  Yurie Otake  (Kyoto University),  Koichi Ito   (Tohoku University),  Wataru Makino  (Tohoku University),  Jotaro Urabe  (Tohoku University),  Takafumi Aoki   (Tohoku University)

International Conference on Pattern Recognition, pp. 696--710, August 2026.

Graphical Abstract
Abstract

Plankton monitoring is essential for assessing aquatic ecosystems but is limited by the labor-intensive nature of manual microscopic analysis. Automating the segmentation of plankton from crowded images is crucial, however, it faces two major challenges: (i) the scarcity of pixel-level annotated datasets and (ii) the difficulty of distinguishing plankton from debris and overlapping individuals using conventional CNN-based methods. To address these issues, we propose PlankFormer, a novel framework for plankton instance segmentation. First, to overcome the data shortage, we introduce a method to generate labeled Pseudo Community Images (PCI) by synthesizing individual plankton images onto diverse backgrounds, including those created by generative models. Second, we propose a segmentation model utilizing a Vision Transformer (ViT) backbone with a Mask2Former decoder. To robustly capture the global structural features of plankton against occlusion and debris, we employ a Masked Autoencoder (MAE) for self-supervised pretraining on unlabeled individual images. Experimental results on realworld datasets demonstrate that our method significantly outperforms conventional methods, such as Mask R-CNN, particularly in challenging environments with high debris density. We demonstrate that our synthetic training strategy and MAE-based architecture enable highprecision segmentation with requiring less manual annotations for individual plankton images.