Timely detection and treatment of micro aneurysms is a critical step to prevent the development of vision threatening eye diseases such as diabetic retinopathy. However, detecting micro aneurysms in fundus images is a highly challenging task due to the low image contrast, misleading cues of other red lesions, and the large variation of imaging conditions. Existing methods tend to fail in face of the large intra-class variation and small inter-class variations for micro aneurysm detection in fundus images. Recently, hybrid text/image mining computer aided diagnosis (CAD) systems have emerged to offer a promise of bridging the semantic gap between images and diagnostic information. This project, developed by interleaved deep mining technique to cope intelligently with the unbalanced micro aneurysm detection problem. Specifically, we propose a clinical report guided multi-sieving convolutional neural network (MS-CNN) which leverages a small amount of supervised information in clinical reports to identify the potential micro aneurysm regions via the image-to-text mapping in the feature space. These potential micro aneurysm regions are then interleaved with fundus image information for multi-sieving deep mining in a highly unbalanced classification problem. Critically, the clinical reports are employed to bridge the semantic gap between low level image features and high level diagnostic information. We build an efficient micro aneurysm detection framework based on the hybrid text/image interleaving and validate its performance on challenging clinical datasets acquired from Diabetic Retinopathy patients. Extensive evaluations are carried out in terms of fundus detection and classification. Experimental results show that our framework achieves 99.7% precision and 87.8% recall, comparing favorably with state of the art algorithms. Integration of expert domain knowledge and image information demonstrates the feasibility of reducing the difficulty of training classifiers under extremely unbalanced data distributions.

Let's Talk