A survey on deep learning in medical image analysis
Medical Image AnalysisPublished 26 July 2017Open access
Geert Litjens, Thijs Kooi, Babak Ehteshami Bejnordi, Arnaud A. A. Setio, Francesco Ciompi, Mohsen Ghafoorian
Citations14,332
SJR quartileQ1
SJR score3.29
SNIP3.91
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
This paper reviews the major deep learning concepts pertinent to medical image analysis and summarizes over 300 contributions to the field, most of which appeared in the last year, to survey the use of deep learning for image classification, object detection, segmentation, registration, and other tasks.
Abstract
Deep learning algorithms, in particular convolutional networks, have rapidly become a methodology of choice for analyzing medical images. This paper reviews the major deep learning concepts pertinent to medical image analysis and summarizes over 300 contributions to the field, most of which appeared in the last year. We survey the use of deep learning for image classification, object detection, segmentation, registration, and other tasks. Concise overviews are provided of studies per application area: neuro, retinal, pulmonary, digital pathology, breast, cardiac, abdominal, musculoskeletal. We end with a summary of the current state-of-the-art, a critical discussion of open challenges and directions for future research.
Keywords
Computer ScienceMedicineEngineering
Deep Residual Learning for Image Recognition
222,082 Citations2016Kaiming He, Xiangyu Zhang +2 more
This work presents a residual learning framework to ease the training of networks that are substantially deeper than those used previously, and provides comprehensive empirical evidence showing that these residual networks are easier to optimize, and can gain accuracy from considerably increased depth.
Proceedings of the IEEEGradient-based learning applied to document recognition
58,219 Citations1998Yann LeCun, Léon Bottou +2 more
This paper reviews various methods applied to handwritten character recognition and compares them on a standard handwritten digit recognition task, and Convolutional neural networks are shown to outperform all other techniques.
International Journal of Computer VisionImageNet Large Scale Visual Recognition Challenge
40,207 Citations2015Olga Russakovsky, Jia Deng +10 more
The creation of this benchmark dataset and the advances in object recognition that have been possible as a result are described, and the state-of-the-art computer vision accuracy with human accuracy is compared.
Neural ComputationA Fast Learning Algorithm for Deep Belief Nets
16,404 Citations2006Geoffrey E. Hinton, Simon Osindero +1 more
A fast, greedy algorithm is derived that can learn deep, directed belief networks one layer at a time, provided the top two layers form an undirected associative memory.
UvA-DARE (University of Amsterdam)Auto-Encoding Variational Bayes
15,628 Citations2013Diederik P. Kingma, Max Welling
A stochastic variational inference and learning algorithm that scales to large datasets and, under some mild differentiability conditions, even works in the intractable case is introduced.
Lecture notes in computer scienceIdentity Mappings in Deep Residual Networks
10,094 Citations2016Kaiming He, Xiangyu Zhang +2 more
The propagation formulations behind the residual building blocks suggest that the forward and backward signals can be directly propagated from one block to any other block, when using identity mappings as the skip connections and after-addition activation.
IEEE Transactions on Neural NetworksLearning long-term dependencies with gradient descent is difficult
8,461 Citations1994Yoshua Bengio, P. Simard +1 more
This work shows why gradient based learning algorithms face an increasingly difficult problem as the duration of the dependencies to be captured increases, and exposes a trade-off between efficient learning by gradient descent and latching on information for long periods.
Random search for hyper-parameter optimization
7,937 Citations2012James Bergstra, Yoshua Bengio
JAMADevelopment and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs
7,152 Citations2016Varun Gulshan, Lily Peng +13 more
An algorithm based on deep machine learning had high sensitivity and specificity for detecting referable diabetic retinopathy and diabetic macular edema in retinal fundus photographs from adults with diabetes.
IEEE Transactions on Medical ImagingDeep Convolutional Neural Networks for Computer-Aided Detection: CNN Architectures, Dataset Characteristics and Transfer Learning
5,664 Citations2016Hoo-Chang Shin, Holger R. Roth +7 more
Two specific computer-aided detection problems, namely thoraco-abdominal lymph node (LN) detection and interstitial lung disease (ILD) classification are studied, achieving the state-of-the-art performance on the mediastinal LN detection, and the first five-fold cross-validation classification results are reported.
arXiv (Cornell University)Practical Bayesian Optimization of Machine Learning Algorithms
5,612 Citations2012Jasper Snoek, Hugo Larochelle +1 more
This work describes new algorithms that take into account the variable cost of learning algorithm experiments and that can leverage the presence of multiple cores for parallel experimentation and shows that these proposed algorithms improve on previous automatic procedures and can reach or surpass human expert-level optimization for many algorithms.
Deep visual-semantic alignments for generating image descriptions
4,978 Citations2015Andrej Karpathy, Li Fei-Fei
A model that generates natural language descriptions of images and their regions using a novel combination of Convolutional Neural Networks over image regions, bidirectional Recurrent Neural Networks over sentences, and a structured objective that aligns the two modalities through a multimodal embedding is presented.
Annual Review of Biomedical EngineeringDeep Learning in Medical Image Analysis
4,788 Citations2017Dinggang Shen, Guorong Wu +1 more
This review covers computer-assisted analysis of images in the field of medical imaging and introduces the fundamentals of deep learning methods and their successes in image registration, detection of anatomical and cellular structures, tissue segmentation, computer-aided disease diagnosis and prognosis, and so on.
The MIT Press eBooksGreedy Layer-Wise Training of Deep Networks
4,704 Citations2007Yoshua Bengio, Pascal Lamblin +2 more
These experiments confirm the hypothesis that the greedy layer-wise unsupervised training strategy mostly helps the optimization, by initializing weights in a region near a good local minimum, giving rise to internal distributed representations that are high-level abstractions of the input, bringing better generalization.
arXiv (Cornell University)Caffe: Convolutional Architecture for Fast Feature Embedding
4,290 Citations2014Yangqing Jia, Evan Shelhamer +6 more
Journal of Machine Learning ResearchStacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion
4,118 Citations2010VincentPascal, LarochelleHugo +3 more
This work clearly establishes the value of using a denoising criterion as a tractable unsupervised objective to guide the learning of useful higher level representations.
arXiv (Cornell University)Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
3,282 Citations2014Kyunghyun Cho, Bart van Merriënboer +5 more
Medical Image AnalysisBrain tumor segmentation with Deep Neural Networks
3,245 Citations2016Mohammad Havaei, Axel Davy +7 more
A fast and accurate fully automatic method for brain tumor segmentation which is competitive both in terms of accuracy and speed compared to the state of the art, and introduces a novel cascaded architecture that allows the system to more accurately model local label dependencies.
IEEE Transactions on Medical ImagingConvolutional Neural Networks for Medical Image Analysis: Full Training or Fine Tuning?
3,044 Citations2016Nima Tajbakhsh, J. Shin +5 more
This paper considered four distinct medical imaging applications in three specialties involving classification, detection, and segmentation from three different imaging modalities, and investigated how the performance of deep CNNs trained from scratch compared with the pre-trained CNNs fine-tuned in a layer-wise manner.
Lecture notes in computer scienceA Practical Guide to Training Restricted Boltzmann Machines
2,802 Citations2012Geoffrey E. Hinton
This guide is an attempt to share expertise at training restricted Boltzmann machines with other machine learning researchers.
IEEE Transactions on Medical ImagingBrain Tumor Segmentation Using Convolutional Neural Networks in MRI Images
2,678 Citations2016Sérgio Pereira, Adriano Pinto +2 more
This paper proposes an automatic segmentation method based on Convolutional Neural Networks (CNN), exploring small 3 ×3 kernels, which allows designing a deeper architecture, besides having a positive effect against overfitting, given the fewer number of weights in the network.
arXiv (Cornell University)Striving for Simplicity: The All Convolutional Net
2,603 Citations2014Jost Tobias Springenberg, Alexey Dosovitskiy +2 more
IEEE Transactions on Medical ImagingGuest Editorial Deep Learning in Medical Imaging: Overview and Future Promise of an Exciting New Technique
1,673 Citations2016Hayit Greenspan, Bram van Ginneken +1 more
The papers in this special section focus on the technology and applications supported by deep learning, which have proven to be powerful tools for a broad range of computer vision tasks.
Lecture notes in computer scienceMitosis Detection in Breast Cancer Histology Images with Deep Neural Networks
1,556 Citations2013Dan Cireşan, Alessandro Giusti +2 more
This work uses deep max-pooling convolutional neural networks to detect mitosis in breast histology images using as context a patch centered on the pixel to classify each pixel in the images.
arXiv (Cornell University)Going Deeper with Convolutions
1,381 Citations2014Christian Szegedy, Wei Liu +7 more
Pattern RecognitionExplaining nonlinear classification decisions with deep Taylor decomposition
1,354 Citations2016Grégoire Montavon, Sebastian Lapuschkin +3 more
A novel methodology for interpreting generic multilayer neural networks by decomposing the network classification decision into contributions of its input elements by backpropagating the explanations from the output to the input layer is introduced.
IEEE Transactions on Medical ImagingLung Pattern Classification for Interstitial Lung Diseases Using a Deep Convolutional Neural Network
1,312 Citations2016Marios Anthimopoulos, Stergios Christodoulidis +3 more
A comparative analysis proved the effectiveness of the proposed CNN against previous methods in a challenging dataset, and demonstrated the potential of CNNs in analyzing lung patterns.
Neural Information Processing SystemsDeep Neural Networks Segment Neuronal Membranes in Electron Microscopy Images
1,280 Citations2012Dan Cireşan, Alessandro Giusti +2 more
This work addresses a central problem of neuroanatomy, namely, the automatic segmentation of neuronal structures depicted in stacks of electron microscopy images, using a special type of deep artificial neural network as a pixel classifier to segment biological neuron membranes.
Journal of Pathology InformaticsDeep learning for digital pathology image analysis: A comprehensive tutorial with selected use cases
1,262 Citations2016Andrew Janowczyk, Anant Madabhushi
This paper investigates concepts through seven unique DP tasks as use cases to elucidate techniques needed to produce comparable, and in many cases, superior to results from the state-of-the-art hand-crafted feature-based classification approaches.
IEEE Transactions on Medical ImagingPulmonary Nodule Detection in CT Images: False Positive Reduction Using Multi-View Convolutional Networks
1,247 Citations2016Arnaud A. A. Setio, Francesco Ciompi +8 more
IEEE Transactions on Medical ImagingLocality Sensitive Deep Learning for Detection and Classification of Nuclei in Routine Colon Cancer Histology Images
1,240 Citations2016Korsuk Sirinukunwattana, Shan E Ahmed Raza +4 more
A Spatially Constrained Convolutional Neural Network (SC-CNN) to perform nucleus detection and a novel Neighboring Ensemble Predictor (NEP) coupled with CNN to more accurately predict the class label of detected cell nuclei are proposed.
IEEE Transactions on Medical ImagingAutomated Melanoma Recognition in Dermoscopy Images via Very Deep Residual Networks
1,110 Citations2016Lequan Yu, Hao Chen +3 more
This study corroborates that very deep CNNs with effective training mechanisms can be employed to solve complicated medical image analysis tasks, even with limited training data.
Investigative Ophthalmology & Visual ScienceImproved Automated Detection of Diabetic Retinopathy on a Publicly Available Dataset Through Integration of Deep Learning
1,065 Citations2016Michael D. Abràmoff, Yiyue Lou +5 more
A deep-learning enhanced algorithm for the automated detection of DR, achieves significantly better performance than a previously reported, otherwise essentially identical, algorithm that does not employ deep learning.
Lecture notes in computer scienceThe Importance of Skip Connections in Biomedical Image Segmentation
960 Citations2016Michal Drozdzal, Eugene Vorontsov +3 more
This paper extends Fully Convolutional Networks by adding short skip connections, that are similar to the ones introduced in residual networks, in order to build very deep FCNs (of hundreds of layers).
IEEE Transactions on Medical ImagingEndoNet: A Deep Architecture for Recognition Tasks on Laparoscopic Videos
959 Citations2016Andru Putra Twinanda, Sherif Shehata +4 more
This paper proposes a novel method for phase recognition that uses a convolutional neural network (CNN) to automatically learn features from cholecystectomy videos and that relies uniquely on visual information.
IEEE Transactions on Medical ImagingStacked Sparse Autoencoder (SSAE) for Nuclei Detection on Breast Cancer Histopathology Images
851 Citations2015Jun Xu, Lei Xiang +5 more
A Stacked Sparse Autoencoder, an instance of a deep learning strategy, is presented for efficient nuclei detection on high-resolution histopathological images of breast cancer and out-performed nine other state of the art nuclear detection strategies.
arXiv (Cornell University)Deep Learning for Identifying Metastatic Breast Cancer
800 Citations2016D. Wang, Aditya Khosla +3 more
The power of using deep learning to produce significant improvements in the accuracy of pathological diagnoses is demonstrated, by combining the deep learning system's predictions with the human pathologist's diagnoses.
IEEE Transactions on Medical ImagingAutomatic Segmentation of MR Brain Images With a Convolutional Neural Network
776 Citations2016Pim Moeskops, Max A. Viergever +4 more
This paper presents a method for the automatic segmentation of MR brain images into a number of tissue classes using a convolutional neural network, and demonstrates its robustness to differences in age and acquisition protocol.
Scientific ReportsComputer-Aided Diagnosis with Deep Learning Architecture: Applications to Breast Lesions in US Images and Pulmonary Nodules in CT Scans
750 Citations2016Jie-Zhi Cheng, Dong Ni +7 more
The experimental results show the significant performance boost by the SDAE-based CADx algorithm over the two conventional methods, suggesting that deep learning techniques can potentially change the design paradigm of the CADx systems without the need of explicit design and selection of problem-oriented features.
NeuroImageBrainNetCNN: Convolutional neural networks for brain networks; towards predicting neurodevelopment
727 Citations2016Jeremy Kawahara, Colin J. Brown +6 more
The BrainNetCNN framework is applied to predict cognitive and motor developmental outcome scores from structural brain networks of infants born preterm and outperforms a fully connected neural‐network with the same number of model parameters on both phantoms with focal and diffuse injury patterns.
Lecture notes in computer scienceDeepOrgan: Multi-level Deep Convolutional Networks for Automated Pancreas Segmentation
686 Citations2015Holger R. Roth, Le Lü +5 more
This paper presents a probabilistic bottom-up approach for pancreas segmentation in abdominal computed tomography CT scans, using multi-level deep convolutional networks ConvNets, and proposes and evaluates several variations of deep ConvNETS in the context of hierarchical, coarse-to-fine classification on image patches and regions, i.e. superpixels.
IEEE Transactions on Medical ImagingAutomatic Detection of Cerebral Microbleeds From MR Images via 3D Convolutional Neural Networks
672 Citations2016Qi Dou, Hao Chen +7 more
This paper proposes a novel automatic method to detect CMBs from magnetic resonance (MR) images by exploiting the 3D convolutional neural network (CNN), outperforming previous methods using low-level descriptors or 2D CNNs by a significant margin.
Lecture notes in computer scienceAutomatic Liver and Lesion Segmentation in CT Using Cascaded Fully Convolutional Neural Networks and 3D Conditional Random Fields
625 Citations2016Patrick Ferdinand Christ, Mohamed Ezzeldin A. Elshaer +11 more
The results show that CFCN-based semantic liver and lesion segmentation achieves Dice scores over \(94\,\%\) for liver with computation times below 100 s per volume.
IEEE Transactions on Medical ImagingAggNet: Deep Learning From Crowds for Mitosis Detection in Breast Cancer Histology Images
611 Citations2016Shadi Albarqouni, Christoph Baur +4 more
An experimental study on learning from crowds that handles data aggregation directly as part of the learning process of the convolutional neural network (CNN) via additional crowdsourcing layer (AggNet), which gives valuable insights into the functionality of deep CNN learning from crowd annotations and proves the necessity of data aggregation integration.
Medical Image AnalysisA combined deep-learning and deformable-model approach to fully automatic segmentation of the left ventricle in cardiac MRI
595 Citations2016M. R. Avendi, Arash Kheradvar +1 more
This work employs deep learning algorithms combined with deformable models to develop and evaluate a fully automatic LV segmentation tool from short-axis cardiac MRI datasets and shows that it outperforms the state-of-the art methods.
IEEE Transactions on Biomedical EngineeringMultilevel Contextual 3-D CNNs for False Positive Reduction in Pulmonary Nodule Detection
595 Citations2016Qi Dou, Hao Chen +3 more
A novel method employing three-dimensional convolutional neural networks for false positive reduction in automated pulmonary nodule detection from volumetric computed tomography (CT) scans and a simple yet effective strategy to encode multilevel contextual information to meet the challenges coming with the large variations and hard mimics of pulmonary nodules.
IEEE Transactions on Medical ImagingImproving Computer-Aided Detection UsingConvolutional Neural Networks and Random View Aggregation
589 Citations2015Holger R. Roth, Le Lü +6 more
Experimental results show the ability of ConvNets to generalize well to different medical imaging CADe applications and scale elegantly to various data sets.
Lecture notes in computer scienceDeep Feature Learning for Knee Cartilage Segmentation Using a Triplanar Convolutional Neural Network
570 Citations2013Adhish Prasoon, Kersten Petersen +4 more
A novel system for voxel classification integrating three 2D CNNs, which have a one-to-one association with the xy, yz and zx planes of 3D image, respectively, which performs better than a state-of-the-art method using 3D multi-scale features.
Lecture notes in computer scienceMulti-scale Convolutional Neural Networks for Lung Nodule Classification
562 Citations2015Wei Shen, Mu Zhou +3 more
A hierarchical learning framework--Multi-scale Convolutional Neural Networks (MCNN)-- is proposed to capture nodule heterogeneity by extracting discriminative features from alternatingly stacked layers to sufficiently quantify nodule characteristics.
DCAN: Deep Contour-Aware Networks for Accurate Gland Segmentation
552 Citations2016Hao Chen, Xiaojuan Qi +2 more
An efficient deep contour-aware network (DCAN) to solve this challenging problem under a unified multi-task learning framework and can be efficient when applied to large-scale histopathological data without resorting to additional steps to generate contours based on low-level cues for post-separating.
arXiv (Cornell University)Detecting Cancer Metastases on Gigapixel Pathology Images
521 Citations2017Yun Liu, Krishna Gadepalli +11 more
This work presents a framework to automatically detect and localize tumors as small as 100 x 100 pixels in gigapixel microscopy images sized 100,000 x100,000 pixels and achieves image-level AUC scores above 97% on both the Camelyon16 test set and an independent set of 110 slides.
NeuroImageDeep MRI brain extraction: A 3D convolutional neural network for skull stripping
515 Citations2016Jens Kleesiek, Gregor Urban +5 more
Medical Image AnalysisDCAN: Deep contour-aware networks for object instance segmentation from histology images
504 Citations2016Hao Chen, Xiaojuan Qi +4 more
A novel deep contour‐aware network (DCAN) under a unified multi‐task learning framework for more accurate detection and segmentation of objects of interest from histology images is proposed.
Lecture notes in computer scienceDeep Learning Based Imaging Data Completion for Improved Brain Disease Diagnosis
486 Citations2014Rongjian Li, Wenlu Zhang +5 more
This work proposed a deep learning based framework for estimating multi-modality imaging data in the form of convolutional neural networks, where the input and output are two volumetric modalities.
Lecture notes in computer scienceDeepVessel: Retinal Vessel Segmentation via Deep Learning and Conditional Random Field
451 Citations2016Huazhu Fu, Yanwu Xu +3 more
This paper formulate the retinal vessel segmentation problem as a boundary detection task and solve it using a novel deep learning architecture based on a multi-scale and multi-level Convolutional Neural Network with a side-output layer to learn a rich hierarchical representation.
IEEE Transactions on Medical ImagingUnsupervised Deep Learning Applied to Breast Density Segmentation and Mammographic Risk Scoring
437 Citations2016Michiel Kallenberg, Kersten Petersen +9 more
The state-of-the-art results show that the learned breast density scores have a very strong positive relationship with manual ones, and that the learning texture scores are predictive of breast cancer.
IEEE Journal of Biomedical and Health InformaticsMultimodal Neuroimaging Feature Learning With Multimodal Stacked Deep Polynomial Networks for Diagnosis of Alzheimer's Disease
434 Citations2017Jun Shi, Zheng Xiao +3 more
Experimental results indicate that MM-SDPN is superior over the state-of-the-art multimodal feature-learning-based algorithms for AD diagnosis.
IEEE Transactions on Medical ImagingFast Convolutional Neural Network Training Using Selective Data Sampling: Application to Hemorrhage Detection in Color Fundus Images
423 Citations2016Mark J. J. P. van Grinsven, Bram van Ginneken +3 more
This paper proposes a method to improve and speed-up the CNN training for medical image analysis tasks by dynamically selecting misclassified negative samples during training using the selective sampling method.
IEEE Transactions on Medical ImagingClassification of mass and normal breast tissue: a convolution neural network classifier with spatial domain and texture images
418 Citations1996Berkman Sahiner, Heang‐Ping Chan +5 more
The authors' results demonstrate the feasibility of using a convolution neural network for classification of masses and normal tissue on mammograms using a generalized, fast and stable implementation of the CNN.
arXiv (Cornell University)A New 2.5D Representation for Lymph Node Detection using Random Sets of Deep Convolutional Neural Network Observations
417 Citations2014Holger R. Roth, Le Lü +7 more
International Journal of Neural SystemsEnsembles of Deep Learning Architectures for the Early Diagnosis of the Alzheimer’s Disease
406 Citations2016Andrés Ortíz, Jorge Munilla +2 more
Lecture notes in computer scienceA Deep Learning Architecture for Image Representation, Visual Interpretability and Automated Basal-Cell Carcinoma Cancer Detection
400 Citations2013Ángel Cruz-Roa, John Arévalo +2 more
A deep learning architecture for automated basal cell carcinoma cancer detection that integrates image representation learning, image classification and result interpretability is presented and Experimental results show that the representation learned from a large histology image data set has the best overall performance.
Computer Vision and Image UnderstandingHough-CNN: Deep learning for segmentation of deep brain regions in MRI and ultrasound
376 Citations2017Fausto Milletarì, Seyed‐Ahmad Ahmadi +9 more
A novel approach to perform segmentation by leveraging the abstraction capabilities of convolutional neural networks (CNNs) based on Hough voting, which is robust, multi-region, flexible and can be easily adapted to different modalities is proposed.
IEEE Journal of Biomedical and Health InformaticsAutomatic Detection and Classification of Colorectal Polyps by Transferring Low-Level CNN Features From Nonmedical Domain
370 Citations2016Ruikai Zhang, Yali Zheng +5 more
Automatic algorithms can assist endoscopists in identifying polyps that are adenomatous but have been incorrectly judged as hyperplasia and, therefore, enable timely resection of these polyps at an early stage before they develop into invasive cancer.
Computerized Medical Imaging and GraphicsGlaucoma detection using entropy sampling and ensemble learning for automatic optic cup and disc segmentation
367 Citations2016Julian Zilly, Joachim M. Buhmann +1 more
This work presents a novel method to segment retinal images using ensemble learning based convolutional neural network (CNN) architectures, which outperforms existing methods on the public DRISHTI-GS data set on several metrics.
NeuroImageDeep neural network with weight sparsity control and pre-training extracts hierarchical features and enhances classification performance: Evidence from whole-brain resting-state functional connectivity patterns of schizophrenia
360 Citations2015JungHoe Kim, Vince D. Calhoun +2 more
The proposed schemes and reported findings attained by using the DNN classifier and whole-brain FC data suggest that such approaches show improved ability to learn hidden patterns in brain imaging data, which may be useful for developing diagnostic tools for SZ and other neuropsychiatric disorders and identifying associated aberrant FC patterns.
Proceedings of the AAAI Conference on Artificial IntelligenceVolumetric ConvNets with Mixed Residual Connections for Automated Prostate Segmentation from 3D MR Images
353 Citations2017Lequan Yu, Xin Yang +3 more
The proposed volumetric convolutional neural network with mixed residual connections is general enough and can be easily extended to other medical image analysis tasks, especially ones with limited training data.
Lecture notes in computer scienceDeep Learning and Data Labeling for Medical Applications
351 Citations2016Julien Cornebise, Jaime S. Cardoso +10 more
The proposed K-support spatial pooling strategy in deep CNNs combines the popularly applied mean and max pooling methods, and then avoids awfully emphasizing of the maximum activation but preferring a group of activations in a supported region.
Deep features to classify skin lesions
336 Citations2016Jeremy Kawahara, Aïcha BenTaieb +1 more
It is demonstrated that a linear classifier, trained on features extracted from a convolutional neural network pretrained on natural images, distinguishes among up to ten skin lesions with a higher accuracy than previously published state-of-the-art results on the same dataset.
arXiv (Cornell University)Recent Advances in Convolutional Neural Networks
328 Citations2015Jiuxiang Gu, Zhenhua Wang +10 more
arXiv (Cornell University)A Fully Convolutional Neural Network for Cardiac Segmentation in Short-Axis MRI
321 Citations2016Phi Vu Tran
This work proposes to tackle the problem of automated left and right ventricle segmentation through the application of a deep fully convolutional neural network architecture for pixel-wise labeling in cardiac magnetic resonance imaging.
Medical Image AnalysisAutomatic coronary artery calcium scoring in cardiac CT angiography using paired convolutional neural networks
304 Citations2016Jelmer M. Wolterink, Tim Leiner +4 more
CAC can be accurately automatically identified and quantified in CCTA using the proposed pattern recognition method, which might obviate the need to acquire a dedicated CSCT scan for CAC scoring, and thus reduce the CT radiation dose received by patients.
NeuroImageState-space model with deep learning for functional dynamics estimation in resting-state fMRI
303 Citations2016Heung‐Il Suk, Chong-Yaw Wee +2 more
This paper proposes a novel methodological architecture that combines deep learning and state-space modelling, and applies it to rs-fMRI based Mild Cognitive Impairment (MCI) diagnosis, and designs a Deep Auto-Encoder to discover hierarchical non-linear functional relations among regions, which transform the regional features into an embedding space, whose bases are complex functional networks.
IEEE Transactions on Medical Imagingq-Space Deep Learning: Twelve-Fold Shorter and Model-Free Diffusion MRI Scans
302 Citations2016Vladimir Golkov, Alexey Dosovitskiy +6 more
It is demonstrated how deep learning, a group of algorithms based on recent advances in the field of artificial neural networks, can be applied to reduce diffusion MRI data processing to a single optimized step and how classical data processing can be streamlined by means of deep learning.
Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIEDeep learning with non-medical training used for chest pathology identification
295 Citations2015Yaniv Bar, Idit Diamant +2 more
This is a first-of-its-kind experiment that shows that deep learning with large scale non-medical image databases may be sufficient for general medical image recognition tasks.
Lecture notes in computer science3D Deep Learning for Multi-modal Imaging-Guided Survival Time Prediction of Brain Tumor Patients
287 Citations2016Dong Nie, Han Zhang +3 more
3D convolutional neural networks are adopted and a new network architecture is proposed for using multi-channel data and learning supervised features to automatically extract features from multi-modal preoperative brain images of high-grade glioma patients to predict overall survival time.
International Journal of Computer Assisted Radiology and SurgeryAutomatic 3D liver location and segmentation via convolutional neural network and graph cut
286 Citations2016Lu Fang, Fa Wu +3 more
Retinal vessel segmentation via deep learning network and fully-connected conditional random fields
285 Citations2016Huazhu Fu, Yanwu Xu +2 more
This paper formulate the vessel segmentation to a boundary detection problem, and utilize the fully convolutional neural networks (CNNs) to generate a vessel probability map that distinguishes the vessels and background in the inadequate contrast region and has robustness to the pathological regions in the fundus image.
Computerized Medical Imaging and GraphicsEnhancing deep convolutional neural network scheme for breast cancer diagnosis with unlabeled data
277 Citations2016Wenqing Sun, Tzu-Liang Tseng +2 more
A graph based semi-supervised learning (SSL) scheme using deep convolutional neural network (CNN) for breast cancer diagnosis using data weighing, feature selection, dividing co-training data labeling, and CNN.
arXiv (Cornell University)Practical recommendations for gradient-based training of deep architectures
267 Citations2012Yoshua Bengio
UltrasonicsA pre-trained convolutional neural network based method for thyroid nodule diagnosis
260 Citations2016Jinlian Ma, Fa Wu +3 more
Experimental results show that the proposed CNN based methods can accurately and effectively diagnose thyroid nodules in ultrasound and lead to significant performance improvement, with an accuracy of 83.02% ± 0.72%.
Lecture notes in computer scienceEstimating CT Image from MRI Data Using 3D Fully Convolutional Networks
259 Citations2016Dong Nie, Xiaohuan Cao +3 more
A 3D fully convolutional neural network is adopted to learn an end-to-end nonlinear mapping from MR image to CT image and this method is accurate and robust for predicting CT image from MRI image, and also outperforms three state-of-the-art methods under comparison.
Lecture notes in computer scienceHeMIS: Hetero-Modal Image Segmentation
256 Citations2016Mohammad Havaei, Nicolas Guizard +2 more
A deep learning image segmentation framework that is extremely robust to missing imaging modalities, which learns, for each modality, an embedding of the input image into a single latent vector space for which arithmetic operations are well defined.
IEEE Transactions on Medical ImagingDeformable MR Prostate Segmentation via Deep Feature Learning and Sparse Patch Matching
249 Citations2015Yanrong Guo, Yaozong Gao +1 more
This work proposes a new deformable MR prostate segmentation method by unifying deep feature learning with the sparse patch matching, and shows superior performance than other state-of-the-art segmentation methods.
UltrasonicsDeep learning based classification of breast tumors with shear-wave elastography
237 Citations2016Qi Zhang, Yang Xiao +5 more
A deep learning architecture for automated extraction of learned-from-data image features from the shear-wave elastography (SWE) that integrates feature learning with feature selection on SWE is built and may be potentially used in clinical computer-aided diagnosis of breast cancer.
Lecture notes in computer scienceRegressing Heatmaps for Multiple Landmark Localization Using CNNs
232 Citations2016Christian Payer, Darko Štern +2 more
Evaluation of different architectures on 2D and 3D hand image datasets show that heatmap regression based on CNNs achieves state-of-the-art landmark localization performance, with SpatialConfiguration-Net being robust even in case of limited amounts of training data.
Medical PhysicsAutomated detection of pulmonary nodules in PET/CT images: Ensemble false‐positive reduction using a convolutional neural network technique
231 Citations2016Atsushi Teramoto, Hiroshi Fujita +2 more
An improved FP-reduction scheme using CNN technique has been developed for the detection of pulmonary nodules in PET/CT images by means of convolutional neural networks (CNNs) and results indicate that the method may be useful in the computer-aided detection ofmonary nodules using PET/ CT images.
IEEE Transactions on Medical ImagingDetecting Cardiovascular Disease from Mammograms With Deep Learning
231 Citations2017Juan Wang, Huanjun Ding +5 more
The results suggest that deep learning can be used effectively to develop an automated system for BAC detection inmammograms to help identify and assess patients with cardiovascular risks.
arXiv (Cornell University)Combining Fully Convolutional and Recurrent Neural Networks for 3D Biomedical Image Segmentation
220 Citations2016Jianxu Chen, Lin Yang +3 more
IEEE Journal of Biomedical and Health InformaticsHEp-2 Cell Image Classification With Deep Convolutional Neural Networks
219 Citations2016Zhimin Gao, Lei Wang +2 more
The results on benchmark datasets demonstrate that the proposed framework can effectively outperform existing models by properly applying data augmentation, and the CNN-based framework has excellent adaptability across different datasets, which is highly desirable for cell image classification under varying laboratory settings.
Lecture notes in computer scienceMulti-input Cardiac Image Super-Resolution Using Convolutional Neural Networks
219 Citations2016Ozan Oktay, Wenjia Bai +8 more
Experimental results show that the CNN model outperforms state-of-the-art SR methods in terms of image quality while being computationally efficient and that image segmentation and motion tracking benefits more from SR-CNN when it is used as an initial upscaling method than conventional interpolation methods for the subsequent analysis.
International Journal of Computer Vision3D Scene Flow Estimation with a Piecewise Rigid Scene Model
214 Citations2015Christoph Vogel, Konrad Schindler +1 more
This work proposes to represent the dynamic scene as a collection of rigidly moving planes, into which the input images are segmented, and shows that such a view-consistent multi-frame scheme significantly improves accuracy, especially in the presence of occlusions, and increases robustness against adverse imaging conditions.
Computerized Medical Imaging and GraphicsStain Normalization using Sparse AutoEncoders (StaNoSA): Application to digital pathology
206 Citations2016Andrew Janowczyk, Ajay Basavanhally +1 more
It is shown how sparse autoencoders can be leveraged to partition images into tissue sub-types, so that color standardization for each can be performed independently.
IEEE Transactions on Image ProcessingThe Segmentation of the Left Ventricle of the Heart From Ultrasound Data Using Deep Learning Architectures and Derivative-Based Search Methods
205 Citations2011Gustavo Carneiro, Jacinto C. Nascimento +1 more
A new supervised learning model designed for the automatic segmentation of the left ventricle of the heart in ultrasound images is presented and it is shown that efficient search methods reduce up to tenfold the complexity of the method while still producing competitive segmentations.
IEEE Journal of Biomedical and Health InformaticsA Convolutional Neural Network for Automatic Characterization of Plaque Composition in Carotid Ultrasound
201 Citations2016Karim Lekadir, Alfiia Galimzianova +7 more
This paper proposes to build a convolutional neural network (CNN) that will automatically extract from the images the information that is optimal for the identification of the different plaque constituents, indicating the potential of deep learning for the challenging task of automatic characterization of plaque composition in carotid ultrasound.
International Journal of Computer Assisted Radiology and SurgeryAutomatic abdominal multi-organ segmentation using deep convolutional neural network and time-implicit level sets
198 Citations2016Peijun Hu, Fa Wu +4 more
A fully automatic method for multi-organ segmentation from abdominal CT images was developed and evaluated and demonstrated its potential in clinical usage with high effectiveness, robustness and efficiency.
Journal of Pathology InformaticsClassification of mitotic figures with convolutional neural networks and seeded blob features
198 Citations2013Christopher D. Malon, Eric Cosatto
This work demonstrates a powerful technique combining segmentation-based features with CNN, identifying the majority of mitotic figures with a fair precision and shows that the approach accommodates information from the additional focal planes and spectral bands from a multi-spectral scanner without major redesign.
Computational and Mathematical Methods in MedicinePulmonary Nodule Classification with Deep Convolutional Neural Networks on Computed Tomography Images
198 Citations2016Wei Li, Peng Cao +2 more
A deep convolutional neural networks method is designed for nodule classification, which has an advantage of autolearning representation and strong generalization ability and consistently outperforms the competing methods.
Lecture notes in computer scienceBeyond Classification: Structured Regression for Robust Cell Detection Using Convolutional Neural Network
197 Citations2015Yuanpu Xie, Fuyong Xing +3 more
A novel convolutional neural network (CNN) based structured regression model is presented, which is shown to be able to handle touching cells, inhomogeneous background noises, and large variations in sizes and shapes.
IEEE Transactions on Image ProcessingA Bottom-Up Approach for Pancreas Segmentation Using Cascaded Superpixels and (Deep) Image Patch Labeling
184 Citations2016Amal Farag, Le Lü +4 more
Under six-fold cross-validation, the bottom-up segmentation method significantly outperforms its MALF counterpart and the segmentation framework using deep patch labeling confidences is also more numerically stable, as reflected in the smaller performance metric standard deviations.
…