Top Research Papers on Deep Learning

This textbook gives a comprehensive understanding of the foundational ideas and key concepts of modern deep learning architectures and techniques.

This article provides a conceptual distinction between relevant terms and concepts, explains the process of automated analytical model building through machine learning and deep learning, and discusses the challenges that arise when implementing such intelligent systems in the field of electronic markets and networked business.

This work describes model-based optimization and data-centric deep learning as edges of a continuous spectrum varying in specificity and parameterization, and provides a tutorial-style presentation to the methodologies lying in the middle ground of this spectrum, referred to as model- based deep learning.

A systematic overview of trends, challenges, and opportunities in applications of deep-learning methods in seismology is presented, covering approaches, limitations, and opportunity.

With a direct analysis of neural networks, this paper presents a mathematically tight generalization theory to partially address an open problem regarding the generalization of deep learning.Unlike previous bound-based theory, our main theory is quantitatively as tight as possible for every dataset individually, while producing qualitative insights competitively.Our results give insight into why and how deep learning can generalize well, despite its large capacity, complexity, possible algorithmic instability, nonrobustness, and sharp minima, answering to an open question in the literature.We ...

This is the first comprehensive and self-contained introduction to deep reinforcement learning, covering all aspects from fundamentals and research to applications. It includes examples and codes to help readers practice and implement the techniques.

How can neural networks learn the rich internal representations required for difficult tasks such as recognizing objects or understanding language?

This book describes how neural networks operate from the mathematical point of view. As a result, neural networks can be interpreted both as function universal approximators and information processors. The book bridges the gap between ideas and concepts of neural networks, which are used nowadays at an intuitive level, and the precise modern mathematical language, presenting the best practices of the former and enjoying the robustness and elegance of the latter. This book can be used in a graduate course in deep learning, with the first few parts being accessible to senior undergraduates. In a...

In the past few years, deep learning has been successfully applied to various omics data. However, the applications of deep learning in metabolomics are still relatively low compared to others omics. Currently, data pre-processing using convolutional neural network architecture appears to benefit the most from deep learning. Compound/structure identification and quantification using artificial neural network/deep learning performed relatively better than traditional machine learning techniques, whereas only marginally better results are observed in biological interpretations. Before deep learn...

Compared to traditional machine learning, deep learning can learn deeper abstract data representation and understand scattered data properties. It has gained considerable attention for its extraordinary performances. However, existing deep learning algorithms perform poorly on new tasks. Meta-learning, known as learning to learn, is one of the effective techniques to overcome this issue. Meta-learning’s generalization ability to unknown tasks is improved by employing prior knowledge to assist the learning of new tasks. There are mainly three types of meta-learning methods: metric-based, model-...

This paper is using deep learning to uncover higher-level representational features, to clearly explain transfer learning, to provide current solutions and evaluate applications in diverse areas of transfer learning as well as deep learning.

Reinforcement learning is a learning paradigm for solving sequential decision-making problems. Recent years have witnessed remarkable progress in reinforcement learning upon the fast development of deep neural networks. Along with the promising prospects of reinforcement learning in numerous domains such as robotics and game-playing, transfer learning has arisen to tackle various challenges faced by reinforcement learning, by transferring knowledge from external expertise to facilitate the efficiency and effectiveness of the learning process. In this survey, we systematically investigate the r...

An insight survey for machine learning along with deep learning applications in various domains is provided and some applications with new normal COVID-19 blues are exemplified.

This review of talks and discussions in the "Deep Learning and Reinforcement Learning" session of the symposium, International Symposium on Artificial Intelligence and Brain Science, discusses whether the authors can achieve comprehensive understanding of human intelligence based on the recent advances of deep learning and reinforcement learning algorithms.

This survey systematically investigates the recent progress of transfer learning approaches in the context of deep reinforcement learning, and provides a framework for categorizing the state-of-the-art transfer Learning approaches under which to analyze their goals, methodologies, compatible reinforcement learning backbones, and practical applications.

It is concluded that ML and DL are powerful new tools for predictive modelling and data analysis, comparable to other traditional statistical tools.

A state-of-the-art survey on an intelligent robot with the capability of autonomous deciding and learning reveals that the latest research in deep learning and reinforcement learning has paved the way for highly complex tasks to be performed by robots.

This is the first comprehensive survey in the efficient deep learning space that covers the landscape of model efficiency from modeling techniques to hardware support and the seminal work there.

This work explores the suitability of adopting AutoDL for Water Quality Assessment by drawing a comparison between AutoDL and a conventional models and analysis to foresee the quality of the water, an appropriate class based on Water Quality Index segregating water bodies into different classes.

This paper aims to introduce deep learning to the power quality community by reviewing the latest applications and discussing the open challenges of this technology. Publications covering deep learning to power quality are stratified in terms of application, type of data, and learning technique. This work shows that the majority of the deep learning applications to power quality are based on unrealistic synthetic data and supervised learning without proper labelling. Some applications with deep learning have already been solved by previous machine learning methods or expert systems. The main b...

This paper reviews the state-of-art deep ensemble models and serves as an extensive summary for the researchers and concludes with some potential future research directions.

Recent key developments in ensemble deep learning are shared and a look is looked at at how their contribution has benefited a wide range of bioinformatics research from basic sequence analysis to systems biology.

This work introduces an optimal transportation (OT) view of generative adversarial networks (GANs). Natural datasets have intrinsic patterns, which can be summarized as the manifold distribution principle: the distribution of a class of data is close to a low-dimensional manifold. GANs mainly accomplish two tasks: manifold learning and probability distribution transformation. The latter can be carried out using the classical OT method. From the OT perspective, the generator computes the OT map, while the discriminator computes the Wasserstein distance between the generated data distribution an...

A template-free deep learning model is developed to predict the activation energy given reactant and product graphs and it is demonstrated that the model achieves accurate predictions and agrees with an intuitive understanding of chemical reactivity.

The concept of sequential recommendation is illustrated, a categorization of existing algorithms in terms of three types of behavioral sequences are proposed, and the key factors affecting the performance of DL-based models are summarized.

This survey provides a comprehensive introduction to Bayesian deep learning and reviews its recent applications on recommender systems, topic models, control, and so on.

The application of DL techniques to fNirS studies has shown to mitigate many of the hurdles present in fNIRS studies such as lengthy data preprocessing or small sample sizes while achieving comparable or improved classification accuracy.

We use deep neural networks to estimate an asset pricing model for individual stock returns that takes advantage of the vast amount of conditioning information, keeps a fully flexible form, and accounts for time variation. The key innovations are to use the fundamental no-arbitrage condition as criterion function to construct the most informative test assets with an adversarial approach and to extract the states of the economy from many macroeconomic time series. Our asset pricing model outperforms out-of-sample all benchmark approaches in terms of Sharpe ratio, explained variation, and pricin...

As an advanced artificial intelligence technique for solving learning problems, deep learning (DL) has achieved great success in many real-world applications and attracted increasing attention in recent years. However, as the performance of DL depends on many factors such as the architecture and hyperparameters, how to optimize DL has become a hot research topic in the field of DL and artificial intelligence. Evolutionary computation (EC), including evolutionary algorithm and swarm intelligence, is a kind of efficient and intelligent optimization methodology inspired by the mechanisms of biolo...

This work investigates and summarizes key methods of Deep Meta-Learning, which are categorized into (i) metric-, (ii) model-, and (iii) optimization-based techniques, and identifies the main open challenges.

For the first time, the exciting practical advances in modern artificial intelligence capabilities can be matched with a set of effective principles, providing a timeless blueprint for theoretical research in deep learning.

Deep neural networks can achieve great successes when presented with large data sets and sufficient computational resources. However, their ability to learn new concepts quickly is limited. Meta-learning is one approach to address this issue, by enabling the network to learn how to learn. The field of Deep Meta-Learning advances at great speed, but lacks a unified, in-depth overview of current techniques. With this work, we aim to bridge this gap. After providing the reader with a theoretical foundation, we investigate and summarize key methods, which are categorized into (i) metric-, (ii) mod...

A formal classification method for the existing work in deep active learning is provided, along with a comprehensive and systematic overview, to investigate whether AL can be used to reduce the cost of sample annotation while retaining the powerful learning capabilities of DL.

It is shown that progress in all five prominent application areas is strongly reliant on increases in computing power, and that progress along current lines is rapidly becoming economically, technically, and environmentally unsustainable.

The synthetic-to-real domain adaptation problem that inevitably arises in applications of synthetic data is discussed, including synthetic- to-real refinement with GAN-based models and domain adaptation at the feature/model level without explicit data transformations.

Deep learning's recent history has been one of achievement: from triumphing over humans in the game of Go to world-leading performance in image classification, voice recognition, translation, and other tasks. But this progress has come with a voracious appetite for computing power. This article catalogs the extent of this dependency, showing that progress across a wide variety of applications is strongly reliant on increases in computing power. Extrapolating forward this reliance reveals that progress along current lines is rapidly becoming economically, technically, and environmentally unsust...

This textbook covers both classical and modern models in deep learning and includes examples and exercises throughout the chapters.

Familiarity with the key concepts described will help radiologists understand advances of deep learning in medical imaging and facilitate clinical adoption of these techniques.

Abstract The goal of spectral imaging is to capture the spectral signature of a target. Traditional scanning method for spectral imaging suffers from large system volume and low image acquisition speed for large scenes. In contrast, computational spectral imaging methods have resorted to computation power for reduced system volume, but still endure long computation time for iterative spectral reconstructions. Recently, deep learning techniques are introduced into computational spectral imaging, witnessing fast reconstruction speed, great reconstruction quality, and the potential to drastically...

Using convolutional neural networks on a large new dataset, deep learning applied to echocardiography can identify local cardiac structures, estimate cardiac function, and predict systemic phenotypes that modify cardiovascular risk but not readily identifiable to human interpretation.