Top Research Papers on Neural Networks

The success of GNNs in extrapolating algorithmic tasks to new data relies on encoding task-specific non-linearities in the architecture or features, and a hypothesis is suggested for which theoretical and empirical evidence is provided.

Spiking neural networks (SNNs) are biology-inspired artificial neural networks (ANNs) that comprise of spiking neurons to process asynchronous discrete signals. While more efficient in power consumption and inference speed on the neuromorphic hardware, SNNs are usually difficult to train directly from scratch with spikes due to the discreteness. As an alternative, many efforts have been devoted to converting conventional ANNs into SNNs by copying the weights from ANNs and adjusting the spiking threshold potential of neurons in SNNs. Researchers have designed new SNN architectures and conversio...

Current GNN-based methods are reviewed, highlighting the ways that they have been used in several applications related to brain graphs such as missing brain graph synthesis and disease classification, and charting a path toward a better application of GNN models in network neuroscience field for neurological disorder diagnosis and population graph integration.

This textbook provides a compact but comprehensive treatment that provides analytical and design steps to recurrent neural networks from scratch.

This study proposes a novel network model, called operational neural networks (ONNs), which can be heterogeneous and encapsulate neurons with any set of operators to boost diversity and to learn highly complex and multi-modal functions or spaces with minimal network complexity and training data.

This Primer provides a practical and accessible introduction to GNNs, describing their properties and applications to the life and physical sciences and explores how they are applied across the life and physical sciences.

A biomimetic device based on a dual gated MoS2 field effect transistor capable of encoding analog signals into stochastic spike trains at energy cost of 1–5 pJ/spike is demonstrated.

The role of graph convolutional filters in GNNs is discussed and it is shown that any architecture built with such filters has the fundamental properties of permutation equivariance and stability to changes in the topology.

In this paper, neural bursting and synchronization are imitated by modeling two neural network models based on the Hopfield neural network, which consists of four neurons, which correspond to realizing neural bursting firings.

The results suggest that regularization is critical for worst-group performance in the overparameterized regime, even if it is not needed for average performance, and introduce and provide convergence guarantees for a stochastic optimizer for this group DRO setting.

This is the first work that the machine learning PDE’s solver has a convergent rate, such as in numerical methods, and can be applied in inverse problems and surface PDEs, although without proof.

Link prediction is a very fundamental task on graphs. Inspired by traditional path-based methods, in this paper we propose a general and flexible representation learning framework based on paths for link prediction. Specifically, we define the representation of a pair of nodes as the generalized sum of all path representations, with each path representation as the generalized product of the edge representations in the path. Motivated by the Bellman-Ford algorithm for solving the shortest path problem, we show that the proposed path formulation can be efficiently solved by the generalized Bellm...

The proposed work involves the approach of deep neural network and incorporates a CNN based model to classify the MRI as "Tumour DETECTED" or "TUMOUR Not DETECTed" and captures a mean accuracy score of 96.08% with fscore of 97.3.

A comprehensive survey of algorithms proposed for binary neural networks, mainly categorized into the native solutions directly conducting binarization, and the optimized ones using techniques like minimizing the quantization error, improving the network loss function, and reducing the gradient error are presented.

A review of the progress of optical neural networks based on the principle of artificial neural networks, the essence of optical matrix multiplier for linear operation and the nonlinearity in optical neural network is introduced.

DyGNN, a Dynamic Graph Neural Network model, which can model the dynamic information as the graph evolving is proposed, which keeps updating node information by capturing the sequential information of edges (interactions), the time intervals between edges and information propagation coherently.

This paper scrutinizes the application of CNNs in various fields, including image classification, facial recognition, audio retrieval, electrocardiogram analysis, and object detection, and posits the amalgamation of CNNs with recurrent neural networks as a potential alternative for training datasets.

Convolutional Neural Networks employs a definitely algorithm of steps to follow including methods like Backpropagation, Convolutional Layers, Feature formation and Pooling.

The purpose of this Special Issue was to publish high-quality research and review papers that cover the application of various types of artificial neural networks in solving relevant tasks and problems of widely defined agriculture.

CNNs are primarily used to solve difficult image-driven pattern recognition tasks and with their precise yet simple architecture, offer a simplified method of getting started with ANNs.

This textbook covers both classical and modern models in deep learning and includes examples and exercises throughout the chapters.

This work lights up SNN's potential as a general backbone to support various applications in the field of SNN research, with a great balance between effectiveness and energy efficiency.

This survey covers the main ideas behind SNNs and reviews recent trends in learning rules and network architectures, with a particular focus on biologically inspired strategies.

The proposed framework incorporates an interpretable compatibility matrix for modeling the heterophily or homophily level in the graph, which can be learned in an end-to-end fashion, enabling it to go beyond the assumption of stronghomophily.

In an Artificial Neural Network, activation functions are very important as they help in learning and making sense of non-linear and complicated mappings between the inputs and corresponding outputs.

A reproducible GNN benchmarking framework is introduced, with the facility for researchers to add new models conveniently for arbitrary datasets, and a principled investigation into the recent Weisfeiler-Lehman GNNs (WL-GNNs) compared to message passing-based graph convolutional networks (GCNs).

The proposed orthogonal convolution requires no additional parameters and little computational overhead and consistently outperforms the kernel orthogonality alternative on a wide range of tasks such as image classification and inpainting under supervised, semi-supervised and unsupervised settings.

This survey comprehensively review this rapidly developing area of dynamic networks by dividing dynamic networks into three main categories: sample-wise dynamic models that process each sample with data-dependent architectures or parameters; spatial-wiseynamic networks that conduct adaptive computation with respect to different spatial locations of image data; and temporal-wise Dynamic networks that perform adaptive inference along the temporal dimension for sequential data.

This paper concentrates upon the use of RNN and CNN in the feature extraction of images and the challenges and a brief literature review of the neural networks like CNN and RNN.

An overview of the construction method of wireless communication graph for various wireless networks and the progress of several classical paradigms of graph neural networks are introduced, as well as several applications of GNNs in wireless networks such as resource allocation and several emerging fields.

This article implements complex-valued operations in an optical neural chip that integrates input preparation, weight multiplication and output generation within a single device.

Pattern Recognition with Neural Networks in C++ covers pattern classification and neural network approaches within the same framework and provides an intuitive explanation of each method for each network paradigm.

Compared with the ANN model, the solar radiation prediction using the RNN model has a higher prediction accuracy, with a 47% improvement in Normalized Mean Bias Error (NMBE) and a 26% improved in Root-Mean-Squared Error (RMSE).

A comprehensive review of recent advances in the application of graph neural networks to the IoT field is presented, including a deep dive analysis of GNN design in various IoT sensing environments, an overarching list of public data and source codes from the collected publications, and future research directions.

This work proves that graph convolutions with integral Lipschitz filters, in combination with the frequency mixing effect of the corresponding nonlinearities, yields an architecture that is both stable to small changes in the underlying topology, and discriminative of information located at high frequencies.

An operational framework to unify this vast and diverse literature by describing pooling operators as the combination of three functions: selection, reduction, and connection (SRC).

Comprehensive evaluations indicate that the proposed HGNN framework can consistently outperform existing methods with a significant margin, especially when modeling implicit data correlations.

Attention is brought to the promising applicability of artificial neural networks applied to the control of microgrid distributed generation sources, as well as in scheduling, power sharing, supervisory control and optimization.

This work gives a comprehensive overview of uncertainty estimation in neural networks, reviews recent advances in the field, highlights current challenges, and identifies potential research opportunities, and a comprehensive introduction to the most crucial sources of uncertainty is given.

A survey of methods for pruning deep neural networks is presented, categorising over 150 studies based on the underlying approach used and focusing on three categories: methods that use magnitude based pruning, methods that utilise clustering to identify redundancy, and methods that using sensitivity analysis to assess the effect of pruning.