Top Research Papers on Neural Networks

Spiking neural networks (SNNs) are biology-inspired artificial neural networks (ANNs) that comprise of spiking neurons to process asynchronous discrete signals. While more efficient in power consumption and inference speed on the neuromorphic hardware, SNNs are usually difficult to train directly from scratch with spikes due to the discreteness. As an alternative, many efforts have been devoted to converting conventional ANNs into SNNs by copying the weights from ANNs and adjusting the spiking threshold potential of neurons in SNNs. Researchers have designed new SNN architectures and conversio...

The success of GNNs in extrapolating algorithmic tasks to new data relies on encoding task-specific non-linearities in the architecture or features, and a hypothesis is suggested for which theoretical and empirical evidence is provided.

This textbook provides a compact but comprehensive treatment that provides analytical and design steps to recurrent neural networks from scratch.

Current GNN-based methods are reviewed, highlighting the ways that they have been used in several applications related to brain graphs such as missing brain graph synthesis and disease classification, and charting a path toward a better application of GNN models in network neuroscience field for neurological disorder diagnosis and population graph integration.

A biomimetic device based on a dual gated MoS2 field effect transistor capable of encoding analog signals into stochastic spike trains at energy cost of 1–5 pJ/spike is demonstrated.

In recent years, breakthroughs in artificial intelligence (AI) have dramatically transformed many fields. Developments in neural networks primarily drive the advances. These powerful models have demonstrated unprecedented capabilities in learning complex patterns, making predictions, and generating creative content. From the pixels on a smartphone camera to the recommendation systems on a streaming service, neural networks are now ubiquitous and are part of daily interactions. But what exactly are neural networks? They are mathematical models designed to recognize patterns. By learning from da...

This study proposes a novel network model, called operational neural networks (ONNs), which can be heterogeneous and encapsulate neurons with any set of operators to boost diversity and to learn highly complex and multi-modal functions or spaces with minimal network complexity and training data.

This Primer provides a practical and accessible introduction to GNNs, describing their properties and applications to the life and physical sciences and explores how they are applied across the life and physical sciences.

The role of graph convolutional filters in GNNs is discussed and it is shown that any architecture built with such filters has the fundamental properties of permutation equivariance and stability to changes in the topology.

The results suggest that regularization is critical for worst-group performance in the overparameterized regime, even if it is not needed for average performance, and introduce and provide convergence guarantees for a stochastic optimizer for this group DRO setting.

This survey comprehensively review this rapidly developing area of dynamic networks by dividing dynamic networks into three main categories: sample-wise dynamic models that process each sample with data-dependent architectures or parameters; spatial-wiseynamic networks that conduct adaptive computation with respect to different spatial locations of image data; and temporal-wise Dynamic networks that perform adaptive inference along the temporal dimension for sequential data.

In this paper, neural bursting and synchronization are imitated by modeling two neural network models based on the Hopfield neural network, which consists of four neurons, which correspond to realizing neural bursting firings.

The proposed work involves the approach of deep neural network and incorporates a CNN based model to classify the MRI as "Tumour DETECTED" or "TUMOUR Not DETECTed" and captures a mean accuracy score of 96.08% with fscore of 97.3.

This is the first work that the machine learning PDE’s solver has a convergent rate, such as in numerical methods, and can be applied in inverse problems and surface PDEs, although without proof.

Link prediction is a very fundamental task on graphs. Inspired by traditional path-based methods, in this paper we propose a general and flexible representation learning framework based on paths for link prediction. Specifically, we define the representation of a pair of nodes as the generalized sum of all path representations, with each path representation as the generalized product of the edge representations in the path. Motivated by the Bellman-Ford algorithm for solving the shortest path problem, we show that the proposed path formulation can be efficiently solved by the generalized Bellm...

A review of the progress of optical neural networks based on the principle of artificial neural networks, the essence of optical matrix multiplier for linear operation and the nonlinearity in optical neural network is introduced.

The proposed orthogonal convolution requires no additional parameters and little computational overhead and consistently outperforms the kernel orthogonality alternative on a wide range of tasks such as image classification and inpainting under supervised, semi-supervised and unsupervised settings.

DyGNN, a Dynamic Graph Neural Network model, which can model the dynamic information as the graph evolving is proposed, which keeps updating node information by capturing the sequential information of edges (interactions), the time intervals between edges and information propagation coherently.

A reproducible GNN benchmarking framework is introduced, with the facility for researchers to add new models conveniently for arbitrary datasets, and a principled investigation into the recent Weisfeiler-Lehman GNNs (WL-GNNs) compared to message passing-based graph convolutional networks (GCNs).

This textbook covers both classical and modern models in deep learning and includes examples and exercises throughout the chapters.