Top Research Papers on Neural Networks

,

This paper introduces a real-time approach for generating a symbolic representation of the function learned by a neural network via another neural network (called the Interpretation Network, or I-Net), which maps network parameters to a symbolic representations of the network function.

This work proposes to represent neural networks as computational graphs of parameters, which allows them to harness powerful graph neural networks and transformers that preserve permutation symmetry, and enables a single model to encode neural computational graphs with diverse architectures.

A reproducible GNN benchmarking framework is introduced, with the facility for researchers to add new models conveniently for arbitrary datasets, and a principled investigation into the recent Weisfeiler-Lehman GNNs (WL-GNNs) compared to message passing-based graph convolutional networks (GCNs).

The results of applying the artificial neural networks methodology to acute nephritis diagnosis based upon selected symptoms show abilities of the network to learn the patterns corresponding to symptoms of the person.

Inductive biases are any assumptions that learners utilize to learn the world and predict the output that reduce the amount of data needed to fit the model while constraining the model’s flexibility.

This paper derives three different variants of the PathNN model that aggregate single shortest paths, all shortest paths and all simple paths of length up to K, and proves that two of these variants are strictly more powerful than the 1-WL algorithm, and experimentally validate the theoretical results.

This paper provides a comprehensive overview of CNNs and their applications in image recognition tasks, and reviews recent developments in CNNs, including attention mechanisms, capsule networks, transfer learning, adversarial training, quantization and compression, and enhancing the reliability and efficiency ofCNNs through formal methods.

A novel strategic pipeline is proposed that transfers the weights to the target SNN by combining threshold balance and soft-reset mechanisms and enables almost no accuracy loss between the converted SNNs and conventional ANNs with only $\sim1/10$ of the typical SNN simulation time.

A novel framework leveraging Graph Neural Networks to Generate Neural Networks (GNN2GNN) where powerful NN architectures can be learned out of a set of available architecture-performance pairs, and paves the way towards generalisation between datasets.

While some RNN architectures possess the capability to maintain a memory of the previous inputs/ outputs, to compute output, the memory states need to encompass information of many previous states, which can be difficult especially when performing tasks with long-term dependencies.

This work proposes a new quantum neural network model for quantum neural computing using (classically controlled) single-qubit operations and measurements on real-world quantum systems with naturally occurring environment-induced decoherence, which greatly reduces the difficulties of physical implementations.

This work demonstrates that diffusion models can also generate high-performing neural network parameters, and empirically finds that the generated models are not memorizing the trained ones.

Current GNN-based methods are reviewed, highlighting the ways that they have been used in several applications related to brain graphs such as missing brain graph synthesis and disease classification, and charting a path toward a better application of GNN models in network neuroscience field for neurological disorder diagnosis and population graph integration.

This chapter systematically organize the existing research of GNNs along three axes: foundations, frontiers, and applications, and introduces the fundamental aspects of GNNs ranging from the popular models and their expressive powers, to the scalability, interpretability and robustness of GNNs.

This work analyzes a mathematical model of networks of heterogeneous spiking neurons and reveals how a mostly overlooked property of the brain—neural heterogeneity—allows for the emergence of computationally specialized networks.

This chapter has attempted to depict the types of neural networks and machine learning as well as their applications in different industrial disciplines such as science, commerce, and medicine.

This work proposes a novel machine learning framework that can be used for designing an effective branching strategy as well as for computing better lower bounds, and learns two graph neural networks that both directly treat the network they want to verify as a graph input and perform forward-backward passes through the GNN layers.

A systematic empirical analysis into plasticity loss is conducted, finding that loss of plasticity is deeply connected to changes in the curvature of the loss landscape, but that it often occurs in the absence of saturated units.

Stitchable Neural Networks (SN-Net) is presented, a novel scalable and efficient framework for model deployment that cheaply produces numerous networks with different complexity and performance trade-offs given a family of pretrained neural networks, which the authors call anchors.

A novel method of constructing deep SNN models with knowledge distillation (KD) that uses ANN as the teacher model and SNN as the student model and it has a superb ability of noise immunity for various types of artificial noises and natural signals.

The purpose of this research is to classify the rice image dataset and detect the rice images using neural networks in experiments using public datasets.

The epinet is introduced: an architecture that can supplement any conventional neural network, including large pretrained models, and can be trained with modest incremental computation to estimate uncertainty, and the epistemic neural network (ENN) is introduced as an interface for models that produce joint predictions.

This survey comprehensively review this rapidly developing area of dynamic networks by dividing dynamic networks into three main categories: sample-wise dynamic models that process each sample with data-dependent architectures or parameters; spatial-wiseynamic networks that conduct adaptive computation with respect to different spatial locations of image data; and temporal-wise Dynamic networks that perform adaptive inference along the temporal dimension for sequential data.

This paper presents a model-agnostic methodology, namely Network In Graph Neural Network (NGNN), that allows arbitrary GNN models to increase their model capacity by making the model deeper, by inserting non-linear feedforward neural network layer(s) within each GNN layer.

It is shown that existing NLP-based optimization attacks are insufficiently powerful to reliably attack aligned text models: even when current NLP-based attacks fail, the authors can find adversarial inputs with brute force.

A novel approach to synthesizing neural networks as barrier certificates, which can provide safety guarantees for neural network controlled systems, and implements the tool NetBC, which is more effective and scalable than the existing polynomial barrier certificate-based method.

A new family of deep neural networks, where instead of the conventional representation of network layers as N-dimensional weight tensors, they use a continuous layer representation along the filter and channel dimensions, which can be applied to prune the model directly on an edge device while suffering only a small performance loss.

The Neural Bellman-Ford Network (NBFNet) is proposed, a general graph neural network framework that solves the path formulation with learned operators in the generalized Bell man-Ford algorithm, and outperforms existing methods by a large margin in both transductive and inductive settings.

CNNs are primarily used to solve difficult image-driven pattern recognition tasks and with their precise yet simple architecture, offer a simplified method of getting started with ANNs.

The core of e3nn are equivariant operations such as the TensorProduct class or the spherical harmonics functions that can be composed to create more complex modules such as convolutions and attention mechanisms, which can be used to efficiently articulate Tensor Field Networks, 3D Steerable CNNs, Clebsch-Gordan Networks, SE(3) Transformers and other E(3).

The definition of over-smoothing is axiomatically defined as the exponential convergence of suitable similarity measures on the node features of graph neural networks and extended to the rapidly emerging field of continuous-time GNNs.

In this paper, we show that the Kolmogorov two hidden layer neural network model with a continuous, discontinuous bounded and unbounded activation function in the second hidden layer can precisely represent continuous, discontinuous bounded and all unbounded multivariate functions, respectively.

This work derives an efficient approximate Sum-Product loopy belief propagation inference algorithm for discrete higher-order PGMs, and neuralizes the novel message passing scheme into a Factor Graph Neural Network (FGNN) module by allowing richer representations of the message update rules, which facilitates both efficient inference and powerful end-to-end learning.

A survey of confidence calibration problems in the context of neural networks and an empirical comparison of calibration methods is presented and a problem statement, calibration definitions, and different approaches to evaluation are analyzed.

An information-theoretic objective, InfoMax-Bottleneck~(IMB), is introduced, to carry out KF by optimizing the mutual information between the learned representations and input, and the derived factor networks yield gratifying performances on not only the dedicated tasks but also disentanglement, while enjoying much better interpretability and modularity.

This work lights up SNN's potential as a general backbone to support various applications in the field of SNN research, with a great balance between effectiveness and energy efficiency.

It is demonstrated that grouping tasks according to the Chomsky hierarchy allows us to forecast whether certain architectures will be able to generalize to out-of-distribution inputs, including negative results where even extensive amounts of data and training time never lead to any non-trivial generalization.

The theoretical foundation for the development of SuperHyperGraph Neural Networks (SHGNNs) and Plithogenic Graph Neural Networks is established, expanding the applicability of neural networks to these advanced graph structures.

It is found that a trained neural network is typically more clusterable than randomly initialized networks, and often clusterable relative to random networks with the same distribution of weights.

The common thread behind the recent Nobel Prize in Physics to John Hopfield and those conferred to Giorgio Parisi in 2021 and Philip Anderson in 1977 is disorder. Quoting Philip Anderson:"more is different". This principle has been extensively demonstrated in magnetic systems and spin glasses, and, in this work, we test its validity on Hopfield neural networks to show how an assembly of these models displays emergent capabilities that are not present at a single network level. Such an assembly is designed as a layered associative Hebbian network that, beyond accomplishing standard pattern reco...

The potential of RNNs to be better realized through sparse parameterizations is found, which significantly improve the stability and expressiveness of model performance across a wider array of hyperparameters while improving performance differentials at significantly reduced weight counts.

It is discovered that a BERT-based model with state-of-the-art performance successfully realizes parts of the natural logic model's causal structure, whereas a simpler baseline model fails to show any such structure, demonstrating that BERT representations encode the compositional structure of MQNLI.

This examination underscores the potential of counterfeit insights models utilizing neural systems in diagnosing cases requiring gastric surgery.

It is shown that Deep Sets, one of the well-known permutation-invariant neural networks, can be generalized in the sense of a quasi-arithmetic mean, and the behavior of Deep Sets is sensitive to the choice of the aggregation function.

NGNN is a plug-and-play framework that can be combined with various base GNNs and it is proved that NGNN can discriminate almost all r-regular graphs, where 1-WL always fails.

The proposed methodology provides insight into the connection between neural networks and classical numerical methods, and it is believed that it can have applications in fields such as Uncertainty Quantification and explainable artificial intelligence (XAI).

The purpose of this study is to familiarise the reader with the foundations of neural networks and highlight the different learning approaches and algorithms used in Machine Learning and Deep Learning.

This is the first work that the machine learning PDE’s solver has a convergent rate, such as in numerical methods, and can be applied in inverse problems and surface PDEs, although without proof.

Equivariant neural network potentials are a breakthrough new tool that are already enabling us to simulate systems at the molecular scale with unprecedented accuracy and speed, relying on nothing but fundamental physical laws.