Top Research Papers on Learning

The paper shows that artificial intelligence is fairly useful for the obligations of the automatic disease detection and switch mastering (as a method for reusing the existing understanding in the new software) is also beneficial.

This paper proposed the concept of programmable gradient information (PGI) to cope with the various changes required by deep networks to achieve multiple objectives, and designed a new lightweight network architecture -- Generalized Efficient Layer Aggregation Network (GELAN), based on gradient path planning.

This chapter presents techniques for statistical machine learning using Support Vector Machines (SVM) to recognize the patterns and classify them, predicting structured objects using SVM, k-nearest neighbor method for classification, and Naive Bayes classifiers.

A hybrid model is introduced, MambaFormer, that combines Mamba with attention blocks, surpassing individual models in tasks where they struggle independently, and suggests that hybrid architectures offer promising avenues for enhancing ICL in language models.

This work presents a new paradigm for continual learning that aims to train a more succinct memory system without accessing task identity at test time, and achieves competitive results against rehearsal-based methods even without a re-hearsal buffer.

It is shown that models can achieve non-trivial performance with only TR, and TR does not further improve with larger models or more demonstrations; LLMs acquire TL as the model scales, and TL's performance consistently improves with more demonstrations in context.

This work introduces MetaICL (Meta-training for In-Context Learning), a new meta-training framework for few-shot learning where a pretrained language model is tuned to do in-context learning on a large set of training tasks.

Dysample, an ultra-lightweight and effective dynamic upsampler that requires no customized CUDA package and has much fewer parameters, FLOPs, GPU memory, and latency, outperforms other upsamplers across five dense prediction tasks.

A state-of-the-art survey on an intelligent robot with the capability of autonomous deciding and learning reveals that the latest research in deep learning and reinforcement learning has paved the way for highly complex tasks to be performed by robots.

This work proposes an efficient method for retrieving prompts for in-context learning using annotated data and an LM, and trains an efficient dense retriever from this data, which is used to retrieve training examples as prompts at test time.

An overview of Deep Reinforcement Learning, including its basic components, key algorithms and techniques, and applications in areas s.a. robotics, game playing, and autonomous driving is provided.

The progress, challenges, and future work in ICL are summarized and a formal definition of ICL is presented and its correlation to related studies are clarified and potential directions for further research are provided.

This intensive workshop provides a comprehensive exploration of deep learning applications in life sciences, focusing on practical techniques for analyzing complex biological datasets. Participants will gain theoretical and hands-on experience with deep learning tools such as TensorFlow and Keras, learning to construct neural networks and apply them to areas like genomics and personalized medicine.

Novel insights are provided into how ICL leverages label information, revealing both capabilities and limitations and it is found that ICL struggles to fully overcome prediction preferences acquired from pre-training data and, further, that ICL does not consider all in-context information equally.

This work proposes an offline RL method that never needs to evaluate actions outside of the dataset, but still enables the learned policy to improve substantially over the best behavior in the data through generalization.

1. Submit a PDF of your homework, with an appendix listing all your code, to the Gradescope assignment entitled “Homework 7 Write-Up”. In addition, please include, as your solutions to each coding problem, the specific subset of code relevant to that part of the problem. You may typeset your homework in LaTeX or Word (submit PDF format, not .doc/.docx format) or submit neatly handwritten and scanned solutions. Please start each question on a new page. If there are graphs, include those graphs in the correct sections. Do not put them in an appendix. We need each solution to be self-contained on...

This article provides a concise but holistic review of the recent advances made in using machine learning to achieve safe decision-making under uncertainties, with a focus on unifying the language and frameworks used in control theory and reinforcement learning research.

D-Adaptation is an approach to automatically setting the learning rate which asymptotically achieves the optimal rate of convergence for minimizing convex Lipschitz functions, with no back-tracking or line searches, and no additional function value or gradient evaluations per step. Our approach is the first hyper-parameter free method for this class without additional multiplicative log factors in the convergence rate. We present extensive experiments for SGD and Adam variants of our method, where the method automatically matches hand-tuned learning rates across more than a dozen diverse machi...

This work proposes CLS-ER, a novel dual memory experience replay (ER) method which maintains short-term and long-term semantic memories that interact with the episodic memory which achieves state-of-the-art performance on standard benchmarks as well as more realistic general continual learning settings.

A comprehensive overview of deep learning modelling that can serve as a resource for academics and industry people alike is provided, including an overview of real-world application areas where deep learning techniques can be utilised.

This work makes the first theoretical progress on this question via an analysis of the loss landscape for linear transformers trained over random instances of linear regression, and proves the global minimum of the training objective implements a single iteration of preconditioned gradient descent.

An innovative convolutional neural network architecture aimed at addressing challenges of detection and classification of apple fruit diseases is proposed and experimentally validated, achieving a remarkable classification accuracy of 95.37%.

Results show that Transformers can learn to implement two distinct algorithms to solve a single task, and can adaptively select the more sample-efficient algorithm depending on the sequence of in-context examples, and that extant LLMs, e.g. LLaMA-2, GPT-4, can compete with nearest-neighbor baselines on prediction tasks that are guaranteed to not be in their training set.

With preliminary systems optimization, TTT-Linear is already faster than Transformer at 8k context and matches Mamba in wall-clock time, and TTT-MLP still faces challenges in memory I/O, but shows larger potential in long context, pointing to a promising direction for future research.

This work revisits existing approaches and combines different techniques to scale the pretraining in terms of data and model size, and proposes an automatic pipeline to build a dedicated, diverse, and curated image dataset instead of uncurated data, as typically done in the self-supervised literature.

The digital competencies teachers and learners require to use this chatbot ethically and effectively to support language learning are presented.

Federated learning allows several actors to collaborate on the development of a single, robust machine learning model without sharing data, allowing crucial issues such as data privacy, data security, data access rights, and access to heterogeneous data to be addressed.

The experimental results show that the ANDV A F -test feature selection algorithm along with the Support Vector Machine classifier, is a viable approach for developing an advanced intelligent system that can identify heart disease.

O-LoRA is proposed, a simple and efficient approach for continual learning in language models, effectively mitigating catastrophic forgetting while learning new tasks, and excels in preserving the generalization ability of LLMs on unseen tasks.

This work investigates the hypothesis that transformer-based in-context learners implement standard learning algorithms implicitly, by encoding smaller models in their activations, and updating these implicit models as new examples appear in the context, and suggests that in- context learning is understandable in algorithmic terms, and that (at least in the linear case) learners may rediscover standard estimation algorithms.

The difference of interde-pendent, of the epistemology and of the subject who construct knowledge is clarified, and it is argued as the research of issues that the number of research on collaborative learning is a little, and the necessity for the research on conditions that collaborativeLearning is brought and the relation of the origin and development of collaboration and knowledge is required.

.

The aim of this research is to investigate the influence of flipbook learning media, learning interest, and learning motivation on junior high school students' learning outcomes. The method used is the regression method with a quantitative approach. This research was conducted at Junior High School 1 Yogyakarta with a sample of 64 class VIII social studies students. Data collection consists of interviews, observations, and documentation. Prerequisite test analysis consists of tests for normality, multicollinearity, and heteroscedasticity. Hypothesis testing using simple regression, and multipl...

It is found that the pretrained transformer can be used to solve a range of RL problems in-context, exhibiting both exploration online and conservatism offline, despite not being explicitly trained to do so.

Experimental results demonstrate that ICCL, developed during the instruction-tuning stage, is effective for representative open-source LLMs, and makes the code publicly available.

Findings reveal that AI/ML algorithms are instrumental in personalizing learning experiences, and underscore the potential of adaptive learning to revolutionize education by catering to individual learner needs.

The curious researcher is empowered to navigate the terrain of methods, understand the role of the various knobs, and gain the know-how required to explore how delicious SSL can be by laying the foundations and latest SSL recipes in the style of a cookbook.

Reflexion is a novel framework to reinforce language agents not by updating weights, but instead through linguistic feedback, which obtains significant improvements over a baseline agent across diverse tasks (sequential decision-making, coding, language reasoning).

This paper builds upon prior work and applies contrastive representation learning to action-labeled trajectories, in such a way that the (inner product of) learned representations exactly corresponds to a goal-conditioned value function.

This work proposes a novel meta-learning based paradigm that can retain the advantages of unimodal existence and further boost the performance of multimodal fusion, and introduces the Adaptive Multimodal Meta-Learning (AMML), which achieves state-of-the-art performance.

OptiPrompt is proposed, a novel and efficient method which directly optimizes in continuous embedding space and is able to predict an additional 6.4% of facts in the LAMA benchmark.

The utility of Machine Learning, Deep Learning, Reinforcement Learning, and Deep Reinforcement learning in Quantitative Finance and the Stock Market is explained and potential future study paths are outlined based on the overview that was presented before.

This work proposes FCCL (Federated CrossCorrelation and Continual Learning), which leverages unlabeled public data for communication and construct cross-correlation matrix to learn a generalizable representation under domain shift for heterogeneity problem and catastrophic forgetting.

This work develops a reinforcement learning method inspired by direct collocation to generate parkour skills, including climbing over high obstacles, leaping over large gaps, crawling beneath low barriers, squeezing through thin slits, and running, and distill these skills into a single vision-based parkour policy and transfer it to a quadrupedal robot using its egocentric depth camera.

Over the past 2 decades, great strides have been made in analyzing the cognitive processes involved in learning and instruction. During the same period, however, attention to motivational issues has been minimal. It is now time to redress this imbalance. As Bruner (1966) has put the case: The will to learn is an intrinsic motive, one that finds both its source and its reward in its own exercise. The will to learn becomes a ‘problem’ only under specialized circumstances like those of a school, where a curriculum is set, students confined and a path fixed. The problem exists not so much in lea...

How labels of in-context examples affect predictions, how label relationships learned during pre-training interact with input–label examples provided in- context, and how ICL aggregates label information across in- Context examples are studied.

This paper reinterprets the projection head in SSL as a per-pixel projection and predicts a set of spatial alignment maps from the original features by this weight-sharing projection head, so that the projected embeddings could be exactly aligned and thus guide the feature learning better.

A deep reinforcement learning–based framework for full-body control of humanoid robots was developed, enabling a game of one-versus-one soccer and the robots exhibited emergent behaviors in the form of dynamic motor skills such as the ability to recover from falls and also tactics like defending the ball against an opponent.

This work proposes a new task-adaptive weight update rule that greatly enhances the fast adaptation process of MAML and introduces a small meta-network that can generate per-step hyperparameters for each given task: learning rate and weight decay coefficients.

This work proposes a general reinforcement learning-based backdoor attack framework where the attacker first trains a (non-myopic) attack policy using a simulator built upon its local data and common knowledge on the FL system, which is then applied during actual FL training.