Top Research Papers on LLMs

This work explores LLM compression in a task-agnostic manner, which aims to preserve the multi-task solving and language generation ability of the original LLM, and adopts structural pruning that selectively removes non-critical coupled structures based on gradient information, maximally preserving the majority of the LLM's functionality.

This work introduces Tx-LLM, a generalist large language model (LLM) fine-tuned from PaLM-2 which encodes knowledge about diverse therapeutic modalities and believes it represents an important step towards LLMs encoding biochemical knowledge and could have a future role as an end-to-end tool across the drug discovery development pipeline.

This survey comprehensively review LLM-empowered recommender systems from various perspectives including pre-training, fine-tuning, and prompting paradigms, and comprehensively discusses the promising future directions in this emerging field.

This paper explores network binarization, a radical form of quantization, compressing model weights to a single bit, specifically for Large Language Models (LLMs) compression and proposes a novel approach, Partially-Binarized LLM (PB-LLM), which can achieve extreme low-bit quantization while maintaining the linguistic reasoning capacity of quantized LLMs.

Purpose: Empathy, a cornerstone of human interaction, is a unique quality to humans that Large Language Models (LLMs) are believed to lack. Our study aims to review the literature on the capacity of LLMs in demonstrating empathy Methods: We conducted a literature search on MEDLINE up to July 2023. Seven publications ultimately met the inclusion criteria. Results: All studies included in this review were published in 2023. All studies but one focused on ChatGPT-3.5 by OpenAI. Only one study evaluated empathy based on objective metrics, and all others used subjective human assessment. The studie...

A taxonomy encompassing 126 MM-LLMs, each characterized by its specific formulations is introduced, and the performance of selected MM-LLMs on mainstream benchmarks and key training recipes to enhance the potency of MM-LLMs are reviewed.

The basic principles and features of large language models, a type of AI model that is trained on vast amounts of text data to understand and generate human-like language outputs, are studied.

This paper aims to explore the potential of LLMs in graph machine learning, especially the node classification task, and investigates two possible pipelines: LLMs-as-Enhancers and LLMs-as-Predictors.

This work proposes to inject the 3D world into large language models and introduce a whole new family of 3D-LLMs that can take 3D point clouds and their features as input and perform a diverse set of 3D-related tasks, including captioning, dense captioning, 3D question answering, task decomposition, 3D grounding, 3D-assisted dialog, navigation, and so on.

It is found that the latest models, such as GPT-4, can consistently identify the most predictive features regardless of the query mechanism and across various prompting strategies, which suggests that LLM-based feature selection consistently achieves strong performance competitive with data-driven methods such as the LASSO.

This paper proposes ST-LLM, an effective video-LLM baseline with Spatial-Temporal sequence modeling inside LLM, and develops a dynamic masking strategy with tailor-made training objectives to address the overhead and stability issues introduced by uncompressed video tokens within LLMs.

This paper conducts an assessment of the privacy protection mechanisms employed by LLMs at various stages, followed by a detailed examination of their efficacy and constraints, and delineates the spectrum of data privacy threats.

This paper explores the scopes, applications, methodologies, and evaluation metrics related to strategic reasoning with LLMs, highlighting the burgeoning development in this area and the interdisciplinary approaches enhancing their decision-making performance.

This work empirically evaluate the power of 3 open SotA LLMs in zero-shot setting (LLaMA-2-13B, Mixtral 8x7B, and Gemma-7B), and introduces a new hallucination metric - Simple Hallucination Index (SHI).

Evalita-LLM, a new benchmark designed to evaluate Large Language Models on Italian tasks, is described, and an iterative methodology, where candidate tasks and candidate prompts are validated against a set of LLMs used for development is proposed.

This paper explored the deployment strategies, economic considerations, and sustainability challenges associated with the state-of-the-art LLMs, and discussed the deployment debate between Retrieval-Augmented Generation and fine-tuning, highlighting their respective advantages and limitations.

It is revealed that LLMs cannot work as expected on social prediction when given general input features without shortcuts, and possible reasons for this phenomenon are investigated, which suggest potential ways to enhance LLMs for social prediction.

The paper aims to provide a comprehensive analysis of the transformative impact of LLMs across various enterprise sectors and provides a comprehensive overview of current popular LLMs in Enterprise applications, in various domains, and discusses the Ethical, Technical, and Regulatory challenges, future trends, and developments in this dynamic field.

The research reveals LLMs, whilst demonstrating impressive capabilities in text generation, exhibit varying degrees of bias across different dimensions, offering a quantifiable measure to compare biases across models and over time, offering a vital tool for systems engineers, researchers and regulators in enhancing the fairness and reliability of LLMs.

RETA-LLM provides more plug-and-play modules to support better interaction between IR systems and LLMs, including {request rewriting, document retrieval, passage extraction, answer generation, and fact checking} modules.

The causes of hallucinations in Large Language Models are understood, the implications are explored, and potential strategies for mitigation are discussed to enhance the reliability of AI-generated content.

This survey reviews the recent developments in Graph ML and explores how LLMs can be utilized to enhance the quality of graph features, alleviate the reliance on labeled data, and address challenges such as graph heterogeneity and out-of-distribution (OOD) generalization.

LLM+P is the first framework that incorporates the strengths of classical planners into large language models, and is able to provide optimal solutions for most problems, while LLMs fail to provide even feasible plans for most Problems.

This work proposes a human-LLM collaboration framework (i.e., LLM-in-the-loop) to conduct TA with in-context learning (ICL), which yields similar coding quality to that of human coders but reduces TA's labor and time demands.

The value of fine-tuning is demonstrated and it is shown that by adding more career data from a different population, fine-tuning smaller LLMs surpasses the performance of fine-tuning larger models.

TnT-LLM, a two-phase framework that employs LLMs to automate the process of end-to-end label generation and assignment with minimal human effort, and generates more accurate and relevant label taxonomies when compared against state-of-the-art baselines, and achieves a favorable balance between accuracy and efficiency for classification at scale.

A comprehensive survey of key dimensions that are crucial to consider when assessing LLM trustworthiness is presented, which indicates that, in general, more aligned models tend to perform better in terms of overall trustworthiness.

Time-LLM is a reprogramming framework to repurpose LLMs for general time series forecasting with the backbone language models kept intact and is demonstrated to be a powerful time series learner that outperforms state-of-the-art, specialized forecasting models.

This paper introduces a lazy asynchronous multi-level approach that takes advantage of the fact that the tensors making up the model and optimizer state shards remain immutable for extended periods of time, which makes it possible to copy their content in the background with minimal interference during the training process.

This paper constructed Head-to-Tail, a benchmark that consists of 18K question-answer pairs regarding head, torso, and tail facts in terms of popularity and designed an automated evaluation method and a set of metrics that closely approximate the knowledge an LLM confidently internalizes.

LLM-PBE is a toolkit crafted specifically for the systematic evaluation of data privacy risks in LLMs, designed to analyze privacy across the entire lifecycle of LLMs, incorporating diverse attack and defense strategies, and handling various data types and metrics.

The purpose of this study is to provide readers, developers, academics, and users interested in LLM-based chatbots and virtual intelligent assistant technologies with use full information and future directions.

The best method achieves an accuracy between 92% and 100% in detecting if an LLM is contaminated with seven datasets, containing train and test/validation partitions, when contrasted with manual evaluation by human experts.

Overall, this review corroborates the promising utilization of LLMs to positively impact dementia care by boosting cognitive abilities, enriching social interaction, and supporting caregivers.

This paper extensively discusses different pretraining objectives, benchmarks, and transfer learning methods used in LLMs, and explores how LLMs can perform well across many domains and datasets if sufficiently trained on a large and diverse dataset.

The findings show that LLM-powered robots elevate expectations for sophisticated non-verbal cues and excel in connection-building and deliberation, but fall short in logical communication and may induce anxiety.

A detailed threat model and systematization of knowledge of red-teaming attacks on LLMs are presented and a taxonomy of attacks based on the stages of the LLM development and deployment process is developed to improve the security and robustness of LLM-based systems.

This work introduces a 1-bit LLM variant, namely BitNet b1.58, in which every single parameter of the LLM is ternary, which defines a new scaling law and recipe for training new generations of LLMs that are both high-performance and cost-effective.

The integration of sparsity awareness, context-adaptive loading, and a hardware-oriented design paves the way for effective inference of LLMs on devices with limited memory.

This work explores the feasibility of using large language understanding capabilities (LLMs) to identify rule violations on Reddit and examines how an LLM-based moderator (LLM-Mod) reasons about 744 posts across 9 subreddits that violate different types of rules.

This work introduces a label-free node classification on graphs with LLMs pipeline, LLM-GNN, which amalgamates the strengths of both GNNs and LLMs while mitigating their limitations while leveraging the confidence scores derived from LLMs to advanced node selection.

This paper designs a novel Test-Time Training pipeline, LLMTTT, which conducts the test-time adaptation under the annotations by LLMs on a carefully-selected node set and introduces a hybrid active node selection strategy that considers not only node diversity and representativeness, but also prediction signals from the pre-trained model.

The need for emotional intelligence in transformer-based LLMs and the various existing studies that have evaluated this aspect will be discussed.

An automated solution for large-scale LLM jailbreak susceptibility assessment called LLM-F UZZER, inspired by fuzz testing, which generates additional jailbreak prompts tailored to specific LLMs and highlights that many open-source and commercial LLMs suffer from severe jailbreak issues, even after safety fine-tuning.

The article delves into the capabilities of LLMs in generating both fake news and fake profiles, highlighting their dual role as both a tool for disinformation and a powerful means of detection.

A novel framework of LLM-enabled graphs for networking optimization is proposed, and a case study on UAV networking is presented, concentrating on optimizing UAV trajectory and communication resource allocation to validate the effectiveness of the proposed framework.

Tele-LLMs, the first series of language models ranging from 1B to 8B parameters, specifically tailored for telecommunications are developed and open-source, demonstrating that these models outperform their general-purpose counterparts on Tele-Eval while retaining their previously acquired capabilities, thus avoiding the catastrophic forgetting phenomenon.

AD-LLM is introduced, the first benchmark that evaluates how LLMs can help with NLP anomaly detection and finds that LLMs can work well in zero-shot AD, that carefully designed augmentation methods are useful, and that explaining model selection for specific datasets remains challenging.

The Knowledge Graph Large Language Model (KG-LLM) is introduced, a novel framework that leverages large language models (LLMs) for knowledge graph tasks and significantly improves the models' generalization capabilities, leading to more accurate predictions in unfamiliar scenarios.

This study introduces a novel approach, coined LLM-Rec, which incorporates four distinct prompting strategies of text enrichment for improving personalized text-based recommendations, and empirical experiments reveal that using LLM-augmented text significantly enhances recommendation quality.