The Natural Language Processing Reading Group at Stony Brook University meets weekly to discuss recent research papers in NLP and related fields.
Join the Google Group here.

Learning to Evict from Key-Value Cache


Abstract: The growing size of Large Language Models (LLMs) makes efficient inference challenging, primarily due to the memory demands of the autoregressive Key-Value (KV) cache. Existing eviction or compression methods reduce cost but rely on heuristics, such as recency or past attention scores, which serve only as indirect proxies for a token's future utility and introduce computational overhead. We reframe KV cache eviction as a reinforcement learning (RL) problem: learning to rank tokens by their predicted usefulness for future decoding. To this end, we introduce KV Policy (KVP), a framework of lightweight per-head RL agents trained on pre-computed generation traces using only key and value vectors. Each agent learns a specialized eviction policy guided by a holistic reward, derived from future utility, that evaluates the quality of the ranking across all cache budgets, requiring no modifications to the underlying LLM or additional inference. Evaluated across two model families on the long-context benchmark RULER (up to 128K tokens) and the multi-turn dialogue benchmark OASST2-4k, KVP significantly outperforms strong baselines. Zero-shot tests on standard downstream tasks (BoolQ, LongBench passage retrieval, GovReport) further show that KVP generalizes beyond its training distribution and to considerably longer sequence lengths. These results demonstrate that learning to predict future token utility is a powerful and scalable paradigm for adaptive KV cache management.

Location: NCS 220

Zoom Link: https://stonybrook.zoom.us/j/92942833294?pwd=ysKMaQx7Hp0lkq5nOb3zJ9ivNZVvLv.1&jst=2
The Challenges of Machine Learning in Adversarial Settings by Patrick McDaniel, Pennsylvania State University

Abstract: Advances in AI and machine learning have enabled new applications and services to interpret and process inputs in previously unthinkable complex environments. Autonomous cars, data analytics, adaptive communication and self-aware software systems are now revolutionizing markets by achieving or exceeding human performance. In this talk, I consider the evolving use of machine learning in security-sensitive contexts and explore why many systems are vulnerable to nonobvious and potentially dangerous manipulation. Here, we examine sensitivity in any application whose misuse might lead to harm--for instance, forcing adaptive network in an unstable state, crashing an autonomous vehicle or bypassing an adult content filter. I explore the use of machine learning in this area particularly in light of recent discoveries in the creation of adversarial samples and defenses against them and posit on future attacks on machine learning. The talk is concluded with a discussion of the technological and societal challenges we face as a result of current and future advances in intelligent computing.

Bio: Patrick McDaniel is the William L. Weiss Professor of Information and Communications Technology and Director of the Institute for Networking and Security Research in the School of Electrical Engineering and Computer Science at the Pennsylvania State University. Professor McDaniel is also a Fellow of the IEEE and ACM and the director of the NSF Frontier Center for Trustworthy Machine Learning. He also served as the program manager and lead scientist for the Army Research Laboratory's Cyber-Security Collaborative Research Alliance from 2013 to 2018. Patrick's research centrally focuses on a wide range of topics in computer and network security and technical public policy. Prior to joining Penn State in 2004, he was a senior research staff member at AT&T Labs-Research.
https://stonybrook.zoom.us/j/99820812332?pwd=c05BSTVLNmw3L04yZjdEcG5pem1OZz09 Speaker: Alexei Koulakov of Cold Spring Harbor Laboratory Brain evolution as a machine learning problem We have entered a golden age of artificial intelligence research, driven mainly by the advances in ANNs over the last decade or so. Applications of these techniques--to machine vision, speech recognition, autonomous vehicles, machine translation and many other domains--are coming so quickly that many observers predict that the long-elusive goal of Artificial General Intelligence (AGI) is within our grasp. However, we still cannot build a machine capable of building a nest, stalking prey, or loading a dishwasher. I will describe several projects, ranging from theories of evolution of neural development to the perception of smells, in which we are attempting to understand the algorithms that the nervous system is using to solve some of these challenging problems.

Yihui Ren

AI Department, Brookhaven National Laboratory

Abstract: Artificial intelligence is rapidly changing how we approach scientific discovery and engineering design. However, realizing the potential of modern AI in scientific domains requires going beyond simply applying models developed for natural language: scientific data have distinct structures, and scientific workflows often require tight integration among learned representations, domain knowledge, simulation, and numerical optimization.
In this talk, I will present two recent efforts that explore complementary directions for building AI systems for science and engineering. First, FM4NPP investigates whether the foundation-model paradigm can scale to experimental nuclear and particle physics. Using more than 11 million particle-collision events, we develop a self-supervised learning framework for sparse detector data and demonstrate scaling up to 188 million parameters. The resulting representations generalize across diverse downstream tasks and enable data-efficient adaptation with lightweight task-specific components.
Second, I will present AutoSizer, an LLM-driven framework for automatic sizing of analog and mixed-signal circuits. Rather than asking an LLM to perform numerical optimization directly, AutoSizer uses the LLM as a reasoning and orchestration layer that interprets circuit behavior, analyzes simulation feedback, and adaptively refines the optimization process.
Together, these studies illustrate two complementary roles for modern AI in scientific workflows: learning reusable representations from complex scientific data and reasoning over computational tools to guide challenging design and optimization problems. I will discuss the lessons from these efforts and opportunities toward more general, scalable, and reusable AI systems for science and engineering.

Short bio: Dr. Yihui (Ray) Ren is a Senior Computational Scientist in AI/ML and the AI-Codesign Group Lead in the Artificial Intelligence Department within Brookhaven National Laboratory's Computing and Data Sciences Directorate. His research focuses on AI for Science, AI-hardware codesign, real-time AI systems, and the evaluation of emerging AI hardware architectures. He also develops advanced AI methodologies for nuclear safeguards applications. Dr. Ren's key contributions include unpaired domain mapping techniques to mitigate domain shift, generative surrogate modeling, object detection and video synthesis using event-based cameras, and the development of AI foundation models such as the Foundation Model for Nuclear and Particle Physics (FM4NPP). Prior to joining Brookhaven National Laboratory in 2018, he was a postdoctoral researcher at Virginia Tech. He received his Ph.D. in Physics from the University of Notre Dame in 2015.

Location: NCS 120

Zoom: https://stonybrook.zoom.us/j/92507585369?pwd=9bw2hsQmLbladVpPRiaWK94gYlwCGQ.1&jst=2

Abstract: Facial emotion understanding aims to recognize, represent, and interpret human affect from facial behavior, and it is important for affective computing, human-computer interaction, digital humans, and mental health assessment. Existing work has represented facial emotion through discrete emotion categories, continuous affective dimensions such as valence and arousal, facial landmarks or geometry, and more recently semantic or language-based emotion descriptors. With the development of deep learning, supervised facial expression recognition has achieved strong performance on benchmark datasets, while self-supervised learning, multimodal large language models, and controllable facial generation have introduced new ways to learn emotion-related facial representations from images, videos, text, audio, and speech. However, many current models still rely heavily on manually annotated emotion labels, third-party perception judgments, or multimodal contextual cues, making it difficult to determine how much emotional information is captured directly from facial behavior itself, especially in naturalistic and clinically meaningful settings. Because these limitations make it challenging to evaluate whether facial representations capture emotionally meaningful behavior in real-world interactions, we propose to study facial emotion understanding through the relationship between facial behavior and language-derived emotional expression in psychiatric interview videos. Using a large dataset of mental health interviews, we extract multiple types of facial representations, including Action Unit features from FaceReader and OpenFace, non-AU facial behavior features from OpenFace, and 3D facial representations from EMOCA and SMIRK. We train segment-aligned transformer regressors to predict language-derived emotional targets, including valence, arousal, and RoBERTa-derived semantic-affective features from transcript segments. The results show that all facial representations achieve meaningful predictive performance across MSE and Pearson correlation metrics in both within-participant and between-participant evaluation settings. This indicates that facial behavior encodes information related to linguistic emotion at multiple levels: moment-to-moment emotional variation within individuals and broader affective differences across individuals. These findings suggest that structured facial representations can support vision-based emotion understanding in naturalistic mental health interviews and motivate future work on self-supervised, personalized, and controllable facial emotion models.

Speaker: Shao-Yu Chang

Zoom: https://stonybrook.zoom.us/j/3679036240?omn=98419305450
Abstract: Making large language models usable requires post-training methods that align models to human values, robustly handle underspecified inputs, and generalize to diverse instructions. This talk addresses the challenge of developing responsible AI through post-training from the following angles. First, I address contextual robustness. Preference data, for example, is often underspecified, and I show how underspecification in preference data can lead to diverging preferences. Standard reward models fail to properly handle these disagreements, often making decisive choices even when human annotators are split. I argue that more consequential outputs demand more context, and propose clarification question generation as one solution. Second, I will discuss how we can train models to be better instruction followers. I will show that most models severely overfit on a small set of instruction-following constraints and are not able to generalize well to unseen output constraints. I propose to train models with reinforcement learning from verifiable rewards for verifiable instruction following, and show how this leads to improved generalization on constraint following. Throughout the presentation, I will outline how I have applied these insights into developing open generative models, like Tülu and OLMo, and I will conclude with my research agenda for responsible post-training: critical AI evaluation, broader generalization, and expanding what models can reliably do.


Bio: Valentina Pyatkin is a postdoctoral researcher at the Allen Institute for AI and the University of Washington, advised by Prof. Hanna Hajishirzi and Prof. Yejin Choi. Additionally, she is part- time affiliated with the ETH AI Center, where she mentors students and works on post-training for the Swiss AI Initiative. She obtained her PhD in Computer Science from Bar Ilan University. Her work has been awarded an ACL Outstanding Paper Award and the ACL Best Theme Paper Award, and has been supported by a Schmidt Sciences Postdoctoral Award. During her doctoral studies, she conducted research internships at Google and the Allen Institute for AI, where she received the AI2 Outstanding Intern of the Year Award.

Location: NCS 120