Abstract: Many unresolved legal questions over LLMs and copyright center on memorization: whether specific training data have been encoded in the model's weights during training, and whether those memorized data can be extracted in the model's outputs. While many believe that LLMs do not memorize much of their training data, recent work shows that substantial amounts of copyrighted text can be extracted from open-weight models. However, it remains an open question if similar extraction is feasible for production LLMs, given the safety measures these systems implement. We investigate this question using a two-phase procedure: (1) an initial probe to test for extraction feasibility, which sometimes uses a Best-of-N (BoN) jailbreak, followed by (2) iterative continuation prompts to attempt to extract the book. We evaluate our procedure on four production LLMs -- Claude 3.7 Sonnet, GPT-4.1, Gemini 2.5 Pro, and Grok 3 -- and we measure extraction success with a score computed from a block-based approximation of longest common substring (nv-recall). With different per-LLM experimental configurations, we were able to extract varying amounts of text. For the Phase 1 probe, it was unnecessary to jailbreak Gemini 2.5 Pro and Grok 3 to extract text (e.g, nv-recall of 76.8% and 70.3%, respectively, for Harry Potter and the Sorcerer's Stone), while it was necessary for Claude 3.7 Sonnet and GPT-4.1. In some cases, jailbroken Claude 3.7 Sonnet outputs entire books near-verbatim (e.g., nv-recall=95.8%). GPT-4.1 requires significantly more BoN attempts (e.g., 20X), and eventually refuses to continue (e.g., nv-recall=4.0%). Taken together, our work highlights that, even with model- and system-level safeguards, extraction of (in-copyright) training data remains a risk for production LLMs.

Speaker: Xinyue

Location: CS2311

When: Thu: 10/28/2021, 10 am
Where: NCS Room 220, or
Zoom: https://stonybrook.zoom.us/j/97978463739?pwd=aVJFVERQa25jYjJrOFZEcWVuSzJLdz09

Deep Surface MeshesPascal FuaEPFLGeometric Deep Learning has recently made striking progress with the advent of Deep Implicit Fields (SDFs). They allow for detailed modeling of watertight surfaces of arbitrary topology while not relying on a 3D Euclidean grid, resulting in a learnable 3D surface parameterization that is not limited in resolution. Unfortunately, they have not yet reached their full potential for applications that require an explicit surface representation in terms of vertices and facets because converting the SDF to such a 3D mesh representation requires a marching-cube algorithm, whose output cannot be easily differentiated with respect to the SDF parameters. In this talk, I will discuss our approach to overcoming this limitation and implementing convolutional neural nets that output complex 3D surface meshes while remaining fully-differentiable and end-to-end trainable. I will also present applications to single view reconstruction, physically-driven Shape optimization, and bio-medical image segmentation.


Bio:
Pascal Fua received an engineering degree from Ecole Polytechnique, Paris, in 1984 and a Ph.D. in Computer Science from the University of Orsay in 1989. He joined EPFL (Swiss Federal Institute of Technology) in 1996 where he is a Professor in the School of Computer and Communication Science and head of the Computer Vision Lab. Before that, he worked at SRI International and at INRIA Sophia-Antipolis as a Computer Scientist. His research interests include shape modeling and motion recovery from images, analysis of microscopy images, and Augmented Reality. He has (co)authored over 300 publications in refereed journals and conferences. He has received several ERC grants. He is an IEEE Fellow and has been an Associate Editor of IEEE journal Transactions for Pattern Analysis and Machine Intelligence. He often serves as program committee member, area chair, and program chair of major vision conferences and has cofounded three spinoff companies.
The overall purpose of this seminar is to bring together people with interests in Computer Vision theory and techniques and to examine current research issues. This course will be appropriate for people who already took a Computer Vision graduate course or already had research experience in Computer Vision. To enroll in this course, you must either: (1) be in the PhD program or (2) receive permission from the instructors.

Each seminar will consist of multiple short talks (around 10 minutes) by multiple people. Students can register for 1 credit for CSE 656. Registered students must attend and present a minimum of 2 or 3 talks. Everyone else is welcome to attend. Fill in https://forms.gle/pCVXovgfMfQwGqG38 to subscribe to our mailing list for further announcement.
Abstract: Jailbreak attacks circumvent LLMs' built-in safeguards by concealing harmful queries within adversarial prompts. While most existing defenses attempt to mitigate the effects of adversarial prompts, they often prove inadequate as adversarial prompts can take arbitrary, adaptive forms. This paper introduces RobustKV, a novel jailbreak defense that takes a fundamentally different approach by selectively removing critical tokens of harmful queries from key-value (KV) caches. Intuitively, for an adversarial prompt to be effective, its tokens must achieve sufficient `importance' (measured by attention scores), which consequently lowers the importance of tokens in the concealed harmful query. Therefore, by carefully evicting the KVs of low-ranked tokens, RobustKV minimizes the harmful query's presence in the KV cache, thus preventing the LLM from generating informative responses. Extensive evaluation using benchmark datasets and models demonstrates that RobustKV effectively counters state-of-the-art jailbreak attacks while maintaining the LLM's performance on benign queries. Notably, RobustKV creates an interesting effectiveness-evasiveness dilemma for the adversary, leading to its robustness against adaptive attacks.

Speaker: Tanqiu Jiang

Where: NCS 220 and Zoom (https://stonybrook.zoom.us/j/6406956411)
Abstract: Neurosymbolic AI, as the combination of deep learning and knowledge representation/reasoning methods, is currently receiving significant attention due to its promise to both overcome apparent limitations of pure deep learning systems, and to address the knowledge acquisition bottleneck. This presentation will focus on some recent reseach advances made at Kansas State University related to Semantic Web / knowledge graphs and neurosymbolic AI, with particular emphasis on the use of symbolic AI methods for understanding hidden neuron activations, and on using large language models for knowledge graph and ontology engineering.

Short bio: Pascal Hitzler is University Distinguished Professor and endowed Lloyd T. Smith Creativity in Engineering Chair at the Department of Computer Science at Kansas State University, one of the Directors of the Institute for Digital Agriculture and Advanced Analytics (ID3A), and Director of the Center for Artificial Intelligence and Data Science (CAIDS). His research record lists over 400 publications in such diverse areas as neurosymbolic artificial intelligence, semantic web, knowledge graphs, knowledge representation and reasoning, denotational semantics, and set-theoretic topology. He was founding Editor-in-chief of the Semantic Web journal, the leading journal in the field, and is founding Editor-in-chief of the new Neurosymbolic Artificial Intelligence journal. He is co-author of the W3C Recommendation OWL 2 Primer, and of the book Foundations of Semantic Web Technologies by CRC Press, 2010, which was named as one out of seven Outstanding Academic Titles 2010 in Information and Computer Science by the American Library Association's Choice Magazine, and has translations into German and Chinese. He is a founding steering committee member of the Neural-Symbolic Learning and Reasoning Association that runs the annual International Conference on Neurosymbolic Learning and Reasoning (NeSy). For more information about him, see http://www.pascal-hitzler.de.

Location: NCS 120

Zoom: https://stonybrook.zoom.us/j/94657223635?pwd=FHtIvIupDAbQTCKzpDF3kTg6xRZOtj.1&jst=2

From instant question generation to automated data summaries, learn how to leverage Gemini to simplify your entire Google Forms workflow.

Discover how Gemini can now work in Google Forms. Start by creating a whole Google Form from a prompt. Then, watch the real magic after the data is in: We'll dive into Gemini's ability to analyze and summarize open-ended responses, turning a mountain of feedback into a concise, actionable information. Whether you're gathering student feedback or organizing departmental requests, this session will show you how to let AI do the busy work so you can focus on the results.

In this session, you will:

  1. Generate a Google Form from a prompt
  2. Get question suggestions
  3. Summarize text responses with AI

Register here.
Come learn of the exciting research being done across so many fields using AI! The recipients of AI3's seed awards will present their work in our showcase on November 17, 2025 and we would love to see you there!

The schedule is listed below.

Location: New Computer Science Room 120

Session 1 - 10:30 AM to 11:45

Kevin Reed, PI, Introducing the AI Techniques in Assessing the Future Changes of Extreme Precipitation and Associated Flood Risks
Co-PIs: Tangnyu Song, Ishrat Dollan
Consultant: Jayesh Rathi

Ruwen Qin, PI, AI-Assisted Analysis of Materials in Recycling Streams
Consultant: Vismay Vora

Giuseppe Gazzola, PI, Using AI to Investigate National Literatures: Italy, France, Spain 1733- 1794
Consultant: Jayesh Rathi

Joseph Lemelin, PI, IAE2^3: AI Ecologies
Co-PIs: Katherine Johnston, Aruna Balasubramanian, Matthew Salzano

Niranjan Balasubramanian, Co-PI, Molecular Foundations for Sustainability: Data Analytics for Sustainable Cellulose Scaffolding Modifications to Remediate Diverse Water Contamination Challenges
PI: Benjamin Hsiao, Co-PI: I. V. Ramakrishnan

Owen Rambow, PI,Achieving Common Ground Through Language and Vision in Mixed-Initiative Human-Machine Communication Via zoom
Co-PI Susan Brennan

Session 2 - 12:30 PM to 1:45

Jack McSweeney, PI, Developing Machine Learning Approaches to Classify Internal Waves
Consultant: Vismay Vora

Eric Josephs, PI, Learning Design Rules to Personalize Precision CRISPR Gene Therapies with Interpretable AI
Consultant: Deboparna Banerjee

Shyam Sharma, PI, Fostering Writing-to-Learn Skills through Critical AI Literacy: A Faculty Development and Student Support Program
Co-PIs: Rose Tirotta-Esposito, Christine Fena

Ritwik Banerjee, PI, A Pragmatic Approach to AI for Digital Media Integrity: Combating Complex Misinformation Through Fallacies and Propaganda
Co-PI: Ruobing Li

Ziyu Shu, Co-PI, Novel Clinical Applications of Deep Image Prior-based CT Image Reconstruction
PI: Xin Qian, Co-PIs: Tiezhi Zhang, Zhaozheng Yin

Prateek Prasanna, Co-PI, An Artificial Intelligence-Driven Clinical Decision Support Tool for the Management of Abdominal Aortic Aneurysm
PI: Apostolos Tassiopoulos, Co-PI's: Mary Saltz, Janos Hajagos, Tahsin Kurc



Presenters will give a 5-minute talk with 2 minutes for Q & A.
The Natural Language Processing Reading Group at Stony Brook University meets weekly to discuss recent research papers in NLP and related fields.
Join the Google Group here.

Learning to Evict from Key-Value Cache


Abstract: The growing size of Large Language Models (LLMs) makes efficient inference challenging, primarily due to the memory demands of the autoregressive Key-Value (KV) cache. Existing eviction or compression methods reduce cost but rely on heuristics, such as recency or past attention scores, which serve only as indirect proxies for a token's future utility and introduce computational overhead. We reframe KV cache eviction as a reinforcement learning (RL) problem: learning to rank tokens by their predicted usefulness for future decoding. To this end, we introduce KV Policy (KVP), a framework of lightweight per-head RL agents trained on pre-computed generation traces using only key and value vectors. Each agent learns a specialized eviction policy guided by a holistic reward, derived from future utility, that evaluates the quality of the ranking across all cache budgets, requiring no modifications to the underlying LLM or additional inference. Evaluated across two model families on the long-context benchmark RULER (up to 128K tokens) and the multi-turn dialogue benchmark OASST2-4k, KVP significantly outperforms strong baselines. Zero-shot tests on standard downstream tasks (BoolQ, LongBench passage retrieval, GovReport) further show that KVP generalizes beyond its training distribution and to considerably longer sequence lengths. These results demonstrate that learning to predict future token utility is a powerful and scalable paradigm for adaptive KV cache management.

Location: NCS 220

Zoom Link: https://stonybrook.zoom.us/j/92942833294?pwd=ysKMaQx7Hp0lkq5nOb3zJ9ivNZVvLv.1&jst=2

The Della Pietra Lecture Series is pleased to present this lecture by Scott Aaronson:

Abstract: I'll survey some areas where I think theoretical computer science, math, and statistics can potentially contribute to the urgent quest to align powerful AI with humane values. These areas include: the watermarking of AI outputs, mechanistic interpretability (including Paul Christiano's No-Coincidence Principle, and succinct digests of the training process to aid interpretability), and theoretical guarantees for out-of-distribution generalization.

Speaker: Scott Aaronson is Schlumberger Chair of Computer Science at the University of Texas at Austin, and founding director of its Quantum Information Center. He received his bachelor's from Cornell University and his PhD from UC Berkeley. Aaronson's research has focused mainly on the capabilities and limits of quantum computers. His first book, Quantum Computing Since Democritus, was published in 2013 by Cambridge University Press. He received the National Science Foundation's Alan T. Waterman Award, the United States PECASE Award, the Tomassoni-Chisesi Prize in Physics, and the ACM Prize in Computing, and is a Fellow of the ACM and the AAAS and a member of the National Academy of Sciences. He blogs at Shtetl-Optimized, https://www.scottaaronson.com/blog.

Location: Della Pietra Family Auditorium (SCGP 103)