Abstract: Retrieval-augmented generation (RAG) systems empower large language models (LLMs) to access external knowledge during inference. Recent advances have enabled LLMs to act as search agents via reinforcement learning (RL), improving information acquisition through multi-turn interactions with retrieval engines. However, existing approaches either optimize retrieval using search-only metrics (e.g., NDCG) that ignore downstream utility or fine-tune the entire LLM to jointly reason and retrieve--entangling retrieval with generation and limiting the real search utility and compatibility with frozen or proprietary models. In this work, we propose s3, a lightweight, model-agnostic framework that decouples the searcher from the generator and trains the searcher using a Gain Beyond RAG reward: the improvement in generation accuracy over naïve RAG. s3 requires only 2.4k training samples to outperform baselines trained on over 70 × more data, consistently delivering stronger downstream performance across six general QA and five medical QA benchmarks.

Speaker: Peter Zeng

Location: CS2311

This session brings together the scientists, agencies, and community partners generating environmental data across New York City to confront a shared challenge: critical atmospheric and marine data is being collected across the region, but too often in silos that limit its reach and impact.

​The program uses Governors Island's own environmental sensing efforts as a working case study, then opens into a broader conversation about how disparate data streams across NYC can be coordinated, shared, and activated--pointing toward the kind of cross-sector collaboration The Exchange's campus is designed to enable.

Register here.

The University's Main Commencement Ceremony will take place on Friday, May 23, 2025 at 11 am at Kenneth P. LaValle Stadium. Gates open at 10 am.

All guests need a valid ticket to enter LaValle Stadium - no exceptions. Children age 1 and older require a ticket. Seating is first-come, first-served.

Register here.

The overall purpose of this seminar is to bring together people with interests in Computer Vision theory and techniques and to examine current research issues. This course will be appropriate for people who already took a Computer Vision graduate course or already had research experience in Computer Vision. To enroll in this course, you must either: (1) be in the PhD program or (2) receive permission from the instructors.

Each seminar will consist of multiple short talks (around 10 minutes) by multiple people. Students can register for 1 credit for CSE 656. Registered students must attend and present a minimum of 2 or 3 talks. Everyone else is welcome to attend. Fill in https://forms.gle/pCVXovgfMfQwGqG38 to subscribe to our mailing list for further announcement.
Virtual Talk: Contextual Modeling for Natural Language Understanding, Generation and Grounding by Rui Zhang

Zoom link to come.

Abstract: Natural language is a fundamental form of information and communication. In both human-human and human-computer communication, people reason about the context of text and world state to understand language and produce language response. In this talk, I present 
several deep-neural-network-based systems that first understand the meaning of language grounded in various contexts where the language is used, and then generate effective language responses in different forms for information access and human-computer communication. First, 
I will introduce Speaker Interaction RNNs for addressee and response selection in multi-party conversations based on explicit representations for different discourse participants. Then, I will 
present a text summarization approach for generating email subject lines by optimizing quality scores in a reinforcement learning framework. Finally, I will show an editing-based multi-turn SQL query generation system towards intelligent natural language interfaces to databases. 

Bio: Rui Zhang is a final-year PhD student at Yale University advised by Professor Dragomir Radev. His research interest lies in various natural language processing problems in understanding, generation, and grounding. He has been working on (1) End-to-End Neural Modeling for Entities, Sentences, Documents and Multi-party Multi-turn Dialogues, (2) Text Summarization for Emails, News and Scientific Articles, (3) Cross-lingual Information Retrieval for Low-Resource Languages, (4) Context-Dependent Text-to-SQL Semantic Parsing in Human-Computer Interaction. Rui Zhang has published papers and served as Program Committee members at top-tier NLP and AI conferences including ACL, NAACL, EMNLP, AAAI and CoNLL. During his PhD, he has done research internships at IBM Thomas J. Watson Research Center, Grammarly Research and Google AI. He was a graduate student at the University of Michigan and got his Bachelor's degrees at both the University of Michigan and Shanghai Jiao Tong University from the UM-SJTU Joint Institute.

Abstract: This talk is about the two ends of LLM training: pre-training and in-deployment learning. I will present an approach to disentangle knowledge from skill in model pre-training. This brings about a new class of LLMs that externalize knowledge, with dramatically different characteristics from common LLMs along dimensions of scale, factuality, and updateability. On the other end, I will discuss two in-deployment learning methods. I will describe how in-context learning abilities extend beyond supervised settings, showing that LLMs display in-context reinforcement learning from rewards. Finally, if time allows, I will describe continual learning from implicit interaction signals, demonstrating that LLMs can retrospectively decode latent interaction cues by observing how humans respond to their outputs.

Bio: Yoav Artzi is an Associate Professor in the Department of Computer Science and Cornell Tech at Cornell University, a visiting faculty researcher at Google DeepMind, and arXiv's associate faculty director. His research focuses on language modeling and learning in interactive and situated scenarios. His work was acknowledged by awards and honorable mentions at ACL, EMNLP, NAACL, and IROS, as well as a TACL test-of-time award. Yoav holds a B.Sc. from Tel Aviv University and a Ph.D. from the University of Washington.

Location: NCS 120