Abstract : Humans reason about everyday situations by making commonsense-based inferences, derived both from explicitly stated information and implicit, unstated knowledge. In this thesis, I investigate whether NLP models have different aspects of causal knowledge about events and how to improve their understanding of narratives and plans.
Answering questions about why people perform actions in a narrative can test whether NLP systems contain and can effectively apply causal knowledge about events. I introduce TellMeWhy, a dataset concerning why characters in short narratives perform the actions described. An evaluation of then SOTA finetuned models show that they are far worse than humans. To improve models, it is important to understand what aspects of causal knowledge they need and how to best use external sources to inject this knowledge. In KnowWhy, I analyze different ways of injecting knowledge into models, which is difficult since we do not know apriori what type of knowledge will be needed to answer a question, hence requiring a ranking model to pick the most important inference. Results show that this retrieved knowledge helps models of all sizes, thereby improving their understanding of narratives.
Next, I study whether models can reason about causal aspects of plans. I focus on testing whether they understand the underlying causal dependencies reflected in the temporal order of a plan's steps. I introduce CAT-Bench, and find that SOTA models are underwhelming, and that model answers are not consistent across questions about the same step pairs. In their current state, these models cannot yet reliably be used for complex user-facing tasks. I then measure contemporary models' ability to perform user-facing and user-centric plan customization. I introduce the use of semi-symbolic edits in large language model (LLM) based agents and test several multi-LLM-agent architectures for plan customization. While LLMs still lack the ability to understand complex customization hints, my results suggest that LLM-based architectures may be worth exploring further for other customization applications. Finally, I distill complex reasoning capabilities into small language models (SLMs) using synthetic data that reflects a decomposition-then-editing process for plan customization. I demonstrate that explicitly teaching this latent causal reasoning significantly improves the quality of SLM-generated customizations. Overall, my work has improved how well NLP models understand complex reasoning associated with events in different contexts.

Speaker: Yash Kumar Lal

Location: NCS 220 or Zoom https://stonybrook.zoom.us/j/95849648243?pwd=dgPpZtDpgwQrK9z1SaPpNbBifaorzk.1
Tired of drowning in grading? Discover how Gradescope can reduce grading time by 30%-50% for exams, homework, and online assignments. In this 90-minute session, learn to grade efficiently with AI-assisted tools, update rubrics easily, manage paperless submissions, and use analytics to identify learning gaps.

This session will go over:

  • Grade items simultaneously: Use AI-assisted grading to group identical student answers and grade them all at once.
  • Edit rubrics on the fly: Adjust your rubric points at any time, and let Gradescope automatically update previously graded papers retroactively.

  • Go paperless: Effortlessly manage student-submitted PDFs or instructor-scanned exams without the clutter.

  • Spot learning gaps: Use instant analytics dashboards to see exactly which concepts your students mastered and where they struggled.


Register here by 09/13 to attend the Zoom training session.
Title:Deep Contextual Modeling for Natural Language Understanding, Generation, and Grounding Zoom instructions: Join Zoom Meeting https://stonybrook.zoom.us/j/645050299?pwd=TVJVRkc3dlhxdDF5d00xWGlDQkovZz09 Meeting ID: 645 050 299 Password: 810247 One tap mobile +16468769923,,645050299#,,#,810247# US (New York) +13126266799,,645050299#,,#,810247# US (Chicago) Dial by your location +1 646 876 9923 US (New York) +1 312 626 6799 US (Chicago) +1 301 715 8592 US +1 346 248 7799 US (Houston) +1 408 638 0968 US (San Jose) +1 669 900 6833 US (San Jose) +1 253 215 8782 US Meeting ID: 645 050 299 Password: 810247 Find your local number: https://stonybrook.zoom.us/u/aemTiJMXu6 Abstract: Natural language is a fundamental form of information and communication. In both human-human and human-computer communication, people reason about the context of text and world state to understand language and produce language response. In this talk, I present several deep neural network based systems that first understand the meaning of language grounded in various contexts where the language is used, and then generate effective language responses in different forms for information access and human-computer communication. First, I will introduce Speaker Interaction RNNs for addressee and response selection in multi-party conversations based on explicit representations for different discourse participants. Then, I will present a text summarization approach for generating email subject lines by optimizing quality scores in a reinforcement learning framework. Finally, I will show an editing-based multi-turn SQL query generation system towards intelligent natural language interfaces to databases. Bio:Rui Zhang is a final year Ph.D. student at Yale University advised by Professor Dragomir Radev. His research interest lies in various natural language processing problems in understanding, generation, and grounding. He has been working on (1) End-to-End Neural Modeling for Entities, Sentences, Documents, and Multi-party Multi-turn Dialogues, (2) Text Summarization for Emails, News, and Scientific Articles, (3) Cross-lingual Information Retrieval for Low-Resource Languages, (4) Context-Dependent Text-to-SQL Semantic Parsing in Human-Computer Interaction. Rui Zhang has published papers and served as Program Committee members at top-tier NLP and AI conferences including ACL, NAACL, EMNLP, AAAI, CoNLL. During his Ph.D., He has done research internships at IBM Thomas J. Watson Research Center, Grammarly Research, and Google AI. He was a graduate student at the University of Michigan and got his bachelor's degrees at both the University of Michigan and Shanghai Jiao Tong University from the UM-SJTU Joint Institute.

Join University Libraries for an engaging panel discussion where we delve in and learn about the impacts of artificial intelligence on the 2024 US elections! Panelists are Paige Lord, Tom Costello, and Musa al-Gharbi. The discussion will be moderated by Library Dean, Karim Boughida. Co-sponsored by the Office of Diversity, Inclusion, and Intercultural Initiatives.

Please RSVP for Democracy in the Digital Age: AI's Influence on 2024 Elections here.
Title: Cyberinfrastructure for forward prediction and inversion estimation with uncertainty quantification

Seminar Speaker: Dr. Mengyang Gu, Assistant Professor, Department of Statistics and Applied Probability, University of California, Santa Barbara

Abstract: In this talk, we introduce four useful tools for forward prediction and inversion estimation. The first tool is the parallel partial Gaussian process surrogate model for emulating expensive computer simulations with massive coordinates. The tool is implemented in the RobustGaSP package available in R, MATLAB, and Python, for predicting both scalar- and vector-valued outputs with uncertainty assessment. The second tool is implemented in the RobustCalibration package, which handles Bayesian data inversion or model calibration by one or multiple types of experimental observations. A unique feature of the package is the inclusion of fast surrogate models of both scalar- and vector-valued computer simulations that bypass the expensive simulation in one line of code. The third tool is implemented in the AIUQ package, available in both R and MATLAB. In this approach, we show that differential dynamic microscopy, a scattering-based analysis tool that extracts dynamical information from microscopy videos, is equivalent to fitting the temporal auto-covariance in Fourier space, based on a latent factor model we construct. We develop a more efficient estimator and reduce the computational cost to pseudolinear order with respect to the number of observations without approximation, by utilizing the generalized Schur algorithm for the Toeplitz covariance. In the last tool, we developed a new method called the inverse Kalman filter, which enables fast matrix-vector multiplication between a covariance matrix from a dynamic linear model and any real-valued vector with a linear computational cost. These new approaches outline a wide range of applications that include emulating expensive simulation at molecular-, meso- and macro-scales, active learning with error control, nonparametric estimation of particle interaction functions, and data inversion from microscopy and velocity fields.

Join Zoom Meeting: https://bnl.zoomgov.com/j/1606285496?pwd=2yJYSG6lx8gMPiibzgAIBQtKHIjuHV.1
Meeting ID: 160 628 5496
Passcode: 472506
Research challenges in using computer vision in robotics systems Abstract The past decade has seen a remarkable increase in the level of performance of computer vision techniques, including with the introduction of effective deep learning techniques. Much of this progress is in the form of rapidly increasing performance on standard, curated datasets. However, translating these results into operational vision systems for robotics applications remains a formidable challenge. This talk with explore some of the fundamental questions at the boundary between computer vision and robotics that need to be addressed. This includes introspection/self-awareness of performance, anytime algorithms for computer vision, multi-hypothesis generation, rapid learning and adaptation. The discussion will be illustrated by examples from autonomous air and ground robots.
Fall 2026, Wednesdays 2 to 3:20 pm, NCS 220 and Zoom link to be announced soon.

The seminar will be jointly taught by Prof. Dimitris Samaras (samaras@cs.stonybrook.edu).

The overall purpose of this seminar is to bring together people with interests in Computer Vision theory and techniques and to examine current research issues. This course will be appropriate for people who already took a Computer Vision graduate course or already had research experience in Computer Vision.

To enroll in this course, you must either: (1) be in the Ph.D. program or (2) receive permission from the instructors.

Each seminar will consist of multiple short talks (around 15 minutes) by multiple students. Students can register for 1 credit for CSE656. Registered students must attend and present a minimum of 2 talks. Registered students must attend in person. Up to 3 absences will be excused. Everyone else is welcome!
AI3, SBU Libraries and IACS present
at International Love Data Week
sponsored by The Office of the Provost and
Educational and Institutional Effectiveness (EIE)

Special Talk and Panel Discussion

How I Learned to Stop Worrying and Love AI (For Now)


with Paul Fain from The Job and Work Shift

A reporter's take on what we know--and what we don't know--about AI's emerging impacts on the labor market. The discussion will include the latest research from economists and the AI labs themselves about how workers are using AI, and current thinking among experts on how the tech's rapid deployment will play out across job roles, industries, and regions.

Panel discussion to follow with:

  • Lav Varshney, Della Pietra Infinity Professor and inaugural director of the AI Innovation Institute
  • Nicholas Johnson, Director of AI, SBU Libraries
  • Marianna Savoca, Associate Vice President for Career Readiness and Experiential Education
Paul Fain is co-founder of Work Shift, editor of the must-read newsletter, The Job, and host of The Cusp podcast. A veteran higher education reporter, Paul is perhaps the nation's top journalist focused on connections between education and work. He started Work Shift after a decade as a senior reporter and then news editor at Inside Higher Ed, where he led the outlet's coverage of low-income and first-generation students, college completion, community colleges, federal policy, and emerging models of higher education. He also was the founding host of the successful podcast, The Key with Inside Higher Ed, and has contributed chapters for books on innovation in higher education, published by the Harvard University Press and the Stanford University Press. Earlier in his career, Paul was a senior reporter at The Chronicle of Higher Education.

Limited Seats!

Registration is required.