Abstract: This talk is about the two ends of LLM training: pre-training and in-deployment learning. I will present an approach to disentangle knowledge from skill in model pre-training. This brings about a new class of LLMs that externalize knowledge, with dramatically different characteristics from common LLMs along dimensions of scale, factuality, and updateability. On the other end, I will discuss two in-deployment learning methods. I will describe how in-context learning abilities extend beyond supervised settings, showing that LLMs display in-context reinforcement learning from rewards. Finally, if time allows, I will describe continual learning from implicit interaction signals, demonstrating that LLMs can retrospectively decode latent interaction cues by observing how humans respond to their outputs.

Bio: Yoav Artzi is an Associate Professor in the Department of Computer Science and Cornell Tech at Cornell University, a visiting faculty researcher at Google DeepMind, and arXiv's associate faculty director. His research focuses on language modeling and learning in interactive and situated scenarios. His work was acknowledged by awards and honorable mentions at ACL, EMNLP, NAACL, and IROS, as well as a TACL test-of-time award. Yoav holds a B.Sc. from Tel Aviv University and a Ph.D. from the University of Washington.

Location: NCS 120

AI on Campus: Your Thoughts, Your Future

Join the Conversation: Share Your Thoughts about Learning, Academics, and AI

The world of college is changing fast, and Artificial Intelligence (AI) is at the center of it. We are part of the Institute on AI, Pedagogy, and the Curriculum with AAC&U, and we need to hear from the people AI affects most: you!

This is an open discussion for all students to share their honest experiences, their top concerns, and their best ideas about AI in our academic environment. We'll be diving into these key questions:

  • How can AI actually make learning better or easier? What opportunities do you see for using AI tools to enhance your assignments, research, or skills?

  • What are your biggest worries about AI? Is it about cheating, being graded fairly, or preparing for the job market? How is AI impacting your workload or stress levels?

  • What specific tools, workshops, or policies would help you use AI responsibly and successfully? (Think training, software, or clear rules.)

Dates/Times:

  • Wednesday, 2/4 at 2pm

  • Thursday, 2/5 at 12pm

Please register in advance for the Zoom link.

Can't Make It? Share Your Feedback!

Don't worry if you can't attend! You can still share your thoughts via video in our AI Zoom Room or via email: rose.tirotta-esposito@stonybrook.edu.

Videos will not be shared publicly and comments will only be shared in aggregate.

Your voice matters. Come tell us how AI is affecting your studies, your stress, and your success!

  • Dr. Rose Tirotta-Esposito (Assistant Provost; Director of CELT)

  • Dr. Elizabeth Hewitt (Associate Professor in the Department of Technology and Society (DTS) in the College of Engineering and Applied Sciences)

  • Chris Kretz (Associate Librarian and Head of Academic Engagement at SBU Libraries)

  • Prof. Rajiv Lajmi (Assistant Professor in the School of Health Professions and Chair of Applied Health Informatics)

  • Dr. Matthew Salzano (Assistant Professor in the Department of Communication in the School of Communication and Journalism)


Please join University Libraries on March 29 at 1:00 via Zoom as we welcome Dr. Zhang, SUNY Empire Innovation Professor at SBU's Power Lab. This lab is pioneering the research of coordinated networked microgrids (NMs) that can possibly help to restore neighboring distribution grids after a major blackout. That these NMs hold promise to significantly enhance the day-to-day reliability of the power grids, we are proud to host Dr. Zhang as a member of our STEM Speaker Series. Registration required.
https://library.stonybrook.edu/library-events/stem-speaker-series-ai-enabled-provably-resilient-networked-microgrids-with-dr-peng-zhang/

Reception to follow.

Abstract:
In this talk, I will present our journey of developing diverse, adaptive, uncertainty-calibrated AI planning agents that can robustly communicate and collaborate for multi-agent reasoning (on math, commonsense, coding, etc.) as well as for interpretable, controllable multimodal generation (across text, images, videos, audio, layouts, etc.). In the first part, we will discuss improving reasoning via multi-agent discussion among diverse LLMs and structured distillation of these discussion graphs (ReConcile, MAGDi), adaptively learning to balance abstraction, decomposition, refinement, and fast+slow thinking in LLM-agent reasoning (ReGAL, ADaPT, MAgICoRe, System-1.x), as well as confidence calibration in LLMs via speaker-listener pragmatic reasoning and making LLMs better teammates via multi-agent positive-negative persuasion balancing (LACIE, PBT). In the second part, we will discuss interpretable and control-lable multimodal generation via LLM-agents based planning and programming, such as layout-controllable image generation (and evaluation) via visual programming (VPGen+VPEval), consistent multi-scene video generation via LLM-guided planning (VideoDirectorGPT), interactive and composable any-to-any multimodal generation (CoDi, CoDi-2), as well as feedback-driven multi-agent interaction for adaptive environment/data generation via weakness discovery (EnvGen, DataEnvGym).
Bio:
Dr. Mohit Bansal is the John R. & Louise S. Parker Distinguished Professor and the Director of the MURGe-Lab (UNC-NLP Group) in the Computer Science department at UNC Chapel Hill. He received his PhD from UC Berkeley in 2013 and his BTech from IIT Kanpur in 2008. His research expertise is in natural language processing and multimodal machine learning, with a particular focus on multimodal generative models, grounded and embodied semantics, faithful language generation, and interpretable, efficient, and generalizable deep learning.
The INS (International Neuroethics Society) AI and Consciousness Affinity Group is hosting a talk titled Bringing Trustworthiness in Generative AI and Agentic AI Using Thought Knowledge Graphs featuring speaker Manas Gaur, a computer scientist at UMBC.
The talk will examine the interplay between Thought Knowledge Graphs (TKGs) and how they can form more trustworthy and reasoning-based responses in AI. They will also discuss introducing novel methods on implementing TKGs and their overall impact on creating more trustworthy AI systems.
The talk will be held online via Zoom on Monday, December 2 at 1:00pm (EST).
Register to attend.
Abstract: Current Chain-of-Thought (CoT) verification methods predict reasoning correctness based on outputs (black-box) or activations (gray-box), but offer limited insight into \textit{why} a computation fails. We introduce a white-box method: \textbf{Circuit-based Reasoning Verification (CRV)}. We hypothesize that attribution graphs of correct CoT steps, viewed as \textit{execution traces} of the model's latent reasoning circuits, possess distinct structural fingerprints from those of incorrect steps. By training a classifier on structural features of these graphs, we show that these traces contain a powerful signal of reasoning errors. Our white-box approach yields novel scientific insights unattainable by other methods. (1) We demonstrate that structural signatures of error are highly predictive, establishing the viability of verifying reasoning directly via its computational graph. (2) We find these signatures to be highly domain-specific, revealing that failures in different reasoning tasks manifest as distinct computational patterns. (3) We provide evidence that these signatures are not merely correlational; by using our analysis to guide targeted interventions on individual transcoder features, we successfully correct the model's faulty reasoning. Our work shows that, by scrutinizing a model's computational process, we can move from simple error detection to a deeper, causal understanding of LLM reasoning.

Speaker: Xianjun Yang

Location: Old Computer Science Building - CS2311

You are cordially invited to attend the biweekly Brookhaven AI Mixer (BAM). BAM includes one short talk on AI research happening at BNL, followed by an open mixer over coffee and snacks for everyone to network and discuss all things AI. The first half hour will consist of presentations that will be available via ZOOM, and the second half hour will be for in person only networking.

Join us every other Tuesday at noon in CDSD's Training Room (building 725, 2nd floor) to learn about interesting AI methods and applications, engage with potential collaborators, prepare for pending FASST funding calls, and build a community of AI for Science at BNL.

Abstract: The increasing complexity and volume of data from electron microscopy necessitates advanced computational tools for timely and accurate analysis. In this talk, I will present several machine learning (ML) models developed to interpret diverse datasets from transmission electron microscopy (TEM). First, I demonstrate segmentation models for labelling regions of interest from in situ TEM images, such as atomic column positions or reaction sites that allow atomic-level quantitative analysis of data. Second, I introduce a self-supervised CNN model for denoising of low-dose HRTEM images, enabling clearer visualization of atomic features without sacrificing temporal resolution. Finally, a transformer-based model trained to predict copper oxidation states directly from their electron energy loss spectroscopy spectra will be introduced. Together, these projects showcase the power of tailored ML solutions to extract quantitative insights from complex microscopy data.

Biography: Brian Lee is a research associate working for the Electron Microscopy group and Theory and Computation group at the Center for Functional Nanomaterials. Previously, he has received PhD in Mechanical Engineering from Duke University and worked as a postdoc at Purdue University. His research focuses on applying machine learning and simulation techniques for materials science.

Location: CDS, Bldg. 725, Training Room

Join ZoomGov Meeting: https://bnl.zoomgov.com/j/1604383624?pwd=ffQ5cUPNxTI7nzClKQO6cnsNbhF9Vf.1

Meeting ID: 160 438 3624
Passcode: 558449

Yihui Ren

AI Department, Brookhaven National Laboratory

Abstract: Artificial intelligence is rapidly changing how we approach scientific discovery and engineering design. However, realizing the potential of modern AI in scientific domains requires going beyond simply applying models developed for natural language: scientific data have distinct structures, and scientific workflows often require tight integration among learned representations, domain knowledge, simulation, and numerical optimization.
In this talk, I will present two recent efforts that explore complementary directions for building AI systems for science and engineering. First, FM4NPP investigates whether the foundation-model paradigm can scale to experimental nuclear and particle physics. Using more than 11 million particle-collision events, we develop a self-supervised learning framework for sparse detector data and demonstrate scaling up to 188 million parameters. The resulting representations generalize across diverse downstream tasks and enable data-efficient adaptation with lightweight task-specific components.
Second, I will present AutoSizer, an LLM-driven framework for automatic sizing of analog and mixed-signal circuits. Rather than asking an LLM to perform numerical optimization directly, AutoSizer uses the LLM as a reasoning and orchestration layer that interprets circuit behavior, analyzes simulation feedback, and adaptively refines the optimization process.
Together, these studies illustrate two complementary roles for modern AI in scientific workflows: learning reusable representations from complex scientific data and reasoning over computational tools to guide challenging design and optimization problems. I will discuss the lessons from these efforts and opportunities toward more general, scalable, and reusable AI systems for science and engineering.

Short bio: Dr. Yihui (Ray) Ren is a Senior Computational Scientist in AI/ML and the AI-Codesign Group Lead in the Artificial Intelligence Department within Brookhaven National Laboratory's Computing and Data Sciences Directorate. His research focuses on AI for Science, AI-hardware codesign, real-time AI systems, and the evaluation of emerging AI hardware architectures. He also develops advanced AI methodologies for nuclear safeguards applications. Dr. Ren's key contributions include unpaired domain mapping techniques to mitigate domain shift, generative surrogate modeling, object detection and video synthesis using event-based cameras, and the development of AI foundation models such as the Foundation Model for Nuclear and Particle Physics (FM4NPP). Prior to joining Brookhaven National Laboratory in 2018, he was a postdoctoral researcher at Virginia Tech. He received his Ph.D. in Physics from the University of Notre Dame in 2015.

Location: NCS 120

Zoom: https://stonybrook.zoom.us/j/92507585369?pwd=9bw2hsQmLbladVpPRiaWK94gYlwCGQ.1&jst=2

Predicting Subjective Attributes in Visual Data - Zijun Wei

ABSTRACT: Recent progress in deep neural networks has revolutionized many computer vision tasks such as image classification, detection and segmentation. However, in addition to excelling in tasks that predict well-defined objective information, human-centered artificial intelligence systems should also be able to model subjective attributes, as defined by human perceptual behavior, that goes beyond the pure physical content of visual data. Example subjective tasks are the prediction of spatial or temporal regions that are interesting to humans (e.g., attract attention or are visually pleasing) and the recognition of subjective attributes (e.g., visually elicited sentiments). Better models for these tasks will improve the human-computer interaction experience in various applications. This thesis investigates several approaches to address the challenges in predicting those subjective attributes in visual data over a diverse set of tasks. I first present a novel framework for real-time automatic photo composition. The framework consists of a cost-effective data collection workflow, an efficient model training pipeline and a lightweight module to account for personalized preferences. Then I develop a novel and general algorithm to detect interesting segments in sequential data, which can be naturally applied to video summarization tasks. Furthermore, I propose methods that learn to represent sentiments elicited by images, in an unsupervised manner, using linguistic features extracted from large scale Web data. To conclude this thesis, I introduce a human-vision-inspired image classification algorithm that also predicts spatial visual attention even though no attention data was used for training it.  
CSE 600 Seminar Series | Fall 2025



Abstract:

We often talk about AI as if it begins with a dataset and ends with an application. But behind every model lie two sets of actors who are rarely acknowledged in technical documentation: the workers who train AI systems and the researchers who try to make sense of them. This talk brings both groups into view.
Dr. Ben Zhang will offer an on-the-ground examination of the prevailing values and invisible labor that underpin commercial AI production and data production. Drawing on ethnographic research inside AI data annotation centers in China, he introduces the concept of precision labor to unpack the labor dimension of constructing, managing, and performing technical accuracy. This concept highlights the hidden and excessive labor required to reconcile the ambiguity and uncertainty involved in AI training. A precision labor lens challenges the legitimacy and sustainability of the relentless pursuit of technical accuracy, raising new questions about its consequences and implications.
On the other end of the pipeline, as LLMs become embedded in society, social scientists like Dr. Jieshu Wang is scrutinizing their potential biases while employing them as research tools. She will present her recent work auditing LLM responses across different contexts, revealing that LLMs exhibit varying levels of environmental awareness and disproportionately reward institutional prestige in peer-review simulations. She also demonstrates how LLMs can serve as useful tools in social-science pipelines, e.g., extracting location information, inferring demographics, parsing citations, mapping social networks, and analyzing occupational data.
By placing these two worlds side by side - the labor of training AI and the scholarly efforts to study it - we show why responsible AI should go beyond the deployment phase - emphasizing fairness audits, and model explainability. It requires reimaging the values, labor regimes, and social science practices that shape AI systems from annotation to analysis.


Bios:

Dr. Jieshu Wang is an interdisciplinary researcher studying the human and social dimensions of artificial intelligence (AI) and how people can thrive in an AI-integrated future. She combines computational methods with qualitative insights to trace technology trends and understand their broader societal impact. She earned her Ph.D. in Human and Social Dimensions of Science and Technology from Arizona State University, after earlier degrees in Civil Engineering, Economics, and Science and Technology Studies. She has also worked as a patent examiner, an editor at a popular science magazine, and co-founded Synced (机器之心), an AI-focused media company in China. Her research looks both backward and forward. Backward-looking, she examines how AI are created, who creates them, and who is missing from the process. Forward-looking, she studies how AI is transforming the way we live, connect, invent, work, and adapt, as well as how AI might help address challenges such as climate change and workforce transitions.
Dr. Ben Zhang is an Assistant Professor in the Department of Technology. His research explores the production and sociotechnical impacts of AI systems in critical areas such as work, health, and sustainability. Drawing from his background in Human-Computer Interaction (HCI), Human-Centered AI, and Science and Technology Studies (STS), he employs a life-cycle-centered approach to holistically examine the promises and harms of these systems and to inform the design of responsible AI infrastructures across their development, deployment, and governance. Ben received his Ph.D. in Information Science from the University of Michigan. Ben's work has been supported by competitive awards and fellowships, including the University of Michigan Rackham Predoctoral Fellowship and the Weizenbaum Fellowship. His research has appeared in premier computing venues, including ACM CHI, ACM CSCW, and AAAI ICWSM.

Location: NCS 120