Abstract: Robot control has evolved from optimization-based controllers---precise but task-specific---through deep reinforcement learning's learned policies, to Vision-Language-Action (VLA) models that leverage pretrained vision-language backbones for language-conditioned manipulation across diverse tasks.
Despite their promise, VLAs exhibit a critical limitation: they function primarily as trajectory learners rather than skill learners. Recent evaluations reveal that VLAs often fail when faced with even minor variations in object initialization or environmental conditions, suggesting they memorize specific trajectories rather than acquiring generalizable manipulation skills. Attempts to address this through 3D spatial representations have shown limited success, indicating that the missing component may be more fundamental than geometric understanding alone.
This work argues that World Models (WMs)---internal representations that predict future states given actions---constitute the missing piece for robust VLA systems. We present one completed contribution and two ongoing investigations.
We developed a dual-layer world model for human-robot interaction that anticipates both physical scene evolution and latent human preferences for assistive tasks. Building on these foundations, we present ongoing work probing VLA internal representations to verify implicit world model existence, and propose a WM-VLA integration approach operating in the native visual domain through embedding prediction and image decoding.
Together, these contributions and investigations establish a foundation for WM-VLA systems, pointing toward robust, generalizable robot policies.
Speaker: Jason Qin
Location: NCS 220
CG Group member (and SBU faculty) Chao Chen will speak on Fri, March 12, about the use of topological data analysis in machine learning for image analysis.
Chao has shared some of his research with the CG Group previously, and this will be a great opportunity to learn more about this exciting research area related to computational geometry/topology!

Time: Friday, March 12, 2pm-3pm
Place: Zoom
https://stonybrook.zoom.us/my/profweizhu?pwd=RjVIVXg3YUhudzZZQ3pheHUydTJBUT09



Title: Learning with Topological Information - Image Analysis and Label Noise
Speaker: Prof. Chao Chen (SBU)

Abstract: Modern machine learning faces new challenges. We are
analyzing highly complex data with unknown noise. Topology provides
novel structural information to model such data and noise. In this
talk, we discuss two directions in which we are using topological
information in the learning context. In image analysis, we propose a
topological loss to segment and to generate images with not only
per-pixel accuracy, but also topological accuracy. This is necessary
in analysis of images of fine-scale biomedical structures such as
neurons, vessels, etc.  Extracting these structures with correct
topology is essential for the success of downstream
analysis. Meanwhile, we discuss how to use topological information to
train classifiers robust to label noise. This is important in practice
especially when we are using deep neural networks which tend to
overfit noise. These results have been published in NeurIPS, ECCV,
ICML and ICLR.























A new virtual seminar series on Games, Decisions, and Networks will start this Friday. The series aims at bringing together researchers working on foundations and applications of games theory, decision theory, and networks from computer science, control, economics and operation research. 





The advisory board for the series comprises Asu Ozdaglar (MIT), Christos Papadimitriou (Columbia), Drew Fudenberg (MIT), Eva Tardos (Cornell), Matthew O. Jackson (Stanford), Ramesh Johari (Stanford), and Tamer Başar (UIUC). 
The first talk will be given by Costantinos Daskalakis (MIT) on January 22nd at noon ET, titled Equilibrium Computation and the Foundations of Deep Learning. Upcoming speakers include


- Rakesh Vohra (Upenn)
- Sanjeev Goyal (Cambridge)
- Aaron Roth (Upenn)
- Aislinn Bohren (Upenn)
- Jason Marden (UCSB)

and more to be added!
Abstract: As computing and society become increasingly inseparable, we confront a fundamental design challenge: creating AI systems where human-machine interactions authentically embody our diverse values while thoughtfully evolving our social relationships. The recursive nature of these interactions--where human behavior shapes technology design and technological affordances influence human behavior--presents both profound risks and transformative opportunities as we reimagine our collective digital future. What interaction patterns emerge when algorithmic systems become active participants in societal decision-making? How can we design human-AI collaboration that ensures algorithmic systems align with diverse community values while serving the public interest? Through Public Interest AI, we explore a Pluralistic Design Language that creates interaction models for value-sensitive algorithmic ecosystems, strengthening AI-society alignment in both technology design and policy development. Through collaborative interaction with communities, we create systems that augment human capabilities while embedding ethical principles into the sociotechnical design of AI itself--ultimately redefining possibilities at the intersection of technology, policy, and society. This talk will examine the challenges of designing meaningful human-AI systems within social contexts through real-world applications that combine value-sensitive interaction design, human-inspired computing, and societal development to create technologies that advance our shared commitment to the public good.

Bio: Neil Gaikwad is an Assistant Professor of Data Science and Computer Science at UNC Chapel Hill. Additionally, he serves on the Faculty Advisory Council of the UNC Parr Center for Ethics and is a Fellow at the MIT Dalai Lama Center for Ethics and Transformative Values. Neil holds a Ph.D. in Society-Centered AI from MIT and is an alumnus of Carnegie Mellon University's School of Computer Science. Neil's scholarship, published in prominent AI and HCI conferences, has been recognized with several prestigious honors, including the Facebook Research Fellowship, UIST Best Paper Honorable Mention, MIT Engineering Fellowship, Human Rights & Technology Fellowship, Graduate Teaching Award, and the Karl Taylor Compton Prize, MIT's highest student honor. He has been recognized as a Rising Star by both Stanford University and the University of Chicago. Translating research into real-world impact, Neil is a dedicated educator and mentor who has taught over 500 students throughout his career. He has guided more than 30 students to publish influential papers on AI fairness, secure prestigious fellowships, and contribute to shaping AI policy through public interest research. Neil is also the founder of the AI Policy Global Initiative, which has successfully brought together academia, industry, government, and communities to address critical challenges in AI governance and develop collaborative approaches to responsible AI.

Location: Old Computer Science, room 1310
Abstract: Making large language models usable requires post-training methods that align models to human values, robustly handle underspecified inputs, and generalize to diverse instructions. This talk addresses the challenge of developing responsible AI through post-training from the following angles. First, I address contextual robustness. Preference data, for example, is often underspecified, and I show how underspecification in preference data can lead to diverging preferences. Standard reward models fail to properly handle these disagreements, often making decisive choices even when human annotators are split. I argue that more consequential outputs demand more context, and propose clarification question generation as one solution. Second, I will discuss how we can train models to be better instruction followers. I will show that most models severely overfit on a small set of instruction-following constraints and are not able to generalize well to unseen output constraints. I propose to train models with reinforcement learning from verifiable rewards for verifiable instruction following, and show how this leads to improved generalization on constraint following. Throughout the presentation, I will outline how I have applied these insights into developing open generative models, like Tülu and OLMo, and I will conclude with my research agenda for responsible post-training: critical AI evaluation, broader generalization, and expanding what models can reliably do.


Bio: Valentina Pyatkin is a postdoctoral researcher at the Allen Institute for AI and the University of Washington, advised by Prof. Hanna Hajishirzi and Prof. Yejin Choi. Additionally, she is part- time affiliated with the ETH AI Center, where she mentors students and works on post-training for the Swiss AI Initiative. She obtained her PhD in Computer Science from Bar Ilan University. Her work has been awarded an ACL Outstanding Paper Award and the ACL Best Theme Paper Award, and has been supported by a Schmidt Sciences Postdoctoral Award. During her doctoral studies, she conducted research internships at Google and the Allen Institute for AI, where she received the AI2 Outstanding Intern of the Year Award.

Location: NCS 120
CSE 600 Seminar Series | Fall 2025


Abstract: Large reasoning models have demonstrated capabilities to solve competition-level math problems, answer deep research questions, and address complex coding needs. Much of this progress has been enabled by scaling of data: pre-training data to learn vast knowledge, fine-tuning data to learn natural language reasoning, and RL environments to refine that reasoning. In this talk, I will describe the current LLM reasoning paradigm, its boundaries, and the future of LLM reasoning beyond scaling. First, I will describe the state of reasoning models and where I think scaling can lead to some additional (though perhaps limited) successes. I will then shift to discussing more fundamental issues with models that scale will not resolve in the next few years. I will touch on four current limitations: outdated knowledge, generator-validator gaps, limited creativity, and poor compositional generalization. In all cases, fundamental limitations of LLMs or of supervised learning in general make these problems challenging, inviting future study and novel solutions beyond scaling.

Bio: Greg Durrett is an associate professor in the Department of Computer Science and the Center for Data Science at New York University. His research is broadly in the areas of natural language processing and machine learning. Currently, his group's focus is on reasoning about knowledge in text, verifying correctness of generation methods, and studying how to make progress on problems that defy LLM scaling. He is a 2023 Sloan Research Fellow and a recipient of a 2022 NSF CAREER award. He has served in numerous roles for ACL conferences, recently as a member of the NAACL Board since 2024 and as Senior Area Chair for ACL 2025 and EMNLP 2025. He received his BS in Computer Science and Mathematics from MIT and his PhD in Computer Science from UC Berkeley, where he was advised by Dan Klein.
Defending Software Systems from Cyber Attack Campaigns Presented by R. Sekar The DNC hack of 2016, the Equifax breach of 2017, and the spate of ransomware campaigns in 2019 demonstrate the formidable challenges we face in securing our network and software systems against highly stealthy and sophisticated adversaries. In this talk, I will describe two avenues of research we have been pursuing to help tilt the table against such powerful adversaries. The first is software hardening techniques that make software vulnerabilities harder to exploit. To maximize their applicability and ease of use, our techniques are implemented into compilers, or they directly transform binary code. I will outline some of the exciting new developments we have had in this area over the years, including randomization, memory safety, information-flow tracking, control-flow integrity, and code-pointer integrity. We complement this first line of defense with techniques for analyzing and understanding attack campaigns that manage to slip past all deployed defenses. Our techniques can sift through logs consisting of hundreds of millions of events to zoom in on attack activity that may span just a few hundred events. I will describe our experience in mapping out several DARPA-sponsored red team attack campaigns.

Abstract:

Conventional approaches to scientific discovery often prioritize building larger sensors, gathering more data, and scaling up computational power. In this talk, I will present a complementary perspective: extracting insights hidden in the data we already have. The key lies in using AI not as a black-box predictor, but as a tool for interpreting data through its underlying physical process.

I will demonstrate how AI, when integrated with the physics of light propagation, can serve as a computational lens to overcome fundamental limitations in fields ranging from biomedicine to astrophysics. Specifically, I will showcase two compelling applications: non-invasive imaging through scattering biological tissues, and detecting faint exoplanets against the overwhelming brightness of their host stars.

These methods represent a departure from traditional learning-based approaches that rely on fitting models to training labels and hoping for generalization. Instead, with physics-informed strategies that decode how light propagates, we can transform raw measurements into scientifically meaningful insights--without requiring costly hardware upgrades or human-annotated datasets. Finally, I will outline future directions for combining AI with physical principles, enabling us to unlock more phenomena once considered hidden and accelerating discoveries in healthcare, astronomy, and beyond.

Short Bio:

Brandon Y. Feng is a Postdoctoral Associate at MIT Computer Science and Artificial Intelligence Laboratory (CSAIL) and a Visiting Scientist at the Harvard-Smithsonian Center for Astrophysics. His research bridges artificial intelligence and physics to expand the limits of human and machine vision. He develops AI-driven methods that reveal hidden patterns in complex visual data, driving breakthroughs in areas such as exoplanet detection and imaging through scattering tissues. His work has been published in top venues, including Science Advances, CVPR, ICCV, ECCV, and NeurIPS, and has been featured in Science.org, New Scientist, and Phys.org. He holds a Ph.D. in Computer Science from the University of Maryland, along with a B.A. in Computer Science and Statistics and an M.S. in Statistics from the University of Virginia.

Location: NCS 220
Spring 2025, Mondays 3.30 to 4.50 pm, NCS 220.

The seminar will be jointly taught by Prof. Chao Chen, chao.chen.1@stonybrook.edu and Prof. Dimitris Samaras samaras@cs.stonybrook.edu

The overall purpose of this seminar is to bring together people with interests
in Computer Vision theory and techniques and to examine current research issues. This course will be appropriate for people who already took a Computer Vision graduate course or already had research experience in Computer Vision.

To enroll in this course, you must either: (1) be in the Ph.D. program or (2) receive permission from the instructors.

Each seminar will consist of multiple short talks (around 15 minutes) by multiple students. Students can register for 1 credit for CSE656. Registered students must attend and present a minimum of 2 talks. Registered students must attend in person. Up to 3 absences will be excused. Everyone else is welcome to attend.

Join here. Meeting ID: 927 2069 8658. Passcode: 130934.
.