Abstract: Language is not just something we generate, but something humans use to interact with the world around them. Indeed, today's conversational AI agents speak fluently, but often treat language as prediction rather than interaction, producing responses that sound correct while failing to recognize when requests are ungrounded, impossible, or misunderstood. My research asks what it would take for multimodal agents to take the next step and use language effectively to take actions in grounded contexts and in this talk, I argue that many challenges in multimodal LLM design, including alignment, hallucination, and adaptation, are due to a lack of pragmatics: an understanding of the implicit context behind the implied actions of the words in the query. From visual understanding, to automatic speech recognition, to hallucination detection, I will demonstrate that incorporating pragmatic/contextual reasoning substantially improves agent behavior, and that pragmatic reasoning will drive a necessary shift in how we build multimodal conversational agents that can see, listen, act, and speak in context.


Bio: David M. Chan, Ph.D., is a postdoctoral scholar at the University of California, Berkeley, specializing in multimodal conversational AI. His research focuses on developing scalable AI systems that move beyond language prediction toward grounded language use, integrating vision, audio, and language to enable pragmatic interaction, improve AI-human collaboration, and reduce hallucinations in generative models. Beyond academia, he has worked with leading organizations including Amazon, Google, and NASA, to build and deploy safe, efficient, and accessible machine learning systems. He is also the developer and maintainer of TSNE-CUDA, an open-source tool for high-dimensional data visualization, used by over 40,000 researchers in fields ranging from biomedical technology to industrial manufacturing. David holds a Ph.D. and M.Sc. in Computer Science from UC Berkeley, where he was a graduate fellow with the Center for Technology, Society & Policy (CTSP), and dual B.Sc. degrees in Computer Science and Mathematics from the University of Denver.

Location: NCS 120
The overall purpose of this seminar is to bring together people with interests in Computer Vision theory and techniques and to examine current research issues. This course will be appropriate for people who already took a Computer Vision graduate course or already had research experience in Computer Vision. To enroll in this course, you must either: (1) be in the PhD program or (2) receive permission from the instructors.

Each seminar will consist of multiple short talks (around 10 minutes) by multiple people. Students can register for 1 credit for CSE 656. Registered students must attend and present a minimum of 2 or 3 talks. Everyone else is welcome to attend. Fill in https://forms.gle/pCVXovgfMfQwGqG38 to subscribe to our mailing list for further announcement.
Making sense of Twitter @ Bloomberg presented by Daniel Preotiuc-Pietro

ABSTRACT: The Bloomberg Terminal has provided ways for investors and journalists to sift through and understand the immense volume of tweets and discover financially-relevant content ever since the SEC approved the use of Twitter for company disclosures back in 2013.

In the first part of the talk, I will showcase how tweets impact financial markets and how Bloomberg is using Natural Language Processing methods to identify financially relevant tweets that move the markets. Our processing pipeline feeds directly to clients, journalists in the newsroom and powers several news analytic products offered by the company including trending companies and consumer sentiment for publicly traded equities.

However, understanding user pragmatic intent in individual tweets would allow us to gain deeper insights and enable new applications. I will present several recent research studies focused on understanding intent including identifying complaints and the roles with which vulgarity is used in social media and how these can help improve applications such as sentiment analysis and hate speech detection.

BIO: Daniel Preotiuc-Pietro is a Senior Research Engineer and Team Lead at Bloomberg LP, where he works on analyzing and building models for real-world large scale social media and news mining and information extraction. His research interests are focused on understanding the social and temporal aspects of text, especially from social media, with applications in domains such as Social Psychology, Law, Political Science and Journalism. Several of his research studies were featured in popular press including the Washington Post, BBC, New Scientist, Scientific American or FiveThirtyEight. He is a co-organizer of the Natural Legal Language Processing workshop series. Prior to joining Bloomberg LP, Daniel was a postdoctoral researcher at the University of Pennsylvania with the interdisciplinary World Well Being Project and obtained his PhD in Natural Language Processing and Machine Learning at the University of Sheffield, UK.

Join University Libraries for an engaging panel discussion where we delve in and learn about the impacts of artificial intelligence on the 2024 US elections! Panelists are Paige Lord, Tom Costello, and Musa al-Gharbi. The discussion will be moderated by Library Dean, Karim Boughida. Co-sponsored by the Office of Diversity, Inclusion, and Intercultural Initiatives.

Please RSVP for Democracy in the Digital Age: AI's Influence on 2024 Elections here.
The overall purpose of this seminar is to bring together people with interests in Computer Vision theory and techniques and to examine current research issues. This course will be appropriate for people who already took a Computer Vision graduate course or already had research experience in Computer Vision. To enroll in this course, you must either: (1) be in the PhD program or (2) receive permission from the instructors.

Each seminar will consist of multiple short talks (around 10 minutes) by multiple people. Students can register for 1 credit for CSE 656. Registered students must attend and present a minimum of 2 or 3 talks. Everyone else is welcome to attend. Fill in https://forms.gle/pCVXovgfMfQwGqG38 to subscribe to our mailing list for further announcement.
Abstract: In recent years, we have been developing generative AI methods to design increasingly complex objects. Our goal is to improve performance while ensuring that these objects remain controllable. This requires addressing several challenging problems, including:
  • Modeling composite objects in a way that preserves consistency under deformation.
  • Estimating the uncertainty of the surrogate models used to predict performance during optimization.
  • Co-designing objects for both high performance and ease of control.
In this talk, I will present our approach to these challenges and describe end-to-end design pipelines that have the potential to radically transform computer-assisted engineering.

Speaker: Pascal Fua received an engineering degree from Ecole Polytechnique, Paris, in 1984 and a Ph.D. in Computer Science from the University of Orsay in 1989. He joined EPFL (Swiss Federal Institute of Technology) in 1996, where he is a Professor in the School of Computer and Communication Science and head of the Computer Vision Lab. Before that, he worked at SRI International and at INRIA Sophia-Antipolis as a Computer Scientist.
His research interests include shape modeling and motion recovery from images, analysis of microscopy images, and machine learning. He has (co)authored over 400 publications in refereed journals and conferences. He has received several ERC grants. He is an IEEE Fellow and has been an Associate Editor of the IEEE journal Transactions For Pattern Analysis and Machine Intelligence. He often serves as the program committee member, area chair, and program chair of major vision conferences and has cofounded three spinoff companies.
Abstract: As we enter the AI era, domain scientists face a critical question: What can we do to harness AI effectively for scientific discovery? AI has demonstrated remarkable capabilities, from accelerating simulations to uncovering hidden patterns in complex datasets. While these advancements offer unprecedented opportunities, they also raise concerns--AI models often function as black boxes, making it difficult to connect their outputs to established scientific principles. This lack of interpretability can undermine trust and limit adoption, particularly in fields like meteorology where physical understanding is critical.
In this talk, I will explore how interpretable AI can bridge this gap, highlighting its potential to generate explicit, physically meaningful equations rather than opaque neural networks. Through four case studies from my lab, I will showcase how interpretable AI can enhance scientific understanding:
  1. Satellite Precipitation Retrieval: Using AI-based approaches to interpret precipitation retrieval algorithms from AMSU data, we identified critical microwave channels (89 and 150 GHz) that directly link to physical processes in the atmosphere.
  2. Quantitative Precipitation Estimation (QPE): By applying symbolic regression models to polarimetric radar data, we derived mathematical expressions that outperform traditional Z-R relationships and existing QPE algorithms, offering new insights into rainfall microphysics.
  3. Tornado Probability Prediction: Leveraging reinforcement learning-based symbolic deep learning models, we developed interpretable equations that outperform the traditional Significant Tornado Parameter (STP) index, providing a clearer understanding of the relationships between key atmospheric variables and tornado risk.
  4. Domain-Aware Symbolic Regression for Scientific Equations: In our latest work, we introduced a symbolic regression framework that incorporates domain-specific symbol priors extracted from thousands of scientific publications. By encoding common mathematical structures--such as the prevalence of trigonometric functions in physics or logarithmic forms in biology--into a tree-structured reinforcement learning model, we improved both the accuracy and interpretability of discovered equations. This approach accelerates convergence, enforces physical plausibility, and reveals new governing relationships in climate and geophysical data.
Through these examples, I hope to spark discussion on the evolving role of domain scientists in the AI era and inspire new ways to integrate AI with physical understanding in atmospheric research.

IACS Seminar Speaker: Yixin Wen, University of Florida

Location: IACS Seminar Room or Zoom

Join Zoom Meeting: https://stonybrook.zoom.us/j/97596399106?pwd=0PBvElFLqov3biO6OlQxSWLWudkIuH.1
Meeting ID: 975 9639 9106
Passcode: 096213