Speaker TBD
Refreshments will be provided
AI + Music Seminar - Details to come.
Abstract: In high-dimensional data spaces, vast empty regions often exist where no known data points are present. These empty spaces are not merely gaps but hold untapped potential for discovering novel configurations, optimizing parameters, and improving decision-making processes. However, traditional exploration techniques struggle to identify and leverage these regions due to the curse of dimensionality. To address this, we introduce the Empty Space Search Algorithm (ESA), a scalable, physics-inspired method that systematically identifies and explores these uncharted voids. ESA operates by modeling the data space as a dynamic system, using a repulsion-attraction mechanism to locate optimal empty space configurations (ESCs) without requiring exhaustive search. Building upon ESA, we present GapMiner, a visual analytics system that integrates human-in-the-loop AI to iteratively refine and validate ESCs. GapMiner combines parallel coordinate visualization, interactive optimization, and deep learning-based predictive modeling to enhance the efficiency of empty space exploration. This methodology has broad applications, including accelerating convergence in evolutionary algorithms through a more diverse initial population, optimizing adversarial learning strategies, and discovering novel parameter configurations in reinforcement learning. Our approach demonstrates that empty space is not just an absence of data but a frontier for new possibilities in high-dimensional problem-solving.
Bio: Xinyu Zhang received his B.E. in Computer Science from Shandong University, Taishan College, in 2019. He is currently a final-year Ph.D. candidate in the Department of Computer Science at Stony Brook University, advised by Prof. Klaus Mueller. His research focuses on multivariate data analysis, scientific visualization, and reinforcement learning. He has published multiple papers in top-tier journals and conferences, including IEEE TVCG and NeurIPS.
*this seminar will be held in person (food provided on a first come, first serve basis), and online (zoom link below)!
Topic: IACS Student Seminar Speaker: Xinyu Zhang
Time: Feb 26, 2025 12:00 PM Eastern Time (US and Canada)
Join Zoom Meeting
https://stonybrook.zoom.us/j/
Meeting ID: 918 4821 8975
Passcode: 027337
This event brings together people with interests in Computer Vision theory and techniques and examines current research issues in the field.
Each seminar consists of multiple short talks (around 15 minutes) by several students.
Join Zoom Meeting:
https://stonybrook.zoom.us/j/93547152068?pwd=WVpoRVgzelBXeloxdXVEakNSb2M5UT09
Meeting ID: 935 4715 2068 | Passcode: 481832
Each seminar consists of multiple short talks (around 15 minutes) by several students.
Join Zoom Meeting:
https://stonybrook.zoom.us/j/93547152068?pwd=WVpoRVgzelBXeloxdXVEakNSb2M5UT09
Meeting ID: 935 4715 2068 | Passcode: 481832
Predicting Subjective Attributes in Visual Data - Zijun Wei
ABSTRACT: Recent progress in deep neural networks has revolutionized many computer vision tasks such as image classification, detection and segmentation. However, in addition to excelling in tasks that predict well-defined objective information, human-centered artificial intelligence systems should also be able to model subjective attributes, as defined by human perceptual behavior, that goes beyond the pure physical content of visual data. Example subjective tasks are the prediction of spatial or temporal regions that are interesting to humans (e.g., attract attention or are visually pleasing) and the recognition of subjective attributes (e.g., visually elicited sentiments). Better models for these tasks will improve the human-computer interaction experience in various applications. This thesis investigates several approaches to address the challenges in predicting those subjective attributes in visual data over a diverse set of tasks. I first present a novel framework for real-time automatic photo composition. The framework consists of a cost-effective data collection workflow, an efficient model training pipeline and a lightweight module to account for personalized preferences. Then I develop a novel and general algorithm to detect interesting segments in sequential data, which can be naturally applied to video summarization tasks. Furthermore, I propose methods that learn to represent sentiments elicited by images, in an unsupervised manner, using linguistic features extracted from large scale Web data. To conclude this thesis, I introduce a human-vision-inspired image classification algorithm that also predicts spatial visual attention even though no attention data was used for training it.
ABSTRACT: Recent progress in deep neural networks has revolutionized many computer vision tasks such as image classification, detection and segmentation. However, in addition to excelling in tasks that predict well-defined objective information, human-centered artificial intelligence systems should also be able to model subjective attributes, as defined by human perceptual behavior, that goes beyond the pure physical content of visual data. Example subjective tasks are the prediction of spatial or temporal regions that are interesting to humans (e.g., attract attention or are visually pleasing) and the recognition of subjective attributes (e.g., visually elicited sentiments). Better models for these tasks will improve the human-computer interaction experience in various applications. This thesis investigates several approaches to address the challenges in predicting those subjective attributes in visual data over a diverse set of tasks. I first present a novel framework for real-time automatic photo composition. The framework consists of a cost-effective data collection workflow, an efficient model training pipeline and a lightweight module to account for personalized preferences. Then I develop a novel and general algorithm to detect interesting segments in sequential data, which can be naturally applied to video summarization tasks. Furthermore, I propose methods that learn to represent sentiments elicited by images, in an unsupervised manner, using linguistic features extracted from large scale Web data. To conclude this thesis, I introduce a human-vision-inspired image classification algorithm that also predicts spatial visual attention even though no attention data was used for training it.
Hidden Biases. Ethical Issues in NLP, and What to Do about Them presented by Dirk Hovy of Bocconi University
ABSTRACT: Through language, we fundamentally express who we are as humans. This property makes text a fantastic resource for research into the complexity of the human mind, from social sciences to humanities. However, it is exactly that property that also creates some ethical problems. Texts reflect the authors' biases, which get magnified by statistical models. This has unintended consequences for our analysis: If our data is not reflective of the population as a whole, if we do not pay attention to the biases contained, we can easily draw the wrong conclusions, and create disadvantages for our users.
In this talk, I will discuss several types of biases that affect NLP models, their sources, and potential counter measures: (1) Bias stemming from data, i.e., selection bias (if our texts do not adequately reflect the population we want to study), label bias (if the labels we use are skewed) and semantic bias (the latent stereotypes encoded in embeddings); (2) Biases deriving from the models themselves, i.e., their tendency to amplify any imbalances that are present in the data; (3) Design bias, i.e., the biases arising from our (the researchers) decisions which topics to analyze, which data sets to use, and what to do with them. For each bias, I will provide examples and discuss the possible ramifications for a wide range of applications, and various ways to address and counteract these biases, ranging from simple labeling considerations to new types of models.
BIO: Dirk Hovey is an associate professor of Computer Science in the department of marketing at Bocconi University. He received his PhD from the University of Southern California in Los Angeles, where he worked as a research assistant at the Information Sciences Institute.
He works in Natural Language Processing (NLP), a subfield of artificial intelligence. His research focuses on computational social science. His interests include integrating sociolinguistic knowledge into NLP models, using large-scale statistics to model the interaction between people's socio-demographic profile and their language use, and ethics for data science and algorithmic fairness.
ABSTRACT: Through language, we fundamentally express who we are as humans. This property makes text a fantastic resource for research into the complexity of the human mind, from social sciences to humanities. However, it is exactly that property that also creates some ethical problems. Texts reflect the authors' biases, which get magnified by statistical models. This has unintended consequences for our analysis: If our data is not reflective of the population as a whole, if we do not pay attention to the biases contained, we can easily draw the wrong conclusions, and create disadvantages for our users.
In this talk, I will discuss several types of biases that affect NLP models, their sources, and potential counter measures: (1) Bias stemming from data, i.e., selection bias (if our texts do not adequately reflect the population we want to study), label bias (if the labels we use are skewed) and semantic bias (the latent stereotypes encoded in embeddings); (2) Biases deriving from the models themselves, i.e., their tendency to amplify any imbalances that are present in the data; (3) Design bias, i.e., the biases arising from our (the researchers) decisions which topics to analyze, which data sets to use, and what to do with them. For each bias, I will provide examples and discuss the possible ramifications for a wide range of applications, and various ways to address and counteract these biases, ranging from simple labeling considerations to new types of models.
BIO: Dirk Hovey is an associate professor of Computer Science in the department of marketing at Bocconi University. He received his PhD from the University of Southern California in Los Angeles, where he worked as a research assistant at the Information Sciences Institute.
He works in Natural Language Processing (NLP), a subfield of artificial intelligence. His research focuses on computational social science. His interests include integrating sociolinguistic knowledge into NLP models, using large-scale statistics to model the interaction between people's socio-demographic profile and their language use, and ethics for data science and algorithmic fairness.
The University's Main Commencement Ceremony will take place on Friday, May 23, 2025 at 11 am at Kenneth P. LaValle Stadium. Gates open at 10 am.
All guests need a valid ticket to enter LaValle Stadium - no exceptions. Children age 1 and older require a ticket. Seating is first-come, first-served.
Register here.
Title: Building foundation models for scientific data Seminar
Speaker: Ruben Ohana, Ph.D. and Michael McCabe, Ph.D - Flatiron Institute, New York
Abstract: Foundation models are very large architectures trained on large-scale datasets and can be used to transfer knowledge from a domain to another. Scientific data, particularly numerical simulations of partial differential equations (PDEs), presents unique challenges due to its complexity and the need for domain expertise to assess prediction quality, complicating the building of the first foundation models in this field. In this talk, we will develop our approach of building foundation models for scientific data, highlighting the requirements and expectations for achieving meaningful results. We will also introduce The Well, a comprehensive collection of datasets encompassing multi-scale simulations of fluid dynamics, astrophysics, and biological systems. The Well serves as a foundation for developing models that generalize across diverse physical phenomena, aiming to accelerate scientific discovery through large-scale learning.
Join Zoom Meeting: https://bnl.zoomgov.com/j/1606898802?pwd=GbbPiLGHlEokDskxjeFheMFWfuboxO.1
Meeting ID: 160 689 8802
Passcode: 281575
Speaker: Ruben Ohana, Ph.D. and Michael McCabe, Ph.D - Flatiron Institute, New York
Abstract: Foundation models are very large architectures trained on large-scale datasets and can be used to transfer knowledge from a domain to another. Scientific data, particularly numerical simulations of partial differential equations (PDEs), presents unique challenges due to its complexity and the need for domain expertise to assess prediction quality, complicating the building of the first foundation models in this field. In this talk, we will develop our approach of building foundation models for scientific data, highlighting the requirements and expectations for achieving meaningful results. We will also introduce The Well, a comprehensive collection of datasets encompassing multi-scale simulations of fluid dynamics, astrophysics, and biological systems. The Well serves as a foundation for developing models that generalize across diverse physical phenomena, aiming to accelerate scientific discovery through large-scale learning.
Join Zoom Meeting: https://bnl.zoomgov.com/j/1606898802?pwd=GbbPiLGHlEokDskxjeFheMFWfuboxO.1
Meeting ID: 160 689 8802
Passcode: 281575
Abstract: Foundation models are rapidly becoming capable assistants for knowledge work, but their deployment in real settings is limited by three gaps: they do not transfer reliably across environments, their internal reasoning is opaque, and their behavior is hard to control precisely. In this talk, I argue that these limitations are not only about model size -- they are fundamentally about whether learning captures and leverages the underlying structure of the data-generating process. I use causal thinking as a practical lens to model what is invariant, what changes, and what can be intervened on, and I further show how this leads to learning principles that improve trustworthiness.
I will first present methods for learning unifying mechanisms from heterogeneous data, across domains and modalities, to enable reliable transfer and controllable generation. Next, I will show how structured concepts can be recovered even from seemingly unstructured data, by analyzing and improving self-supervised objectives (such as masking and diffusion) through hierarchical latent-variable models. These concept structures can then be used to interpret generative models and support targeted, multi-level edits. Finally, I connect these two threads to generalization beyond the training distribution. I will discuss natural conditions for extrapolation and a compositional generation framework that improves prompt following for novel concept combinations. I will conclude with a brief outlook on self-improving world models and AI-assisted scientific discovery.
Bio: Lingjing Kong is a Ph.D. candidate in the Computer Science Department at Carnegie Mellon University. His research focuses on Causal AI for transferable, interpretable, and controllable systems, with an emphasis on understanding and exploiting the structure of real-world data to make foundation models actionable and more reliable. He develops identification principles and scalable algorithms for learning unified models from heterogeneous data, uncovering hierarchical concept structures in unstructured data (e.g., images and text), and generalizing beyond training support through compositionality and extrapolation. His work has appeared in top ML venues including ICML, NeurIPS, CVPR, ICLR, and EMNLP and has been prototyped and applied in industry.
Location: NCS 120
I will first present methods for learning unifying mechanisms from heterogeneous data, across domains and modalities, to enable reliable transfer and controllable generation. Next, I will show how structured concepts can be recovered even from seemingly unstructured data, by analyzing and improving self-supervised objectives (such as masking and diffusion) through hierarchical latent-variable models. These concept structures can then be used to interpret generative models and support targeted, multi-level edits. Finally, I connect these two threads to generalization beyond the training distribution. I will discuss natural conditions for extrapolation and a compositional generation framework that improves prompt following for novel concept combinations. I will conclude with a brief outlook on self-improving world models and AI-assisted scientific discovery.
Bio: Lingjing Kong is a Ph.D. candidate in the Computer Science Department at Carnegie Mellon University. His research focuses on Causal AI for transferable, interpretable, and controllable systems, with an emphasis on understanding and exploiting the structure of real-world data to make foundation models actionable and more reliable. He develops identification principles and scalable algorithms for learning unified models from heterogeneous data, uncovering hierarchical concept structures in unstructured data (e.g., images and text), and generalizing beyond training support through compositionality and extrapolation. His work has appeared in top ML venues including ICML, NeurIPS, CVPR, ICLR, and EMNLP and has been prototyped and applied in industry.
Location: NCS 120