Abstract: Drawing on group-theoretic and information-theoretic foundations, we propose information lattice learning (ILL) as a general framework to learn rules of a signal (e.g., an image or a probability distribution). In our definition, a rule is a coarsened signal used to help us gain one interpretable insight about the original signal. To make full sense of what might govern the signal's intrinsic structure, we seek multiple disentangled rules arranged in a hierarchy, called a lattice. Compared to representation/rule-learning models optimized for a specific task (e.g., classification), ILL focuses on explainability: it is designed to mimic human experiential learning and discover rules akin to those humans can distill and comprehend. We will detail the mathematical foundations and algorithms of ILL, and illustrate how it addresses the fundamental question what makes X an X by creating rule-based explanations designed to help humans understand. Our focus is on explaining X rather than (re)generating it. We show ILL's efficacy and interpretability on benchmarks and assessments, as well as a demonstration of ILL-enhanced classifiers achieving human-level digit recognition using only one or a few MNIST training examples (1-10 per class). We present applications in knowledge discovery, using ILL to distill music theory from scores and chemical laws from molecules and further revealing connections between them. We close with some early work on understanding the principles that govern scattering amplitudes in Super Yang-Mills theory, rather than just predicting them.

Biography: Lav R. Varshney is the Della Pietra Infinity Professor and inaugural director of the AI Innovation Institute at Stony Brook University. He is co-founder and CEO of Kocree, Inc., a startup company building novel human-controllable AI for discovery and creativity, and chief scientist of Ensaras, Inc., a startup company focused on AI and wastewater treatment. He holds appointments at RAND Corporation and at Brookhaven National Laboratory. He was previously on the faculty of the University of Illinois Urbana-Champaign, a visiting scholar at Northwestern's Kellogg School of Management, a principal research scientist at Salesforce Research AI, and a research staff member at IBM Research. He is a former White House staffer, having served on the National Security Council staff as a White House Fellow, where he contributed to national AI and wireless communications policy. His research interests include information theory and artificial intelligence. He received his B.S. degree from Cornell University and his S.M. and Ph.D. degrees from the Massachusetts Institute of Technology.

Location: Room 102


Abstract:
Large language models (LLMs) have transformed the way humans write code, bringing unprecedented automation to software development. In this talk, I will first provide an overview of my research on enhancing LLMs' code intelligence, optimizing each step of the development pipeline towards more complex software engineering tasks. I will then delve into my key contributions, focusing on how to equip LLMs with a deeper, more comprehensive understanding of software programs. Finally, I will discuss the future of AI-driven software engineering, envisioning a new era of automation that is more reliable, intelligent, and cost-efficient.

Bio:
Yangruibo (Robin) Ding is a Ph.D. candidate in the Department of Computer Science at Columbia University. His research is at the intersection of Software Engineering and Machine Learning, focusing on developing large language models (LLMs) for code. He trains LLMs to generate, analyze, and refine software programs and constructs benchmarks to systematically evaluate LLMs in solving software engineering tasks. He also studies how to improve LLMs' reasoning capability to tackle complex programming tasks, such as debugging and patching. His interdisciplinary research has been published in top-tier conferences of software engineering, programming languages, natural language processing, and machine learning. He won an ACM SIGSOFT Distinguished Paper Award, an IEEE TSE Best Paper Runner-up, and received an IBM Ph.D. Fellowship.
Location:
NCS 120
We live in a new scientific paradigm: the Big Data era, in which a lot of data is available for almost anything. In this new paradigm, the driving force is to use data directly to learn about chemical and physics systems employing artificial intelligence. This paradigm has proven helpful in simulating realistic physical, biological, and chemical models, yielding impressive results. Similarly, the insight gained in these situations can be used to improve our understanding of fundamental processes. In that regard, we want to answer the question: Can a machine learn chemistry? The answer to this question is still debatable, but we will show our ideas and methods to find the answer. We will also discuss our results on predicting atom-diatom reactions and other avenues and work in progress in our group.

Please register for the STEM Speaker Series: Can a Machine learn Chemistry here.

Abstract: Pretraining vision encoders with self-supervision (SSL) leads to stronger representations that excel across diverse downstream tasks. One of the key factors enabling self-supervision is extracting multiple views of the same scene to formulate either: 1) View-invariant pretraining (DINO, SimCLR, iBOT), where the objective is predicting the same representation for different views of the scene; or 2) Cross-view pretraining (cross-view Masked Autoencoders), where the objective is predicting missing parts of one view using other views. For extracting multiple views, view-invariant methods rely on a combination of handcrafted augmentations (random cropping, color jittering, gaussian blur, etc.) of the same image, whereas cross-view pretraining methods rely on image cropping or video frames. In this work, we present methods to effectively incorporate synthetic views from diffusion models into SSL training.
For view-invariant pretraining, we introduce Gen-SIS, a method that leverages the ability of diffusion models to generate interpolated images through interpolation in conditioning space. We introduce a disentanglement pretext task: disentangling two source images from an interpolated synthetic image. This disentanglement task, in addition to vanilla single-source generative augmentation for view extraction, improves visual pretraining of various view-invariant methods (DINO, SimCLR, iBOT).
For cross-view pretraining, we introduce CDG-MAE, a novel cross-view masked autoencoder (MAE) based method that uses diverse synthetic views generated from static images via an image-conditioned diffusion model to learn dense correspondences. We present a quantitative method to evaluate the local and global consistency of the generated views to choose the right diffusion model for cross-view pretraining. These generated views exhibit substantial changes in pose and perspective, providing a rich training signal that overcomes the limitations of video (expensive) and crop-based (less variation) methods. CDG-MAE substantially narrows the gap to video-based MAE methods on video label propagation tasks while maintaining the data advantages of image-only MAEs.

Speaker: Varun Belagali

Location: NCS 120
Zoom: https://stonybrook.zoom.us/j/93647452432?pwd=hZaX7LXCAD8KPHWYE1Afw2sDI3owpv.1
Abstract: The rapid growth of observational data presents unprecedented opportunities to enhance both the predictability and mechanistic understanding of Earth systems. However, fully harnessing big Earth data needs computational frameworks that bridge the gap between physics-based models and machine learning. In this talk, I will first demonstrate how AI methods can significantly improve the prediction of environmental systems. Despite their predictive accuracy, machine learning models often lack physical interpretability, limiting their ability for scientific inquiry. To address this, I will introduce the developed hybrid, differentiable modeling framework that unifies physical models with machine learning in an end-to-end trainable system. This framework autonomously learns from large observations while maintaining physical clarity. The machine learning components can be seamlessly embedded into physical backbones to assimilate multi-source data, support automatic parameterization, and represent uncertain processes. I will showcase applications of this framework in simulating and understanding the terrestrial water cycle and its interactions with ecosystems at continental and global scales. This talk will highlight how differentiable modeling not only improves the modeling ability in both data-rich and data-scarce scenarios, but also provides a systematic pathway to enhancing model structures, deciphering uncertain physical relations, and facilitating knowledge discovery in Earth system sciences.


IACS Seminar Speaker: Dapeng Feng, Stanford Univeristy

Location: IACS Seminar Room