Join us as we celebrate this year's Brook & Beyond Challenge finalists.
The Office for Research and Innovation invites you to hear about the two-month journey in which the Brook & Beyond team supported eight cohorts in bringing their bold ideas from the lab to the marketplace. It's an energizing evening that highlights the collaboration, creativity, and entrepreneurial spirit driving discovery across the University.
Meet this year's award recipients, hear pitches from the emerging founders, and applaud their achievements.
Connect, celebrate, and be part of the momentum shaping the future of innovation at
Stony Brook University.
Refreshments will be served. Registration is required.
Register Here.

Abstract:

It is known that models like large language models (LLMs) can often suggest colloquial plans given verbal descriptions of tasks, yet they are unable to reliably provide executable and verifiable plans given formally specified environments. In this talk, I will discuss a strand of efforts to have LLMs generate accurate and explainable plans in textual simulations. Instead of directly generating the plan or actions, LLMs are prompted to generate Planning Domain Definition Language (PDDL) that specifies the environment (domain file) and the task (problem file), which can then be deterministically solved with an off-the-shelf planner. In a 3-phase study, my collaborators and I first observed that it is possible but very challenging for LLMs to generate long-form code such as PDDL domain and problem files given textual specifications. Next, we devise methodologies for LLMs to iteratively generate and refine problem files while exploring a partially-observed, simulated, textual environment. Finally, we show that domain files are even more difficult to generate correctly, even on well-established planning tasks such as BlocksWorld. Finally, I will discuss ongoing efforts to improve said ability of structured generation and promising frontiers to explore.

Bio:
Li Harry Zhang is an assistant professor at Drexel University, focusing on Natural Language Processing (NLP) and artificial intelligence (AI). He obtained his PhD degree from the University of Pennsylvania advised by Prof. Chris Callison-Burch. Prior, he obtained his Bachelor's degree at the University of Michigan mentored by Prof. Rada Mihalcea and Prof. Dragomir Radev. His current research uses large language models (LLMs) to reason and plan via symbolic and structured representations. He has published more than 20 peer-reviewed papers in NLP and AI conferences, such as ACL, EMNLP, and AACL, that have been cited more than 1,000 times. He also consistently serves as Area Chair, Session Chair, and reviewer in those venues. Being a musician, producer, and content creator having over 50,000 subscribers, he is also passionate in the research of AI music and creativity.

University Libraries Present: Qualitative data can be challenging to analyze and interpret effectively. In this workshop, SBU Libraries' Data Literacies Lead, Ahmad Pratama will show you how to extract meaningful insights from textual data, including understanding sentiment trends. Learn to explore qualitative data with Python using word clouds, basic natural language processing (NLP) techniques, and lexicon-based sentiment analysis with VADER.
https://stonybrook.zoom.us/meeting/register/k0r6mPYCRayk2AOGmyd0qw#/registration
Hieu Le presents Incorporating Physical Illumination Constraints into Deep Learning Shadow Detection and Removal (PhD Proposal)

Shadows provide useful cues to analyze the scene but also hamper many computer vision algorithms such as image segmentation, object detection or tracking. For those reasons, shadow detection and shadow removal have been well studied topics in computer vision. Early approaches for shadow detection and removal focus on physical illumination models of shadows. These methods can express, identify, and remove shadows in a physically plausible manner. However, these models are often hard to optimize and slow in inference due to reliance on hand-designed image features. On the other hand, recent deep-learning approaches have achieved breakthroughs in performances for both shadow detection and removal. They learn to extract useful features automatically through training while being extremely efficient in computation. However, these models are data-dependent, opaque and ignore the physical aspects of shadows.

We propose to incorporate physical illumination constraints into deep-learning frameworks. Thus the mapping learned by the deep-network closely follows the physics of shadows, enabling the network to systematically and realistically modify shadows in images. For shadow detection, we present a novel GAN framework in which the generator can generate realistic images with attenuated shadows that can be used to train a shadow detector. For shadow removal, we propose a method that uses deep-networks to estimate the unknown parameters for a shadow image formation model that removes shadows. The system outputs shadow-free images in high-quality with no image artifacts and achieves state-of-the-art shadow removal performance. Lastly, we propose a system trained without the need for any shadow-free images in which physical constraints play pivotal roles that enable training the networks.

For Zoom information, please email events@cs.stonybrook.edu.

Abstract: Traditional questionnaires remain the primary method for assessing psychological outcomes and beliefs, capturing individuals' and populations' inner states. This dissertation presents an alternative computational method that overcomes key limitations in current mental health monitoring, particularly in spatiotemporal resolution, responses to major events, and automatic belief identification. By analyzing ∼1 billion Tweets from 2 million geo-located users, we created a big data pipeline for estimating depression and anxiety at the county-week level. These Language-Based Mental Health Assessments (LBMHA) demonstrated higher reliability and validity than traditional survey measures. Our approach effectively captured mental health trends and highlighted significant increases in mental illness following major events. Using the LBMHA pipeline, we conducted quasi-experiments, research designs that simulate randomized control trials, to generate explanations for mental health changes due to COVID-19 incidence/death. Utilizing these time-series analyses, we conducted discontinuity forecasting for community-specific anxiety shifts using statistical learning via ensemble and contextual models. To likewise investigate individual internal states, we created a novel task and annotated dataset for self belief language identification. Our fine-tuned language model for self-belief classification, despite its relatively small scale, outperformed GPT-4o. The self belief topics identified by our model successfully predicted depression, anxiety, and stress, offering insights into the relationship between self-conceptualization and mental health. The adoption of scalable language-based assessments with modern distributed computation presents a promising avenue for advancing community and individual mental health research.

Speaker: Siddharth Mangalik

https://stonybrook.zoom.us/j/91251321639?pwd=faggV5jZ7ByFDCFmnLXD3HiYxjQ1Eb.1&jst=2
Abstract: Many scientific and engineering challenges, such as the design of materials or molecules or the control of experimental systems, rely on the existence of fast predictive models that can evaluate potential designs or control policies. Traditionally this has been accomplished through numerical simulation; more recently data-driven machine learning methods have been applied. However, both approaches leave gaps: physical modeling can be accurate and extrapolates well to previously-unstudied conditions, but it is often computationally expensive and relies on physics approximations that may not be valid. Machine learning can generalize from massive amounts of real-world or simulation data, but suffers from physical grounding and extrapolation into new regimes, as well as in settings where large data sets do not exist.
In this talk I explore an intermediate regime, which is hybrid reduced order models: fast simplified physics approximations where some of the unknown or approximated equations are replaced with data-driven machine learning components. Examples include coarse-grained models where the full macroscopic equations cannot be derived from first-principles microscopic equations, multiscale models with unknown closure terms or sub-grid parameterization schemes, and low-order or latent dynamical systems that learn governing equations on a low-dimensional reduced state space. I discuss how such reduced systems can be identified from very limited data, much less than is often needed in traditional machine learning but at much lower time-to-solution than traditional numerical modeling. This facilitates not only system design and control but also uncertainty quantification approaches that search the space of possible equations for predictive models that can explain the data. I will focus on an example from materials science concerning the design of self-assembling block copolymer nanomaterials.

Speaker: Dr. Nathan Urban, Applied Mathematics Department, Brookhaven National Laboratory

Location: Laufer 101

Zoom: https://stonybrook.zoom.us/j/96090260834?pwd=mw8QTHbMOw9oeU9hazZeoq8bN4VIfH.1
Meeting ID: 960 9026 0834 Passcode: 374969

Abstract: Pretraining vision encoders with self-supervision (SSL) leads to stronger representations that excel across diverse downstream tasks. One of the key factors enabling self-supervision is extracting multiple views of the same scene to formulate either: 1) View-invariant pretraining (DINO, SimCLR, iBOT), where the objective is predicting the same representation for different views of the scene; or 2) Cross-view pretraining (cross-view Masked Autoencoders), where the objective is predicting missing parts of one view using other views. For extracting multiple views, view-invariant methods rely on a combination of handcrafted augmentations (random cropping, color jittering, gaussian blur, etc.) of the same image, whereas cross-view pretraining methods rely on image cropping or video frames. In this work, we present methods to effectively incorporate synthetic views from diffusion models into SSL training.
For view-invariant pretraining, we introduce Gen-SIS, a method that leverages the ability of diffusion models to generate interpolated images through interpolation in conditioning space. We introduce a disentanglement pretext task: disentangling two source images from an interpolated synthetic image. This disentanglement task, in addition to vanilla single-source generative augmentation for view extraction, improves visual pretraining of various view-invariant methods (DINO, SimCLR, iBOT).
For cross-view pretraining, we introduce CDG-MAE, a novel cross-view masked autoencoder (MAE) based method that uses diverse synthetic views generated from static images via an image-conditioned diffusion model to learn dense correspondences. We present a quantitative method to evaluate the local and global consistency of the generated views to choose the right diffusion model for cross-view pretraining. These generated views exhibit substantial changes in pose and perspective, providing a rich training signal that overcomes the limitations of video (expensive) and crop-based (less variation) methods. CDG-MAE substantially narrows the gap to video-based MAE methods on video label propagation tasks while maintaining the data advantages of image-only MAEs.

Speaker: Varun Belagali

Location: NCS 120
Zoom: https://stonybrook.zoom.us/j/93647452432?pwd=hZaX7LXCAD8KPHWYE1Afw2sDI3owpv.1
The overall purpose of this seminar is to bring together people with interests in Computer Vision theory and techniques and to examine current research issues. This course will be appropriate for people who already took a Computer Vision graduate course or already had research experience in Computer Vision. To enroll in this course, you must either: (1) be in the PhD program or (2) receive permission from the instructors.

Each seminar will consist of multiple short talks (around 10 minutes) by multiple people. Students can register for 1 credit for CSE 656. Registered students must attend and present a minimum of 2 or 3 talks. Everyone else is welcome to attend. Fill in https://forms.gle/pCVXovgfMfQwGqG38 to subscribe to our mailing list for further announcement.