Abstract: Humans perceive the world through global structures such as parts, branches, and their spatial arrangement. Most deep learning models, however, operate mainly at the pixel level. This disconnect between local and global understanding limits interpretability and control. In this thesis, we explore topology as a mathematical framework for bridging local predictions and global structure in dense prediction and generation tasks. We first incorporate topological constraints into semantic segmentation to preserve anatomical relationships and improve multi-class consistency. We next develop structure-level uncertainty estimation, producing more interpretable and actionable measures of model error over branches and connections rather than isolated pixels. Then, we introduce a topology-guided diffusion framework for controllable image generation using structural attributes such as object count and connectivity. Finally, we extend image generation to the longitudinal task, where we aim to capture structural changes across timepoints. All these contributions together establish topology as a unifying interface for building dense prediction models that are structurally aware, interpretable, and controllable.

Speaker: Saumya Gupta

Location: NCS 220

Zoom: https://stonybrook.zoom.us/j/97950688136?pwd=NCa3XOsgIaMIsTVlQBQJ11n27NzL8s.1
Meeting ID: 979 5068 8136
Passcode: 941798
How to Do Spectral Learning at Scale for Science and Engineering

Abstract: Spectral decompositions such as singular value decompositions (SVDs) and eigenvalue decompositions (EVDs) are central tools across a vast swath of scientific computing and machine learning, with abundant engineering applications. Yet many modern methods for learning such decompositions in high dimensions struggle with instability, bias, and poor scalability, even when approximation power is not the limiting factor. I argue that these difficulties are not intrinsic to spectral problems, but instead arise from a shared reliance on Rayleigh-quotient-based constrained optimization, which forces explicit orthogonality handling through penalties, normalization, or whitening.
To address these challenges, I present a reformulation based on unconstrained variational objectives that implicitly encode spectral structure, eliminating the need for orthogonalization and ad-hoc regularization. This perspective leads to a conceptually simpler and scalable parametric framework for learning ordered spectral representations via nested optimization. The resulting framework is well matched to diverse settings in science and engineering. As examples, I demonstrate its effectiveness on eigenvalue problems for linear PDEs such as the Schrödinger equation, spectral (Koopman) analysis of nonlinear dynamical systems such as molecular dynamics, and structured representation learning with deep neural nets. Collectively, these examples illustrate how abandoning Rayleigh-quotient-based formulations resolves long-standing optimization pathologies across domains.

Bio: Jongha (Jon) Ryu is a postdoctoral associate at MIT EECS. He received his Ph.D. in Electrical and Computer Engineering from UC San Diego. His research develops statistical and mathematical foundations for scientific machine learning, with a focus on scalable spectral methods, efficient generative modeling, and reliable uncertainty quantification for scientific and engineering systems.

Location: NCS 120

The Department of AI and Society (AIS) at the University at Buffalo is hosting a two-day AI and Society Workshop focused on building AI systems by society, for society. This workshop brings together researchers and community organizers to explore how AI systems can be developed through meaningful collaboration across disciplines.

Topics include:

  • Labor and AI
  • Public services and AI
  • Community-centered AI systems
  • Intersections of humanities, social sciences, arts, and computing

The vision of UB's Department of AI and Society is to create a future where AI systems are built by society, for society. AIS centers community engagement at every stage of AI development through collaboration across disciplines and sectors. AIS was established with a $5 million grant from SUNY, and this workshop is made possible through that support.

Who Should Attend?

  • Researchers
  • Students
  • Community organizers
  • Practitioners interested in AI's societal impact

More about the event

Register here


We invite faculty to deliver a 10-minute presentation during our afternoon session at the CELT Symposium on April 11, 2025. Showcase how you use emerging technology (i.e. AI, VR, etc.) to support diverse student populations and enhance learning experiences. Share your innovative strategies and inspire others!

CELT Symposium Theme: A New Era of Inclusivity and Innovation in Higher Education

https://t.e2ma.net/click/5w0gph/5wwlu4oe/9v63j6
The GE Vernova Advanced Research Center invites you to the 2026 AI EDGE Symposium.

This collaborative and hands-on learning experience connects 300+ industry thought leaders in the fields of AI, Edge, Robotics, Cybersecurity, and Controls & Optimization.


Attendees will learn about the latest challenges and innovations with luminaries from key government agencies, industry customers, and technology partners as they discuss the latest advancements and trends in these fields.

Register here.

You are cordially invited to attend the biweekly Brookhaven AI Mixer (BAM). BAM includes three short talks on AI research happening at BNL, followed by an open mixer over coffee and snacks for everyone to network and discuss all things AI. The first half hour will consist of presentations that will be available via ZOOM, and the second half hour will be for in person only networking.

Join us every other Tuesday at noon in CDSD's Training Room (building 725, 2nd floor) to learn about interesting AI methods and applications, engage with potential collaborators, prepare for pending FASST funding calls, and build a community of AI for Science at BNL.

Speakers

Kriti Chopra, Computing & Data Sciences (CDS)
Thomas Flynn, Computing & Data Sciences (CDS)
Wenjie Liao, Chemistry Division

Tuesday, January 7, 2025, 12:00 pm -- CDS, Bldg. 725, Training Room

Join ZoomGov Meeting: https://bnl.zoomgov.com/j/1615289117?pwd=Hqkbj9itxWrFnkhZ8rQXHPInO2gxdF.1

Meeting ID: 161 528 9117
Passcode: 991382

Abstract: Recent studies have highlighted the vulnerability of Natural Language Processing (NLP) and Vision-Language Models (VLMs) to backdoor attacks, posing significant security risks. Understanding these attack strategies is crucial for assessing model robustness and developing effective defenses. This thesis proposal aims to investigate the vulnerability of language and vision-language models, analyze abnormal behaviors in backdoor-attacked models, and develop defense methods to enhance safety of modern machine learning models at deployment.


We investigate the internal mechanisms of backdoored NLP models, identifying a distinct attention focus drifting phenomenon, where trigger tokens hijack attention regardless of the input context. Through comprehensive qualitative and quantitative analysis, we provide insights into the underlying mechanisms that enable backdoor attacks. Building on these insights, we propose detection methods to differentiate backdoored models from clean ones, through inspecting both the attention distribution and the model predictions. To better understand the vulnerability, we develop advanced backdoor attack strategies targeting language models in classification tasks. For BERT variants, we introduce Trojan Attention Loss (TAL), a novel method that directly manipulates attention patterns to enhance backdoor effectiveness, ensuring stealth and robustness. Vision-Language Models have demonstrated strong performance in recent years. Yet their vulnerability is largely underexplored. We investigate advanced backdoor attack strategies on Vision-Language Models, focusing on image-to-text generation tasks. We demonstrate how backdoors can be embedded in complex multimodal tasks while maintaining semantic integrity under poisoned inputs. Additionally, we propose innovative techniques for injecting backdoors without requiring access to the original training data, expanding the feasibility of real-world attacks.

This proposal provides novel insights into the internal mechanisms of backdoored models, propose effective detection strategies, and develop advanced attack techniques that expose critical vulnerabilities. These findings underscore the urgent need for robust security measures to defend against emerging backdoor threats in deep learning models. The results have been published in top venues including ICLR, ECCV, NAACL, EMNLP, etc.

Speaker: Weimin Lyu


Zoom link: https://stonybrook.zoom.us/j/99880605139?pwd=cfWbRG6n9v3GXEa7OqvXa5cOp5eLBv.1
Meeting ID: 998 8060 5139
Passcode: 843302
AI can help you write, you hear. AI can save you time, leverage your skills, enhance your productivity. . . . But you also hear: AI output is not reliable, not adequate for advanced tasks/learning, not ethical to use -- you could get in deep trouble for using AI tools without adequate mastery and caution. Which way is it?
Come join this hands-on workshop where you will explore AI tools and their affordances. Engage in writing tasks to learn how to use AI tools effectively and responsibly.
Sign up for a seat now: https://docs.google.com/forms/d/e/1FAIpQLSd0iDTKkTYnkxFd4LkgqbtP97zQSS4FI_MiPVm7p6IY5SGwSg/viewform

You are cordially invited to attend the biweekly Brookhaven AI Mixer (BAM). BAM includes one short talk on AI research happening at BNL, followed by an open mixer over coffee and snacks for everyone to network and discuss all things AI. The first half hour will consist of presentations that will be available via ZOOM, and the second half hour will be for in person only networking.

Join us every other Tuesday at noon in CDSD's Training Room (building 725, 2nd floor) to learn about interesting AI methods and applications, engage with potential collaborators, prepare for pending FASST funding calls, and build a community of AI for Science at BNL.

Embodied Intelligence at Scientific User Facilities

Abstract: This presentation explores the active work integrating artificial intelligence and robotics at the National Synchrotron Light Source II, and a perspective for the future. Through various case studies, we highlight the optimization of operations, improved experimental outcomes, and the orchestration of distributed multimodal experiments. This ongoing development includes collaborators from across the light and neutron sources in the DOE complex. We will elaborate on the open-source Bluesky project, and its capabilities to support adaptive and autonomous experiments. Additionally, we will discuss how Bluesky can be integrated with open-source robotic control software to unlock new flexible automation for autonomous scientific research, which scales to new experiments and continues to leverage human ingenuity.

Biography: Dr. Phillip M. Maffettone is an Associate Computational Scientist in the Data Science and Systems Integration Division at NSLS-II. His research focuses on accelerating scientific discovery at user facilities through the integration of robotics, artificial intelligence (AI), and advanced experiment orchestration systems. He leads the N3XTware project, constructing the software architecture for the next 12 beamlines to be built at NSLS-II. Prior to this he built the brain on the world's first mobile robotic scientist at the University of Liverpool, and later spearheaded the machine learning platform for a biotechnology start-up, BigHat Biosciences. He holds a DPhil in Inorganic Chemistry from the University of Oxford and a B.S. in Chemical Engineering from the University at Buffalo.

Location: CDS, Bldg. 725, Training Room

Link: https://bnl.zoomgov.com/j/16049713 31?pwd=nc5CV3cOFrdYxordFieP W07tIDmwYb.1

Meeting ID: 160 497 1331
Passcode: 289875

Abstract: Retrieval-augmented generation (RAG) systems empower large language models (LLMs) to access external knowledge during inference. Recent advances have enabled LLMs to act as search agents via reinforcement learning (RL), improving information acquisition through multi-turn interactions with retrieval engines. However, existing approaches either optimize retrieval using search-only metrics (e.g., NDCG) that ignore downstream utility or fine-tune the entire LLM to jointly reason and retrieve--entangling retrieval with generation and limiting the real search utility and compatibility with frozen or proprietary models. In this work, we propose s3, a lightweight, model-agnostic framework that decouples the searcher from the generator and trains the searcher using a Gain Beyond RAG reward: the improvement in generation accuracy over naïve RAG. s3 requires only 2.4k training samples to outperform baselines trained on over 70 × more data, consistently delivering stronger downstream performance across six general QA and five medical QA benchmarks.

Speaker: Peter Zeng

Location: CS2311