KDD 2020 - The premier interdisciplinary conference bringing together researchers and practitioners from data science, data mining, knowledge discovery, large-scale data analytics, and big data. https://www.kdd.org/kdd2020/
Abstract: Many scientific and engineering challenges, such as the design of materials or molecules or the control of experimental systems, rely on the existence of fast predictive models that can evaluate potential designs or control policies. Traditionally this has been accomplished through numerical simulation; more recently data-driven machine learning methods have been applied. However, both approaches leave gaps: physical modeling can be accurate and extrapolates well to previously-unstudied conditions, but it is often computationally expensive and relies on physics approximations that may not be valid. Machine learning can generalize from massive amounts of real-world or simulation data, but suffers from physical grounding and extrapolation into new regimes, as well as in settings where large data sets do not exist.
In this talk I explore an intermediate regime, which is hybrid reduced order models: fast simplified physics approximations where some of the unknown or approximated equations are replaced with data-driven machine learning components. Examples include coarse-grained models where the full macroscopic equations cannot be derived from first-principles microscopic equations, multiscale models with unknown closure terms or sub-grid parameterization schemes, and low-order or latent dynamical systems that learn governing equations on a low-dimensional reduced state space. I discuss how such reduced systems can be identified from very limited data, much less than is often needed in traditional machine learning but at much lower time-to-solution than traditional numerical modeling. This facilitates not only system design and control but also uncertainty quantification approaches that search the space of possible equations for predictive models that can explain the data. I will focus on an example from materials science concerning the design of self-assembling block copolymer nanomaterials.
Speaker: Dr. Nathan Urban, Applied Mathematics Department, Brookhaven National Laboratory
Location: Laufer 101
Zoom: https://stonybrook.zoom.us/j/96090260834?pwd=mw8QTHbMOw9oeU9hazZeoq8bN4VIfH.1
Meeting ID: 960 9026 0834 Passcode: 374969
In this talk I explore an intermediate regime, which is hybrid reduced order models: fast simplified physics approximations where some of the unknown or approximated equations are replaced with data-driven machine learning components. Examples include coarse-grained models where the full macroscopic equations cannot be derived from first-principles microscopic equations, multiscale models with unknown closure terms or sub-grid parameterization schemes, and low-order or latent dynamical systems that learn governing equations on a low-dimensional reduced state space. I discuss how such reduced systems can be identified from very limited data, much less than is often needed in traditional machine learning but at much lower time-to-solution than traditional numerical modeling. This facilitates not only system design and control but also uncertainty quantification approaches that search the space of possible equations for predictive models that can explain the data. I will focus on an example from materials science concerning the design of self-assembling block copolymer nanomaterials.
Speaker: Dr. Nathan Urban, Applied Mathematics Department, Brookhaven National Laboratory
Location: Laufer 101
Zoom: https://stonybrook.zoom.us/j/96090260834?pwd=mw8QTHbMOw9oeU9hazZeoq8bN4VIfH.1
Meeting ID: 960 9026 0834 Passcode: 374969
Join us at the Center for Excellence in Learning and Teaching (CELT) for an interactive Zoom workshop on Generative AI designed for faculty and staff interested in enhancing teaching and assessment practices, increasing student engagement, and navigating the rapidly evolving landscape of AI tools. Participants will be introduced to common AI tools, explore potential instructional uses, and discuss key considerations such as academic integrity, transparency, and equity.
Register now: https://stonybrook.zoom.us/meeting/register/6js1eP64T1ys8tyU57EJ7Q#/registration
Register now: https://stonybrook.zoom.us/meeting/register/6js1eP64T1ys8tyU57EJ7Q#/registration
The Future Histories Studio at Stony Brook University and Guggenheim New York are collaborating to present a day-long symposium on October 24 at the Simons Center for Geometry and Physics. This conference will explore urgent questions at the intersection of artificial intelligence, machine learning, and the human, and is co-organized by Noam Segal, LG Electronics Associate Curator at Guggenheim New York. In this role, Noam plays an important part in researching these topics, promoting a deeper understanding of the ways in which contemporary artists use new technologies, and developing and supporting the Guggenheim's engagement with technology-based art under the LG Guggenheim Art and Technology Initiative.
The event examines the profound transformations brought by automation--how AI compels us to rethink cognition, agency, and the conditions of reason itself. As these systems become ever more embedded in daily life--largely invisible yet deeply consequential--they challenge the very foundations of subjectivity and governance. We are surrounded by logics we cannot fully access, yet which shape our realities, while new forms of alterity arise--distinct modes of reasoning that propose collective unknowns beyond established frameworks of knowledge.
This emerging terrain invites us to consider cognitive plurality, where biological and technological intelligences generate new categories, concepts, and understandings. Once unique to humans--art, authorship, judgment, invention--are now co-articulated with systems of computation and planetary-scale infrastructure. The symposium brings together artists, scholars, and technologists to probe the cultural, philosophical, and ecological implications of this entanglement.
The concept of neurodiversity has shown that neurological differences such as autism, ADHD, and dyslexia are not deficits but variations that enrich collective life. Extending this to machines can be provocative: just as neurodivergence unsettles fixed definitions of intelligence, so too AI challenges anthropocentric assumptions about cognition. Yet the analogy is limited. Neurodiversity is rooted in the lived struggles of human communities, while machines neither think nor struggle. Human cognition involves perception, learning, memory, and reasoning through embodied experience. Machine cognition, by contrast, is computational pattern recognition and statistical modeling, without consciousness or lived context, and with only narrow forms of sensing.
For this reason, the symposium advances a broader framework of cognitive diversity or technodiversity--a recognition of proliferating intelligences, human, machinic, and hybrid, as part of a shared ecology. This shift calls for new models of creativity, responsibility, and collaboration that honor the irreducibility of human thought while engaging the radical alterity of machine logics.
Location: Stony Brook Simons Center for Geometry and Physics, Della Pietra Family Auditorium
This event is co organized by the Guggenheim New York
Climate Uncertainty, Decision Making, and AI for Earth System Predictability Dr. Nathan Urban, Brookhaven National Laboratory
Bio: Nathan Urban is the group leader of the Optimal Experimental Design & Uncertainty Quantification group in the Applied Mathematics Department at Brookhaven National Laboratory's Computing & Data Sciences directorate (CDS). He holds a Ph.D. in theoretical condensed matter physics from Penn State, and has previously held research positions at Los Alamos National Laboratory, Princeton, and Penn State. His research interests include Bayesian inference and spatiotemporal statistics, probabilistic prediction and forecasting, multi-model / model-form / model structural uncertainty quantification, reduced order modeling, scientific machine learning and hybrid physical-data driven modeling, in-situ/streaming data analysis at scale, information fusion, decision making under uncertainty and optimal experimental design, and integrated multiscale computational frameworks for decision support.
Location: IACS Seminar Room
Lunch will be provided
Bio: Nathan Urban is the group leader of the Optimal Experimental Design & Uncertainty Quantification group in the Applied Mathematics Department at Brookhaven National Laboratory's Computing & Data Sciences directorate (CDS). He holds a Ph.D. in theoretical condensed matter physics from Penn State, and has previously held research positions at Los Alamos National Laboratory, Princeton, and Penn State. His research interests include Bayesian inference and spatiotemporal statistics, probabilistic prediction and forecasting, multi-model / model-form / model structural uncertainty quantification, reduced order modeling, scientific machine learning and hybrid physical-data driven modeling, in-situ/streaming data analysis at scale, information fusion, decision making under uncertainty and optimal experimental design, and integrated multiscale computational frameworks for decision support.
Location: IACS Seminar Room
Lunch will be provided
Abstract:
Recent advances in deep learning have significantly enhanced the capabilities of Natural Language Processing (NLP) and Vision-Language Models (VLMs). However, these advancements come with increased vulnerabilities, notably through backdoor attacks that pose severe security threats. This thesis addresses two critical dimensions of Trustworthy AI and Efficient Multimodal Representation Learning: (1) security through analyzing, detecting, and designing backdoor attacks in NLP and VLMs, and (2) efficiency through advanced multimodal representation methods tailored for clinical and medical imaging applications.
In the first dimension, we explore the internal mechanisms exploited by backdoor attacks, identifying the distinctive phenomenon of attention focus drifting in compromised transformer models, where trigger tokens consistently hijack attention. Leveraging these insights, we propose robust detection frameworks, including the attention-based Trojan detector (AttenTD) and a task-agnostic logit-based detection method (TABDet), achieving effective identification of backdoored NLP models across diverse tasks. We further introduce novel backdoor attack methodologies: the Trojan Attention Loss (TAL), enhancing attack efficiency and stealth through direct attention manipulation, and BadCLM, demonstrating critical vulnerabilities in clinical decision-support systems by effectively compromising clinical language models.
Extending our security exploration to multimodal settings, we investigate backdoor attacks on Vision-Language Models (VLMs), particularly in complex image-to-text generation tasks, proposing innovative techniques (TrojVLM, VLOOD) capable of embedding backdoors without direct access to original training data, thus showcasing practical risks in real-world scenarios.
In the second dimension, we address efficiency and interpretability challenges in clinical and pathology applications. We introduce TCP-LLaVA, the first multimodal large language model (MLLM) designed explicitly for Whole Slide Image (WSI) Visual Question Answering (VQA). Utilizing a novel token compression mechanism inspired by transformer-based models, TCP-LLaVA substantially reduces computational resource consumption while maintaining superior VQA performance across multiple tumor subtypes. Additionally, we present a multimodal transformer model integrating structured Electronic Health Records (EHR) with clinical notes, demonstrating enhanced predictive accuracy and interpretability for in-hospital mortality prediction through integrated gradient-based interpretability methods.
Together, these contributions present a comprehensive approach to ensuring AI models are not only secure against malicious manipulation but also efficient and interpretable for critical clinical applications, underscoring the essential need for trustworthy and effective AI systems.
Speaker: Weimin Lyu
Zoom: https://stonybrook.zoom.us/j/
Meeting ID: 239 232 6575
Passcode: 436192
Recent advances in deep learning have significantly enhanced the capabilities of Natural Language Processing (NLP) and Vision-Language Models (VLMs). However, these advancements come with increased vulnerabilities, notably through backdoor attacks that pose severe security threats. This thesis addresses two critical dimensions of Trustworthy AI and Efficient Multimodal Representation Learning: (1) security through analyzing, detecting, and designing backdoor attacks in NLP and VLMs, and (2) efficiency through advanced multimodal representation methods tailored for clinical and medical imaging applications.
In the first dimension, we explore the internal mechanisms exploited by backdoor attacks, identifying the distinctive phenomenon of attention focus drifting in compromised transformer models, where trigger tokens consistently hijack attention. Leveraging these insights, we propose robust detection frameworks, including the attention-based Trojan detector (AttenTD) and a task-agnostic logit-based detection method (TABDet), achieving effective identification of backdoored NLP models across diverse tasks. We further introduce novel backdoor attack methodologies: the Trojan Attention Loss (TAL), enhancing attack efficiency and stealth through direct attention manipulation, and BadCLM, demonstrating critical vulnerabilities in clinical decision-support systems by effectively compromising clinical language models.
Extending our security exploration to multimodal settings, we investigate backdoor attacks on Vision-Language Models (VLMs), particularly in complex image-to-text generation tasks, proposing innovative techniques (TrojVLM, VLOOD) capable of embedding backdoors without direct access to original training data, thus showcasing practical risks in real-world scenarios.
In the second dimension, we address efficiency and interpretability challenges in clinical and pathology applications. We introduce TCP-LLaVA, the first multimodal large language model (MLLM) designed explicitly for Whole Slide Image (WSI) Visual Question Answering (VQA). Utilizing a novel token compression mechanism inspired by transformer-based models, TCP-LLaVA substantially reduces computational resource consumption while maintaining superior VQA performance across multiple tumor subtypes. Additionally, we present a multimodal transformer model integrating structured Electronic Health Records (EHR) with clinical notes, demonstrating enhanced predictive accuracy and interpretability for in-hospital mortality prediction through integrated gradient-based interpretability methods.
Together, these contributions present a comprehensive approach to ensuring AI models are not only secure against malicious manipulation but also efficient and interpretable for critical clinical applications, underscoring the essential need for trustworthy and effective AI systems.
Speaker: Weimin Lyu
Zoom: https://stonybrook.zoom.us/j/
Meeting ID: 239 232 6575
Passcode: 436192
Abstract:
It is known that models like large language models (LLMs) can often suggest colloquial plans given verbal descriptions of tasks, yet they are unable to reliably provide executable and verifiable plans given formally specified environments. In this talk, I will discuss a strand of efforts to have LLMs generate accurate and explainable plans in textual simulations. Instead of directly generating the plan or actions, LLMs are prompted to generate Planning Domain Definition Language (PDDL) that specifies the environment (domain file) and the task (problem file), which can then be deterministically solved with an off-the-shelf planner. In a 3-phase study, my collaborators and I first observed that it is possible but very challenging for LLMs to generate long-form code such as PDDL domain and problem files given textual specifications. Next, we devise methodologies for LLMs to iteratively generate and refine problem files while exploring a partially-observed, simulated, textual environment. Finally, we show that domain files are even more difficult to generate correctly, even on well-established planning tasks such as BlocksWorld. Finally, I will discuss ongoing efforts to improve said ability of structured generation and promising frontiers to explore.
Bio:
Li Harry Zhang is an assistant professor at Drexel University, focusing on Natural Language Processing (NLP) and artificial intelligence (AI). He obtained his PhD degree from the University of Pennsylvania advised by Prof. Chris Callison-Burch. Prior, he obtained his Bachelor's degree at the University of Michigan mentored by Prof. Rada Mihalcea and Prof. Dragomir Radev. His current research uses large language models (LLMs) to reason and plan via symbolic and structured representations. He has published more than 20 peer-reviewed papers in NLP and AI conferences, such as ACL, EMNLP, and AACL, that have been cited more than 1,000 times. He also consistently serves as Area Chair, Session Chair, and reviewer in those venues. Being a musician, producer, and content creator having over 50,000 subscribers, he is also passionate in the research of AI music and creativity.
What new tools and extra powers are available to you through the library's subscription databases? Join faculty librarian Chris Kretz, Head of Academic Engagement, on a tour of what's available and how you might best use them.