Abstract:
Artificial Intelligence for Science (AI4Sci) has become a transformative approach in
modeling and understanding complex physical systems, encompassing different scales such as
atomistic systems and continuum systems. In atomistic systems, AI has shown potential in
accelerating simulations, optimizing molecular dynamics, and predicting material and
molecular properties through data-driven approaches, enhancing computational efficiency
while preserving accuracy. For continuum systems, AI provides powerful tools for solving
partial differential equations (PDEs) and learning physical patterns from data, capturing
intricate dynamics that govern physical and engineering processes. This work explores AI
methods--particularly equivariance for neural networks and neural operators--bridging
atomistic and continuum representations. We analyze the implications of incorporating
symmetries to improve model robustness and learning efficiency, providing a cohesive AI- driven framework for advancing scientific discovery. The findings aim to underscore the role
of AI in enhancing accuracy, applicability, scalability, interopretability, and generalization
across scales, from molecular simulations to physical modeling, opening pathways for next- generation applications in computational science.
Biography:
Wenhan Gao is a third-year Ph.D. student in Applied Mathematics at Stony Brook University, where he works under the supervision of Professor Yi Liu. Wenhan's research focuses on
equivariant neural networks, graph neural networks, and AI for partial differential equations. Wenhan's work seeks to leverage the power of symmetries to aid AI models, particularly in
fields such as computer vision (image and video generation), physical simulation (modeling
climate change), and computational chemistry (drug discovery). He has published papers on
the aforementioned topics in leading venues like NeurIPS, Transactions on Machine Learning
Research (TMLR), and Journal of Computational Physics (JCP). He also has several preprints
under review in leading venues like ICLR and CVPR. In addition to his research, Wenhan has
served as a reviewer for top-tier conferences, including ICLR, NeurIPS, ICML, and KDD, and
as a lecturer for undergraduate and graduate courses at Stony Brook University. Wenhan was
awarded the NeurIPS Travel Award and Excellence in Teaching for Fall 2023.
Abstract: Recent studies have highlighted the vulnerability of Natural Language Processing (NLP) and Vision-Language Models (VLMs) to backdoor attacks, posing significant security risks. Understanding these attack strategies is crucial for assessing model robustness and developing effective defenses. This thesis proposal aims to investigate the vulnerability of language and vision-language models, analyze abnormal behaviors in backdoor-attacked models, and develop defense methods to enhance safety of modern machine learning models at deployment.
We investigate the internal mechanisms of backdoored NLP models, identifying a distinct attention focus drifting phenomenon, where trigger tokens hijack attention regardless of the input context. Through comprehensive qualitative and quantitative analysis, we provide insights into the underlying mechanisms that enable backdoor attacks. Building on these insights, we propose detection methods to differentiate backdoored models from clean ones, through inspecting both the attention distribution and the model predictions. To better understand the vulnerability, we develop advanced backdoor attack strategies targeting language models in classification tasks. For BERT variants, we introduce Trojan Attention Loss (TAL), a novel method that directly manipulates attention patterns to enhance backdoor effectiveness, ensuring stealth and robustness. Vision-Language Models have demonstrated strong performance in recent years. Yet their vulnerability is largely underexplored. We investigate advanced backdoor attack strategies on Vision-Language Models, focusing on image-to-text generation tasks. We demonstrate how backdoors can be embedded in complex multimodal tasks while maintaining semantic integrity under poisoned inputs. Additionally, we propose innovative techniques for injecting backdoors without requiring access to the original training data, expanding the feasibility of real-world attacks.
This proposal provides novel insights into the internal mechanisms of backdoored models, propose effective detection strategies, and develop advanced attack techniques that expose critical vulnerabilities. These findings underscore the urgent need for robust security measures to defend against emerging backdoor threats in deep learning models. The results have been published in top venues including ICLR, ECCV, NAACL, EMNLP, etc.
Speaker: Weimin Lyu
Zoom link: https://stonybrook.zoom.us/j/
Meeting ID: 998 8060 5139
Passcode: 843302
We investigate the internal mechanisms of backdoored NLP models, identifying a distinct attention focus drifting phenomenon, where trigger tokens hijack attention regardless of the input context. Through comprehensive qualitative and quantitative analysis, we provide insights into the underlying mechanisms that enable backdoor attacks. Building on these insights, we propose detection methods to differentiate backdoored models from clean ones, through inspecting both the attention distribution and the model predictions. To better understand the vulnerability, we develop advanced backdoor attack strategies targeting language models in classification tasks. For BERT variants, we introduce Trojan Attention Loss (TAL), a novel method that directly manipulates attention patterns to enhance backdoor effectiveness, ensuring stealth and robustness. Vision-Language Models have demonstrated strong performance in recent years. Yet their vulnerability is largely underexplored. We investigate advanced backdoor attack strategies on Vision-Language Models, focusing on image-to-text generation tasks. We demonstrate how backdoors can be embedded in complex multimodal tasks while maintaining semantic integrity under poisoned inputs. Additionally, we propose innovative techniques for injecting backdoors without requiring access to the original training data, expanding the feasibility of real-world attacks.
This proposal provides novel insights into the internal mechanisms of backdoored models, propose effective detection strategies, and develop advanced attack techniques that expose critical vulnerabilities. These findings underscore the urgent need for robust security measures to defend against emerging backdoor threats in deep learning models. The results have been published in top venues including ICLR, ECCV, NAACL, EMNLP, etc.
Speaker: Weimin Lyu
Zoom link: https://stonybrook.zoom.us/j/
Meeting ID: 998 8060 5139
Passcode: 843302
Abstract: Artificial Intelligence (AI) is no longer a futuristic concept -- it is here, but its development, benefits, and risks remain unevenly distributed across industries, nations, and social groups. In this talk, Jieshu presents her research on the societal dimensions of AI from two perspectives: the forces shaping AI's development (backward-looking) and its current and potential impact on society (forward-looking). She first examines disparities in AI, including women's underrepresentation in AI patents and the geographic concentration of AI innovation, highlighting inequalities in who creates AI and who benefits from it. She then explores AI's societal impact, focusing on workforce transformation and the need for GenAI literacy. She will also discuss AI patents, AI's role in climate change mitigation and adaptation, potential environmental biases in LLMs, and gender-specific patterns in AI portrayals in science fiction.
Bio: Jieshu Wang is a Postdoctoral Research Scholar at Arizona State University (ASU), focusing on the social dimensions of artificial intelligence (AI). With a background in engineering, economics, communication, and science and technology studies, she examines how AI both shapes and is shaped by broader societal forces. Her research employs interdisciplinary methods to explore the social, political, and economic factors influencing AI development, as well as its role in innovation, the economy, the future of work, climate change mitigation, and popular culture. Jieshu holds a Ph.D. in Human and Social Dimensions of Science and Technology from ASU. She is also a science book translator and has translated six books.
Location: Old Computer Science, room 1310
Bio: Jieshu Wang is a Postdoctoral Research Scholar at Arizona State University (ASU), focusing on the social dimensions of artificial intelligence (AI). With a background in engineering, economics, communication, and science and technology studies, she examines how AI both shapes and is shaped by broader societal forces. Her research employs interdisciplinary methods to explore the social, political, and economic factors influencing AI development, as well as its role in innovation, the economy, the future of work, climate change mitigation, and popular culture. Jieshu holds a Ph.D. in Human and Social Dimensions of Science and Technology from ASU. She is also a science book translator and has translated six books.
Location: Old Computer Science, room 1310
The overall purpose of this seminar is to bring together people with interests in Computer Vision theory and techniques and to examine current research issues. This course will be appropriate for people who already took a Computer Vision graduate course or already had research experience in Computer Vision. To enroll in this course, you must either: (1) be in the PhD program or (2) receive permission from the instructors.
Each seminar will consist of multiple short talks (around 10 minutes) by multiple people. Students can register for 1 credit for CSE 656. Registered students must attend and present a minimum of 2 or 3 talks. Everyone else is welcome to attend. Fill in https://forms.gle/pCVXovgfMfQwGqG38 to subscribe to our mailing list for further announcement.
Each seminar will consist of multiple short talks (around 10 minutes) by multiple people. Students can register for 1 credit for CSE 656. Registered students must attend and present a minimum of 2 or 3 talks. Everyone else is welcome to attend. Fill in https://forms.gle/pCVXovgfMfQwGqG38 to subscribe to our mailing list for further announcement.
Abstract: As intelligent systems become more integrated into human environments, fostering trustworthy human-AI collaboration presents a pressing challenge. In this talk, I examine the interplay between an agent's performance and social dynamics in shaping trust in human-AI interactions. My approach combines testbed development, behavioral prototyping, and user study design to create controlled experimental setups that capture real-world interaction complexities, such as ambiguity, multi-agent dynamics, and conflicting goals.
I illustrate this with a recent VR study on multi-user interaction with an autonomous vehicle (AV). Moving beyond dyadic interactions, the study probes human perspectives from the roles of a pedestrian, driver, and AV passenger, all interacting with the AV simultaneously at an ambiguous all-way stop sign intersection. We compare interactions with efficient and prosocial AV behavior strategies, revealing diverging trust perceptions and preferences across user roles. These insights inform a broader research trajectory focused on balancing performance with social considerations in designing trustworthy human-AI collaborations.
Bio: JiHyun Jeong is a postdoctoral researcher at Cornell University working on human-computer interaction and human-robot interaction. Her research develops prototypes and methods to explore performance and social factors that influence collaboration and trust between humans and artificial agents. She holds a Ph.D. and MPS in Information Science from Cornell University, and a BSc in Computer Science and Engineering from Korea University. She is a recipient of an honorable mention for best paper at DIS.
Zoom: https://stonybrook.zoom.
Meeting ID: 987 3823 4619
Passcode: 474618
AI for Conservation: AI and Humans Combating Extinction Together by Daniel I. Rubenstein of Princeton University
ABSTRACT: The state of our planet is not good. We have lost more than 60% of the world's wildlife. Stopping the decline remains a challenge, especially since acquiring appropriate knowledge is expensive, time consuming and risky. Visual observations following the fates of a few individuals was the currency of the realm. But GPS technology and now machine learning provide a non-invasive scalable alternative. Photographs, taken by field scientists, tourists, automated cameras and incidental photographers, are the most abundant source of data on wildlife today. Wildbook, a project of tech for conservation coordinated by a non-profit Wild Me, is an autonomous computational system that starts from massive collections of images and, by detecting various species of animals and identifying individuals, combined with sophisticated data management, turns them into high-resolution information databases, enabling scientific inquiry, conservation and citizen science.
BIO: Dan Rubenstein is the Class of 1877 Professor of Zoology. He is currently Director of Princeton's Environmental Studies Program and is former Chair of Princeton University's Department of Ecology and Evolutionary Biology and Director of Princeton's Program in African Studies. He is a behavioral ecologist who studies how environmental variation and individual differences shape social behavior, social structure, sex
roles and the dynamics of populations. He has special interests in all species of wild horses, zebras and asses, and has done field work on them throughout the world identifying rules governing decision-making, the emergence of complex behavioral patterns and how these understandings influence their management
and conservation. In Kenya he also works with pastoral communities to develop and assess impacts of various grazing strategies on rangeland quality, wildlife use and livelihoods. He has also developed a scout program for gathering data on Grevy's zebras and created curricular modules for local schools to raise awareness about the plight of this endangered species. He engages people as 'Citizen Scientists' and has recently extended his work to measuring the effects of environmental change, including issues pertaining to the global commons
and changes wrought by management and by global warming, on behavior.
ABSTRACT: The state of our planet is not good. We have lost more than 60% of the world's wildlife. Stopping the decline remains a challenge, especially since acquiring appropriate knowledge is expensive, time consuming and risky. Visual observations following the fates of a few individuals was the currency of the realm. But GPS technology and now machine learning provide a non-invasive scalable alternative. Photographs, taken by field scientists, tourists, automated cameras and incidental photographers, are the most abundant source of data on wildlife today. Wildbook, a project of tech for conservation coordinated by a non-profit Wild Me, is an autonomous computational system that starts from massive collections of images and, by detecting various species of animals and identifying individuals, combined with sophisticated data management, turns them into high-resolution information databases, enabling scientific inquiry, conservation and citizen science.
BIO: Dan Rubenstein is the Class of 1877 Professor of Zoology. He is currently Director of Princeton's Environmental Studies Program and is former Chair of Princeton University's Department of Ecology and Evolutionary Biology and Director of Princeton's Program in African Studies. He is a behavioral ecologist who studies how environmental variation and individual differences shape social behavior, social structure, sex
roles and the dynamics of populations. He has special interests in all species of wild horses, zebras and asses, and has done field work on them throughout the world identifying rules governing decision-making, the emergence of complex behavioral patterns and how these understandings influence their management
and conservation. In Kenya he also works with pastoral communities to develop and assess impacts of various grazing strategies on rangeland quality, wildlife use and livelihoods. He has also developed a scout program for gathering data on Grevy's zebras and created curricular modules for local schools to raise awareness about the plight of this endangered species. He engages people as 'Citizen Scientists' and has recently extended his work to measuring the effects of environmental change, including issues pertaining to the global commons
and changes wrought by management and by global warming, on behavior.
The University's Main Commencement Ceremony will take place on Friday, May 23, 2025 at 11 am at Kenneth P. LaValle Stadium. Gates open at 10 am.
All guests need a valid ticket to enter LaValle Stadium - no exceptions. Children age 1 and older require a ticket. Seating is first-come, first-served.
Register here.
The Renaissance School Of Medicine Department of Scientific Affairs and its Single Cell Genomics facility are excited to host a special seminar and discussion on AI and single cell genomics analysis:
With the decreasing cost of sequencing, many biobanks and large research cohorts have moved to whole genome sequencing (WGS) and single-cell RNA-seq. However, making use of this deluge of data remains a challenge. I will discuss statistical and deep learning approaches that we are exploring to address the challenge of noncoding variant interpretation, including our work as part of the Alzheimer's disease sequencing project.
Speaker: David A. Knowles, PhD. Asst. Professor of Computer Science, Interdisciplinary Appointee in Systems Biology, Columbia University Core Faculty Member, New York Genome Center
Join us in person: Health Science Tower Level 3, Lecture Hall 5
With the decreasing cost of sequencing, many biobanks and large research cohorts have moved to whole genome sequencing (WGS) and single-cell RNA-seq. However, making use of this deluge of data remains a challenge. I will discuss statistical and deep learning approaches that we are exploring to address the challenge of noncoding variant interpretation, including our work as part of the Alzheimer's disease sequencing project.
Speaker: David A. Knowles, PhD. Asst. Professor of Computer Science, Interdisciplinary Appointee in Systems Biology, Columbia University Core Faculty Member, New York Genome Center
Join us in person: Health Science Tower Level 3, Lecture Hall 5
Abstract:
Photorealistic editing of human facial expressions and head articulations remains a long-standing topic in the computer graphics and computer vision community. Methods enabling such control have great potential in AR/VR applications where a 3D immersive experience is valuable, especially when this control extends to novel views of the scene in which the human subject appears. Traditionally, 3D Morphable Face Models (3DMMs) have been used to control the facial expressions and head pose of a human head. However, the PCA-based shape and expression spaces of 3DMMs lack the expressivity. They cannot model essential elements of the human head such as hair, skin details, and accessories such as glasses that are paramount for realistic reanimation. In this thesis, we present a set of methods that enables facial reanimation, starting from editing expressions in still face images to creating fully controllable neural 3D portraits with control over facial expressions, head pose, and viewing direction of the scene using only casually captured monocular videos from a smartphone to finally achieving studio-like quality from the said monocular captures.
First, we propose a method for editing facial expressions in near-frontal facial images through the unsupervised disentangling of expression-induced deformations and texture changes. Next, we extend facial expression editing to human subjects in 3D scenes. We represent the scene and the subject in it using a semantically guided neural field. This enables control over the subject's facial expressions and the viewing direction of the scene they're in. We then present a method that learns, in an unsupervised manner, to deform static 3D neural fields using facial expression and head-pose dependent deformations, enabling control over facial expressions and head pose of the subject along with the viewing direction of the 3D scene they're in. Next, we propose a method that makes the learning of the aforementioned deformation field robust to strong illumination effects, which adversely impact the registration of the deformation. We then propose an extension of this unsupervised deformation model to 3D Gaussian splatting by constraining it using a 3D morphable model, resulting in a rendering speed of 18 FPS--a 100x speed improvement over prior work. Finally, we propose a method that bridges the quality gap between 3D portraits created using in-the-wild monocular data and multi-view studio capture data. We accomplish this using a two-stage method. First, we train a StyleGAN to relight and inpaint in-the-wild face texture maps (with strong illumination effects and incompletely captured regions). Next, we both reconstruct and generate identity-specific facial details that may be poorly captured in the in-the-wild captures. Once trained, we can generate studio-like complete avatars from monocular phone captures.
Speaker: Shahrukh Athar
Zoom Link:
https://stonybrook.zoom.us/j/94228500743?pwd=RqOBgG6tbJkKaFBlWFwBkYFX0VRovV.1
Meeting ID: 94228500743
Passcode: 661599
Photorealistic editing of human facial expressions and head articulations remains a long-standing topic in the computer graphics and computer vision community. Methods enabling such control have great potential in AR/VR applications where a 3D immersive experience is valuable, especially when this control extends to novel views of the scene in which the human subject appears. Traditionally, 3D Morphable Face Models (3DMMs) have been used to control the facial expressions and head pose of a human head. However, the PCA-based shape and expression spaces of 3DMMs lack the expressivity. They cannot model essential elements of the human head such as hair, skin details, and accessories such as glasses that are paramount for realistic reanimation. In this thesis, we present a set of methods that enables facial reanimation, starting from editing expressions in still face images to creating fully controllable neural 3D portraits with control over facial expressions, head pose, and viewing direction of the scene using only casually captured monocular videos from a smartphone to finally achieving studio-like quality from the said monocular captures.
First, we propose a method for editing facial expressions in near-frontal facial images through the unsupervised disentangling of expression-induced deformations and texture changes. Next, we extend facial expression editing to human subjects in 3D scenes. We represent the scene and the subject in it using a semantically guided neural field. This enables control over the subject's facial expressions and the viewing direction of the scene they're in. We then present a method that learns, in an unsupervised manner, to deform static 3D neural fields using facial expression and head-pose dependent deformations, enabling control over facial expressions and head pose of the subject along with the viewing direction of the 3D scene they're in. Next, we propose a method that makes the learning of the aforementioned deformation field robust to strong illumination effects, which adversely impact the registration of the deformation. We then propose an extension of this unsupervised deformation model to 3D Gaussian splatting by constraining it using a 3D morphable model, resulting in a rendering speed of 18 FPS--a 100x speed improvement over prior work. Finally, we propose a method that bridges the quality gap between 3D portraits created using in-the-wild monocular data and multi-view studio capture data. We accomplish this using a two-stage method. First, we train a StyleGAN to relight and inpaint in-the-wild face texture maps (with strong illumination effects and incompletely captured regions). Next, we both reconstruct and generate identity-specific facial details that may be poorly captured in the in-the-wild captures. Once trained, we can generate studio-like complete avatars from monocular phone captures.
Speaker: Shahrukh Athar
Zoom Link:
https://stonybrook.zoom.us/j/94228500743?pwd=RqOBgG6tbJkKaFBlWFwBkYFX0VRovV.1
Meeting ID: 94228500743
Passcode: 661599
The annual conference on Neural Information Processing Systems is a multi-track interdisciplinary annual meeting that includes invited talks, demonstrations, symposia, and oral and poster presentations of refereed papers. Along with the conference is a professional exposition focusing on machine learning in practice, a series of tutorials, and topical workshops that provide a less formal setting for the exchange of ideas.
For more information and registration, visit the official website.
For more information and registration, visit the official website.