AI Innovation Institute

Abstract: Recent studies have highlighted the vulnerability of Natural Language Processing (NLP) and Vision-Language Models (VLMs) to backdoor attacks, posing significant security risks. Understanding these attack strategies is crucial for assessing model robustness and developing effective defenses. This thesis proposal aims to investigate the vulnerability of language and vision-language models, analyze abnormal behaviors in backdoor-attacked models, and develop defense methods to enhance safety of modern machine learning models at deployment.


We investigate the internal mechanisms of backdoored NLP models, identifying a distinct attention focus drifting phenomenon, where trigger tokens hijack attention regardless of the input context. Through comprehensive qualitative and quantitative analysis, we provide insights into the underlying mechanisms that enable backdoor attacks. Building on these insights, we propose detection methods to differentiate backdoored models from clean ones, through inspecting both the attention distribution and the model predictions. To better understand the vulnerability, we develop advanced backdoor attack strategies targeting language models in classification tasks. For BERT variants, we introduce Trojan Attention Loss (TAL), a novel method that directly manipulates attention patterns to enhance backdoor effectiveness, ensuring stealth and robustness. Vision-Language Models have demonstrated strong performance in recent years. Yet their vulnerability is largely underexplored. We investigate advanced backdoor attack strategies on Vision-Language Models, focusing on image-to-text generation tasks. We demonstrate how backdoors can be embedded in complex multimodal tasks while maintaining semantic integrity under poisoned inputs. Additionally, we propose innovative techniques for injecting backdoors without requiring access to the original training data, expanding the feasibility of real-world attacks.

This proposal provides novel insights into the internal mechanisms of backdoored models, propose effective detection strategies, and develop advanced attack techniques that expose critical vulnerabilities. These findings underscore the urgent need for robust security measures to defend against emerging backdoor threats in deep learning models. The results have been published in top venues including ICLR, ECCV, NAACL, EMNLP, etc.

Speaker: Weimin Lyu


Zoom link: https://stonybrook.zoom.us/j/99880605139?pwd=cfWbRG6n9v3GXEa7OqvXa5cOp5eLBv.1
Meeting ID: 998 8060 5139
Passcode: 843302

Abstract: Autonomous systems, whether on Earth or in space, rely on 3D perception to understand and interact with the world around them. Yet traditional techniques for 3D understanding often depend on human designed features, fixed sensors, and conventional imaging modalities. This constrained approach can limit every stage of perception, from sensing to interpretation to decision making.
In this talk, we'll explore an alternative paradigm for imaging: physically based neural representations for 3D scenes and 3D sensing systems. We will discuss how recent advances in large scale learned representations can be used to jointly optimize both 3D scene models and the design of sensing systems for 3D capture, with the goal of enabling task specific perception systems.
Unlike modern AI models trained on internet scale datasets, these specialized 3D representations typically operate in data sparse regimes and therefore require a different kind of prior. We'll examine how grounding these learned representations in the physics of light transport can improve our understanding of scene structure, and inform imaging system design even with limited data. By connecting physical insights with learned representations, we'll highlight new possibilities for robust, efficient, and adaptive perception in challenging environments.

Speaker: Nikhil Behari is a graduate student in the Camera Culture group at the MIT Media Lab, advised by Professor Ramesh Raskar. His research interests include computational imaging, 3D scene understanding, and multi-agent decision-making under uncertainty, with a focus on automating imaging system design for 3D perception in human and planetary health. His research is supported by the NASA Space Technology Graduate Research Fellowship. He received his bachelor's in Computer Science and Statistics from Harvard University in 2022.

Join C4E for an interactive workshop with Dr. Raju Rajan, founder of ReWild Long Island, exploring how artificial intelligence can support community-led land stewardship.

ReWild Long Island advances hands-on ecological restoration across Long Island through native planting, organic land care, water protection, composting, and waste reduction.

Date & Time: Wednesday, November 4 | 12:30 PM - 1:50 PM

Location: Wang Center, Room 301

Three-Part Agenda Breakdown:

1. The Vision: ReWild's AI Tool
2. The Community: Shaping the Tool Together
   
3. The Standards: Applying Responsible AI Frameworks

Speaker: Raju Rajan is a technologist with a strong communitarian ethic. Since obtaining a PhD in Communication Networks at the University of Wisconsin, Madison, he has enjoyed a long career as researcher and consultant. His entrepreneurial acumen has led him to found two IT start-ups as well as a number of community groups aimed at organizing people for systemic change.
Raju is one of the founders of ReWild Long Island and currently serves as Board President. He is works with Long Island Together, a progressive collective that organizes around education, immigration and social change.
An avid gardener and amateur tree-enthusiast, Raju also owns and manages Sharadavanam, a small agroforest farm in Southern India dedicated to rewilding and permaculture.
Raju resides in Port Washington, NY with wife Sonia Arora, an educator & community organizer, and son Kabeera Singh, in college.

Please register using this form to join the discussion.

Moderated by Katherine Lucente, BS in Nursing Candidate at Stony Brook University

• Lav Varshney, PhD, MS, BS - Director of the AI Institute at Stony Brook University
• Meghan Reading Turchioe, PhD, MPH, RN, FAHA - Nurse Scientist at Columbia University
• Briana Shiri Last, PhD - Department of Psychology at Stony Brook University

This event is proudly sponsored by the Stony Brook University School of Nursing, the Stony Brook University Hospital Division of Nursing, and the Sigma Theta Tau Kappa Gamma Chapter.

Join Zoom Meeting
https://stonybrook.zoom.us/j/91945227869?pwd=emhoZDFWVTV0MVdPWW5uVk43MjQzUT09

Meeting ID: 919 4522 7869
Passcode: 452304
One tap mobile
+16468769923,,91945227869# US (New York)
+13126266799,,91945227869# US (Chicago)

Dial by your location
        +1 646 876 9923 US (New York)
        +1 312 626 6799 US (Chicago)
        +1 301 715 8592 US (Germantown)
        +1 669 900 6833 US (San Jose)
        +1 253 215 8782 US (Tacoma)
        +1 346 248 7799 US (Houston)
        +1 408 638 0968 US (San Jose)
Meeting ID: 919 4522 7869
Find your local number: https://stonybrook.zoom.us/u/aCvAYWkRg  

Join us as we celebrate this year's Brook & Beyond Challenge finalists.
The Office for Research and Innovation invites you to hear about the two-month journey in which the Brook & Beyond team supported eight cohorts in bringing their bold ideas from the lab to the marketplace. It's an energizing evening that highlights the collaboration, creativity, and entrepreneurial spirit driving discovery across the University.
Meet this year's award recipients, hear pitches from the emerging founders, and applaud their achievements.
Connect, celebrate, and be part of the momentum shaping the future of innovation at
Stony Brook University.
Refreshments will be served. Registration is required.
Register Here.

Join librarian Christine Fena for an interactive workshop that invites you to explore AI tools firsthand, not just as users, but as critical investigators. Through playful experimentation and collaborative discovery, you'll uncover inherent biases, probe algorithmic flaws, and gain a deeper understanding of AI's limitations and societal impacts.

Register for the Zoom workshop here.

Abstract: Large language models (LLMs) may exhibit unintended or undesirable behaviors. Recent works have concentrated on aligning LLMs to mitigate harmful outputs. Despite these efforts, some anomalies indicate that even a well-conducted alignment process can be easily circumvented, whether intentionally or accidentally. Does alignment fine-tuning yield have robust effects on models, or are its impacts merely superficial? In this work, we make the first exploration of this phenomenon from both theoretical and empirical perspectives. Empirically, we demonstrate the elasticity of post-alignment models, i.e., the tendency to revert to the behavior distribution formed during the pre-training phase upon further fine-tuning. Leveraging compression theory, we formally deduce that fine-tuning disproportionately undermines alignment relative to pre-training, potentially by orders of magnitude. We validate the presence of elasticity through experiments on models of varying types and scales. Specifically, we find that model performance declines rapidly before reverting to the pre-training distribution, after which the rate of decline drops significantly. Furthermore, we further reveal that elasticity positively correlates with the increased model size and the expansion of pre-training data. Our findings underscore the need to address the inherent elasticity of LLMs to mitigate their resistance to alignment.

Speaker: Huajian Zhang

Location: CS2311