Abstract: Large Language Models (LLMs) have transitioned from standalone prediction interfaces into integrated systems that incorporate content protection, external knowledge retrieval, and multi-step reasoning. While these functional layers expand model capabilities, they also introduce complex, inter-component dependencies that create novel and systemic security risks. This research provides a systematic deconstruction of the structural vulnerabilities emerging across these functional layers.

In this proposal, we evaluate the security boundaries of LLM systems through three pivotal dimensions:
The Content Layer: We present Watermark under Fire, revealing the inherent fragility of content-based tracing mechanisms under adaptive perturbations and highlighting the limitations of surface-level safety measures.
The Retrieval Layer: We introduce GraphRAG under Fire to examine the security of topology-aware knowledge integration. We reveal how graph-based indexing can be exploited as a structural lever for high-success poisoning attacks.
The Reasoning Layer: We detail AutoRAN, the first framework demonstrating the hijacking of internal safety reasoning in Large Reasoning Models (LRMs). This work proves that the transparency of the reasoning process itself creates a critical and exploitable attack surface.

Collectively, these studies demonstrate a systemic failure of add-on safety mechanisms in securing the broader LLM ecosystem. By identifying recurring patterns of exploitation across different system layers, this research provides the necessary foundation for transitioning from reactive patching to a more unified and architecturally-grounded approach to AI trustworthiness.

Speaker: Jiacheng Liang

Zoom: https://stonybrook.zoom.us/j/6669990420?pwd=dkY0eEw5YXpPSWo3RUE4OE1oVW90UT09&omn=97367037382
Meeting ID: 666 999 0420
Passcode: 075299
Zoom Link: https://github.com/giorgianb/spdhackspring2021/blob/main/bit.ly/spdhack2021

ΣΦΔ Hack Spring 2021 is ΣΦΔ's first annual machine learning hackathon. ΣΦΔ Hack Spring 2021 aims to introduce Stony Brook students to the rich and challenging field of machine learning, and develop the skills necessary to build sophisticated machine learning models on their own.
 
More info here: https://github.com/giorgianb/spdhackspring2021/blob/main/README.md

You are cordially invited to attend the biweekly Brookhaven AI Mixer (BAM). BAM includes one short talk on AI research happening at BNL, followed by an open mixer over coffee and snacks for everyone to network and discuss all things AI. The first half hour will consist of presentations that will be available via ZOOM, and the second half hour will be for in person only networking.

Join us every other Tuesday at noon in CDSD's Training Room (building 725, 2nd floor) to learn about interesting AI methods and applications, engage with potential collaborators, prepare for pending FASST funding calls, and build a community of AI for Science at BNL.

AI and Edge Processing Co-Design for Radiation Detectors

Abstract: Artificial Intelligence (AI) offers exciting new opportunities for enhancing the performance of radiation detectors, ultimately leading to improved physics outcomes. Furthermore, with the explosive growth in data rates being seen by next-generation radiation detectors, deployment of AI algorithms at the edge by embedding intelligence within or near the detector front-end can be transformative. Such integration enables real-time data filtering, noise suppression, feature extraction, and adaptive control, while reducing downstream bandwidth and power consumption. This talk will cover three efforts that bring AI to the forefront of detector technology. First, we demonstrate how AI-based algorithms can be used for position reconstruction in virtual Frisch-grid (VFG) detectors by compensating for charge transport distortions and detector non- uniformities, leading to significantly enhanced fidelity in imaging of gamma-ray interactions. Second, we present a smart readout application specific integrated circuit (ASIC) that combines digital signal processing with co-designed artificial neural networks to enable on-chip regression and classification of detector signals, while meeting stringent constraints on accuracy, speed, and area. Finally, we introduce our recent efforts related to the development of electro-photonic processing architectures that integrate CMOS electronics and silicon photonics for near-sensor AI acceleration. These architectures aim to leverage cross-disciplinary co-design from algorithms to hardware, to achieve low latency and energy-efficient processing of detector data.

Biography: Dr. Prashansa Mukim is an early-career researcher in the Instrumentation Department at BNL, where she works on the design of front-end electronics for extreme environments and the development of co-design methodologies for novel processing modalities and beyond-CMOS technologies. Prior to joining BNL, she was a post-doctoral researcher at the National Institute of Standards and Technology (NIST) in Maryland, where she focused on characterizing the properties of CMOS circuits at cryogenic temperatures and applications of spintronic devices for neuromorphic computing. She received her Ph.D. in Electrical and Computer Engineering from the University of California, Santa Barbara, in 2021.

Location: CDS, Bldg. 725, Training Room

Join ZoomGov Meeting: https://bnl.zoomgov.com/j/1608585935?pwd=UemgEkqijfNf3vIJIGuOa2MdjsunaT.1

Meeting ID: 160 858 5935
Passcode: 076033

The overall purpose of this seminar is to bring together people with interests in Computer Vision theory and techniques and to examine current research issues. This course will be appropriate for people who already took a Computer Vision graduate course or already had research experience in Computer Vision. To enroll in this course, you must either: (1) be in the PhD program or (2) receive permission from the instructors.

Each seminar will consist of multiple short talks (around 10 minutes) by multiple people. Students can register for 1 credit for CSE 656. Registered students must attend and present a minimum of 2 or 3 talks. Everyone else is welcome to attend. Fill in https://forms.gle/pCVXovgfMfQwGqG38 to subscribe to our mailing list for further announcement.
Abstract:

Recent advances in deep learning have significantly enhanced the capabilities of Natural Language Processing (NLP) and Vision-Language Models (VLMs). However, these advancements come with increased vulnerabilities, notably through backdoor attacks that pose severe security threats. This thesis addresses two critical dimensions of Trustworthy AI and Efficient Multimodal Representation Learning: (1) security through analyzing, detecting, and designing backdoor attacks in NLP and VLMs, and (2) efficiency through advanced multimodal representation methods tailored for clinical and medical imaging applications.

In the first dimension, we explore the internal mechanisms exploited by backdoor attacks, identifying the distinctive phenomenon of attention focus drifting in compromised transformer models, where trigger tokens consistently hijack attention. Leveraging these insights, we propose robust detection frameworks, including the attention-based Trojan detector (AttenTD) and a task-agnostic logit-based detection method (TABDet), achieving effective identification of backdoored NLP models across diverse tasks. We further introduce novel backdoor attack methodologies: the Trojan Attention Loss (TAL), enhancing attack efficiency and stealth through direct attention manipulation, and BadCLM, demonstrating critical vulnerabilities in clinical decision-support systems by effectively compromising clinical language models.

Extending our security exploration to multimodal settings, we investigate backdoor attacks on Vision-Language Models (VLMs), particularly in complex image-to-text generation tasks, proposing innovative techniques (TrojVLM, VLOOD) capable of embedding backdoors without direct access to original training data, thus showcasing practical risks in real-world scenarios.

In the second dimension, we address efficiency and interpretability challenges in clinical and pathology applications. We introduce TCP-LLaVA, the first multimodal large language model (MLLM) designed explicitly for Whole Slide Image (WSI) Visual Question Answering (VQA). Utilizing a novel token compression mechanism inspired by transformer-based models, TCP-LLaVA substantially reduces computational resource consumption while maintaining superior VQA performance across multiple tumor subtypes. Additionally, we present a multimodal transformer model integrating structured Electronic Health Records (EHR) with clinical notes, demonstrating enhanced predictive accuracy and interpretability for in-hospital mortality prediction through integrated gradient-based interpretability methods.

Together, these contributions present a comprehensive approach to ensuring AI models are not only secure against malicious manipulation but also efficient and interpretable for critical clinical applications, underscoring the essential need for trustworthy and effective AI systems.

Speaker: Weimin Lyu

Zoom: https://stonybrook.zoom.us/j/2392326575?pwd=SVQ2VkFXTnZZYmJUMXgvTXBuZWM3UT09

Meeting ID: 239 232 6575
Passcode: 436192