AI Innovation Institute

The overall purpose of this seminar is to bring together people with interests in Computer Vision theory and techniques and to examine current research issues. This course will be appropriate for people who already took a Computer Vision graduate course or already had research experience in Computer Vision. To enroll in this course, you must either: (1) be in the PhD program or (2) receive permission from the instructors.

Each seminar will consist of multiple short talks (around 10 minutes) by multiple people. Students can register for 1 credit for CSE 656. Registered students must attend and present a minimum of 2 or 3 talks. Everyone else is welcome to attend. Fill in https://forms.gle/pCVXovgfMfQwGqG38 to subscribe to our mailing list for further announcement.

Abstract: Foundation models brought a paradigm shift on representation learning and the deep learning community. In my talk, I will examine the role of foundation models in medical imaging, focusing on their potential to unify diverse tasks through large-scale, generalist architectures. While these models achieve strong performance, their deployment in healthcare raises challenges related to data limitations, privacy, validation, and trust. We will also discuss domain-specific models for imaging, along with efficient adaptation techniques to adapt such models on domains that they have not been trained on. The presentation will also address key issues of reliability and interpretability, highlighting approaches like conformal prediction and counterfactual intervention to improve uncertainty estimation and model transparency. Overall, the talk will emphasize that despite their promise, foundation models require robust evaluation and trustworthy design to ensure safe and effective use in clinical settings.

Speaker: Maria Vakalopoulou is an assistant professor (MCF) in applied mathematics at CentraleSupelec, University Paris Saclay in France and the group leader of the biomathematics group of MICS Laboratory focusing on mathematical modeling in Life Sciences. She is affliated with Inria Saclay in France and Archimedes Unit in Greece. Her main research interest include the development of computational methods for image perception focusing on earth observation and medical applications. Before that, she was a postdoctoral student at CentraleSupelec, where she worked with Nikos Paragios. She completed her PhD at the Remote Sensing Laboratory at the School of Rural, Surveying and Geo-Informatics Engineering of the National Technical University of Athens under the supervision of Konstantinos Karantzalos.

Location: NCS 220

Abstract:
Quantum Machine Learning (QML) holds significant promise for solving computational challenges across diverse domains. However, its practical deployment is constrained by the limitations of noisy intermediate-scale quantum (NISQ) devices, including noise, limited scalability, and trainability issues in variational quantum circuits (VQCs). We introduce the multi-chip ensemble VQC framework, which partitions high-dimensional computations across smaller quantum chips to enhance scalability, trainability, and noise resilience. We show that this approach mitigates barren plateaus, reduces quantum error bias and variance, and maintains robust generalization through controlled entanglement. Designed to align with current and emerging quantum hardware, the framework demonstrates strong potential for enabling scalable QML on near-term devices, as validated by experiments on standard benchmark datasets (MNIST, FashionMNIST, CIFAR-10).

IACS Student Seminar Speaker:
Junghoon Park, Seoul National University
BA in Economics, Seoul National University, Korea
PhD Candidate for Interdisciplinary Programme in Artificial Intelligence at Seoul National University
Visiting Researcher at Brookhaven National Laboratory


Current Research Interests
Quantum Machine Learning


Recent Papers
Park, J., Cha, J., Chen, S. Y.-C., Yoo, S., & Tseng, H.-H. (2025). Addressing the Current Challenges of Quantum Machine Learning through Multi-Chip Ensembles. In Review at ICML.
Park, J., Kim, K., & Cha, J. (2025). How to Assess AI Ethics: Suggestions for Ethical Rating Agencies. In Review at IJCAI.
Park, J., Cha, J., Chen, S. Y.-C., Yoo, S., & Tseng, H.-H. (2024, 15-20 Sept.). Over the Quantum Rainbow: Explaining Hybrid Quantum Reinforcement Learning. 2024 IEEE International Conference on Quantum Computing and Engineering (QCE).
Park, J., Lee, E., Cho, G., Hwang, H., Kim, B.-G., Kim, G., Joo, Y. Y., & Cha, J. (2024). Gene-Environment Pathways to Cognitive Intelligence and Psychotic-Like Experiences in Children. eLife, 12, RP88117. DOI:10.7554/eLife.88117

This seminar will be held in person (food provided!) in the IACS Seminar Room, and online (zoom link below!)
https://stonybrook.zoom.us/j/96548538719?pwd=jBmI43H68q2UkdcRRjVbTkgrC6F942.1
Meeting ID: 965 4853 8719
Passcode: 493290

The Forty-Second International Conference on Machine Learning will take place at the Vancouver Convention Center from July 13th to July 19th. Register here.

Abstract:

Recent advances in deep learning have significantly enhanced the capabilities of Natural Language Processing (NLP) and Vision-Language Models (VLMs). However, these advancements come with increased vulnerabilities, notably through backdoor attacks that pose severe security threats. This thesis addresses two critical dimensions of Trustworthy AI and Efficient Multimodal Representation Learning: (1) security through analyzing, detecting, and designing backdoor attacks in NLP and VLMs, and (2) efficiency through advanced multimodal representation methods tailored for clinical and medical imaging applications.

In the first dimension, we explore the internal mechanisms exploited by backdoor attacks, identifying the distinctive phenomenon of attention focus drifting in compromised transformer models, where trigger tokens consistently hijack attention. Leveraging these insights, we propose robust detection frameworks, including the attention-based Trojan detector (AttenTD) and a task-agnostic logit-based detection method (TABDet), achieving effective identification of backdoored NLP models across diverse tasks. We further introduce novel backdoor attack methodologies: the Trojan Attention Loss (TAL), enhancing attack efficiency and stealth through direct attention manipulation, and BadCLM, demonstrating critical vulnerabilities in clinical decision-support systems by effectively compromising clinical language models.

Extending our security exploration to multimodal settings, we investigate backdoor attacks on Vision-Language Models (VLMs), particularly in complex image-to-text generation tasks, proposing innovative techniques (TrojVLM, VLOOD) capable of embedding backdoors without direct access to original training data, thus showcasing practical risks in real-world scenarios.

In the second dimension, we address efficiency and interpretability challenges in clinical and pathology applications. We introduce TCP-LLaVA, the first multimodal large language model (MLLM) designed explicitly for Whole Slide Image (WSI) Visual Question Answering (VQA). Utilizing a novel token compression mechanism inspired by transformer-based models, TCP-LLaVA substantially reduces computational resource consumption while maintaining superior VQA performance across multiple tumor subtypes. Additionally, we present a multimodal transformer model integrating structured Electronic Health Records (EHR) with clinical notes, demonstrating enhanced predictive accuracy and interpretability for in-hospital mortality prediction through integrated gradient-based interpretability methods.

Together, these contributions present a comprehensive approach to ensuring AI models are not only secure against malicious manipulation but also efficient and interpretable for critical clinical applications, underscoring the essential need for trustworthy and effective AI systems.

Speaker: Weimin Lyu

Zoom: https://stonybrook.zoom.us/j/2392326575?pwd=SVQ2VkFXTnZZYmJUMXgvTXBuZWM3UT09

Meeting ID: 239 232 6575
Passcode: 436192

Please join us at our BMI Grand Round Seminar, Spring 2022 on Topology-Based Graph Learning given by Dr. Bastian Alexander Rieck, Principal Investigator of the AIDOS Lab, The Institute of AI for Health and the Helmholtz Pioneer Campus, Munich on April 13th, 2022. More details are available on the flyer. Flyer: https://bmi.stonybrookmedicine.edu/sites/default/files/BMI_GrandRounds_Flyer-DrRieck.pdf

talk at 4:00 in 214

Virtual Job Fair for New Stony Brook Graduates & Experienced Alumni Using a platform called Career Fair Plus, participants will be able to schedule 10-minute video meetings with participating employers of interest to them. Recent graduates and alumni can register and learn more about how the fair will be run by registering on Handshake.