AI Innovation Institute

Communication-Efficient Heterogeneity-Aware Machine Learning System and Architecture by Xuehai Qian

ABSTRACT: The key success of deep learning is the increasing size of models that can achieve high accuracy. At the same time, it is difficult to train the complex models with large data sets. Therefore, it is crucial to accelerate training with distributed systems and architectures, where communication and heterogeneity are two key challenges. In this talk, I will present two heterogeneity-aware decentralized training protocols without communication bottleneck. Specifically, Hop supports arbitrary iteration gap between workers by novel queue-based synchronization which can tolerate heterogeneity with system techniques. Prague uses randomized communication to tolerate heterogeneity with a new training algorithm based on partial reduce -- an efficient communication primitive. If time permits, I will present the systematic tensor partitioning for training on heterogeneous accelerator arrays (e.g., GPU/TPU). We believe that our principled approaches are crucial for achieving high-performance and efficient distributed training.

BIO: Xuehai Qian is an assistant professor at University of Southern California. His research interests include domain-specific systems and architectures, performance tuning and resource management of cloud systems and parallel computer architectures. He received his PhD from the University of Illinois Urbana Champaign and was a postdoc at UC Berkeley. He is the recipient of W.J Poppelbaum Memorial Award at UIUC, NSF CRII and CAREER Award, and the inaugural ACSIC (American Chinese Scholar In Computing) Rising Star Award.

The Future of Learning: Rethinking Practice in a Changing World

Thursday, March 26, 2026 (Workshops)
Friday, March 27, 2026 (Symposium)

Open to Stony Brook University Faculty, Staff, and Graduate Students. Hosted by the Center for Excellence in Learning and Teaching, Office of the Provost.

Thursday, March 26, 2026
Workshop: AI Tools and Techniques

• Open to all faculty & staff
• Hands-on, exploratory
• Registration only limited to the size of the room
• Location: In-person, TBD
• Time: 10 AM - 12 PM
• Registration required

Friday, March 27, 2026
Keynote: Teaching and Thinking with AI

• Faculty, TAs, postdocs, and academic staff
• In-person on-campus conference venue
• Location: SAC Balroom
• Time: 9 AM - 3 PM
• Registration required

Keynote Speaker: José Antonio Bowen

José Antonio Bowen has been leading innovation and change for over 40 years at Stanford, Georgetown and the University of Southampton (UK), as a dean at Miami University and SMU and as President of Goucher College. Bowen has worked as a musician with Stan Getz, Dave Brubeck, and many others and his symphony was nominated for the Pulitzer Prize in Music (1985).
Bowen holds four degrees from Stanford and has written over 100 scholarly articles and books, including the Cambridge Companion to Conducting (2003), Teaching Naked (2012 and the winner of the Ness Award for Best Book on Higher Education), Teaching Naked Techniques with C. Edward Watson (2017) and Teaching Change: How to Develop Independent Thinkers using Relationships, Resilience and Reflection (Johns Hopkins University Press, 2021).
Bowen has appeared in The New York Times, Forbes, The Wall Street Journal, and has three TED talks. Stanford honored him as a Distinguished Alumni Scholar (2010) and he has presented keynotes and workshops at more than 300 campuses and conferences 46 states and 17 countries around the world. In 2018, he was awarded the Ernest L. Boyer Award (for significant contributions to American higher education). He is a senior fellow for the American Association of Colleges and Universities.

Register here.

Are you concerned about AI issues with your asynchronous online courses? Is your fully online course vulnerable to AI plagiarism? Do you want to engage your online students using AI? Discover the future of education with our AI-powered solutions designed specifically for online asynchronous courses. This innovative approach uses artificial intelligence to transform the way courses are delivered, making learning more personalized, engaging, and effective.

https://stonybrook.zoom.us/meeting/register/tJMvd-irqTotGtQONZqerPf_TnhXcx8t2sA1

Abstract: Computational pathology has revolutionized cancer diagnosis and research through the analysis of digitized whole slide images (WSIs). However, the giga-pixel size of these images presents profound technical challenges, creating two intertwined bottlenecks: computational inefficiency and label inefficiency. The immense data scale makes standard end-to-end (E2E) training of deep neural networks infeasible due to prohibitive GPU memory requirements, while the reliance on expert pathologists for annotations makes obtaining high-quality labeled data a tedious and expensive process. This proposal confronts these dual challenges by developing a series of novel model architectures, training paradigms, and self-supervised learning methods designed to create a more efficient and effective framework for WSI analysis.

To improve computational efficiency, this proposal first introduces a locally supervised learning paradigm that enables E2E training on entire WSIs by partitioning a network into gradient-isolated modules, circumventing the memory bottleneck of backpropagation. Second, it presents Prompt-MIL, a parameter-efficient fine-tuning framework that reduces the number of trainable parameters, memory consumption, and training time by fine-tuning only few prompts to guide large pre-trained models. Third, this work advances the efficient architecture on WSIs by developing novel State-Space Models (SSMs). It proposes 2DMamba, the first intrinsic Mamba architecture that preserves the crucial 2D spatial structure of images, overcoming the spatial discrepancy inherent in 1D models. Fourth, to address the inefficiency of multi-directional scans in Mamba models, including 2DMamba, it presents Locally Bi-directional Mamba (LBMamba), which introduces a novel, hardware-aware local backward scan that integrates bi-directional scan into a single forward pass, significantly improving throughput performance trade-off. Lastly, it proposes an extension to the LBMamba, warp-level Bi-directional Mamba (WLBMamba) that extends the thread-level bidirectional scan to warp-level bidirectional scan that further improves the throughput performance trade-off.

To improve label efficiency, this proposal proposes a Precise Location-based Matching strategy for self-supervised dense contrastive learning. By allowing a local patch in one augmented view to match multiple overlapping patches in another, creates a more accurate correspondence, leading to superior feature representations for dense prediction tasks like segmentation and detection.

In summary, this proposal presents a holistic investigation into the efficiency bottlenecks in computational pathology. Through these combined contributions in model architecture, training paradigms, and self-supervised learning, this work establishes a more scalable, efficient, and powerful computational framework for analyzing giga-pixel pathology images.

Speaker: Jingwei Zhang

Location: Old Computer Science Room 2114

Zoom: https://stonybrook.zoom.us/j/95187903649?pwd=tV0CNxLu1QKqw7hGmcE1h0rJ2C6n1b.1
Meeting ID: 951 8790 3649 | Passcode: 488916

Abstract: Recent progress in large language and vision models demonstrates how far we can go by scaling with vast internet-scale data. In contrast, physical AI, agents that perceive and act in the real world, still lags far behind. Today, both academia and industry primarily pursue generalizable physical AI by scaling up: collecting large-scale action-video datasets or training world models that enable interaction through learned environments. However, this paradigm is inherently inefficient and will soon reach a data ceiling. In this talk, I argue for a shift from scaling up to scaling out. I introduce reality world simulators, a new paradigm that converts real-world videos into diverse, interactive simulation environments. Instead of relying on more data collection, this approach expands data through structured reconstruction and recomposition, enabling both higher data efficiency and physically grounded interaction. I will present a three-pronged approach: 1) Scaling out via Digital Twins: reconstructing controllable, interactive environments from monocular videos to support diverse agent exploration. 2) Scaling out via Digital Cousins: disentangling scene structure into compositional elements to generate large-scale variations of real-world environments. 3) Scaling out via Embodied Humans: incorporating realistic human dynamics to improve safety and social compliance in robot learning. Finally, I will outline a roadmap toward building generalizable and safe physical AI systems for open-world deployment.

Bio: Dr. Wayne Wu is a postdoctoral researcher at UCLA Computer Science, working closely with Bolei Zhou, and collaborating with Trevor Darrell (UC Berkeley EECS) and Jiaqi Ma (UCLA CEE). He received his Ph.D. in Computer Science and Technology from Tsinghua University in June 2022 and was previously a visiting Ph.D. student at Nanyang Technological University. He also spent seven years in industry, where he led the research and development of products that reached more than 10 million end users worldwide. His research lies at the intersection of computer vision, robotics, and computer graphics. He focuses on developing infrastructure and methods to scale physical AI, enabling robots to work reliably and safely in the open world. He has published over 50 papers at top-tier venues including CVPR, ICCV, ICLR, NeurIPS, and ICRA, with over 9,500 citations and 10,000 GitHub stars. His work has received a CVPR Best Paper Candidate and multiple Oral, Spotlight, and Highlight presentations. He was also honored with the 2025 UCLA Chancellor's Award for Postdoctoral Research, recognizing the best postdocs at UCLA, and he was the only awardee from the School of Engineering. He serves as an Area Chair at CVPR 2026.

Location: NCS 120

EMNLP 2020 - The 2020 Conference on Empirical Methods in Natural Language Processing Online format - https://2020.emnlp.org/

Abstract: Recent studies have highlighted the vulnerability of Natural Language Processing (NLP) and Vision-Language Models (VLMs) to backdoor attacks, posing significant security risks. Understanding these attack strategies is crucial for assessing model robustness and developing effective defenses. This thesis proposal aims to investigate the vulnerability of language and vision-language models, analyze abnormal behaviors in backdoor-attacked models, and develop defense methods to enhance safety of modern machine learning models at deployment.


We investigate the internal mechanisms of backdoored NLP models, identifying a distinct attention focus drifting phenomenon, where trigger tokens hijack attention regardless of the input context. Through comprehensive qualitative and quantitative analysis, we provide insights into the underlying mechanisms that enable backdoor attacks. Building on these insights, we propose detection methods to differentiate backdoored models from clean ones, through inspecting both the attention distribution and the model predictions. To better understand the vulnerability, we develop advanced backdoor attack strategies targeting language models in classification tasks. For BERT variants, we introduce Trojan Attention Loss (TAL), a novel method that directly manipulates attention patterns to enhance backdoor effectiveness, ensuring stealth and robustness. Vision-Language Models have demonstrated strong performance in recent years. Yet their vulnerability is largely underexplored. We investigate advanced backdoor attack strategies on Vision-Language Models, focusing on image-to-text generation tasks. We demonstrate how backdoors can be embedded in complex multimodal tasks while maintaining semantic integrity under poisoned inputs. Additionally, we propose innovative techniques for injecting backdoors without requiring access to the original training data, expanding the feasibility of real-world attacks.

This proposal provides novel insights into the internal mechanisms of backdoored models, propose effective detection strategies, and develop advanced attack techniques that expose critical vulnerabilities. These findings underscore the urgent need for robust security measures to defend against emerging backdoor threats in deep learning models. The results have been published in top venues including ICLR, ECCV, NAACL, EMNLP, etc.

Speaker: Weimin Lyu


Zoom link: https://stonybrook.zoom.us/j/99880605139?pwd=cfWbRG6n9v3GXEa7OqvXa5cOp5eLBv.1
Meeting ID: 998 8060 5139
Passcode: 843302