DeepMath Conference on the Mathematical Theory of Deep Neural Networks
Recent advances in deep neural networks (DNNs), combined with open, easily-accessible implementations, have made DNNs a powerful, versatile method used widely in both machine learning and neuroscience. These advances in practical results, however, have far outpaced a formal understanding of these networks and their training. The dearth of rigorous analysis for these techniques limits their usefulness in addressing scientific questions and, more broadly, hinders systematic design of the next generation of networks. Recently, long-past-due theoretical results have begun to emerge from researchers in a number of fields. The purpose of this conference is to give visibility to these results, and those that will follow in their wake, to shed light on the properties of large, adaptive, distributed learning architectures, and to revolutionize our understanding of these systems.
| Abstract: Astronomers slowly made sense of the cosmos by following the stars night after night. I suggest we examine human identity in a similar way. Let's observe the words individuals use to describe themselves day after day. In this presentation, I will introduce ipseology - a new approach to studying human selves. Ipseology is the systematic, empirical study of ipseity: selfhood, individuality and the elements of identity. The primary idea is that we can learn a lot about people from their self-authored self-descriptions - especially if we follow their revisions over time. I will discuss results from sampling millions of social media bios over more than a decade and present new approaches for observation in the Post-API age. Bio: Dr. Jason Jeffrey Jones is a computational social scientist whose expertise includes online experiments, social networks, high-throughput text analysis and machine learning. He is interested in humans' perceptions of themselves and the developing role of artificial intelligence in society. Dr. Jones is the director of CSSERG (pronounced sea surge): the Computational Social Science of Emerging Realities Group. CSSERG is a team of scholars committed to cross-disciplinary collaboration, united by common computational methodologies and always with eyes on the near future. CSSERG has studied the effectiveness of virtual reality in evoking empathy, the dynamics of gender stereotypes in language over decades and temporal trends in personally expressed identity. This seminar will take place in person and online (zoom link below): Join Zoom Meeting https://stonybrook.zoom.us/j/ Meeting ID: 936 8660 9778 Passcode: 638699 |
Abstract: Large language models (LLMs) may exhibit unintended or undesirable behaviors. Recent works have concentrated on aligning LLMs to mitigate harmful outputs. Despite these efforts, some anomalies indicate that even a well-conducted alignment process can be easily circumvented, whether intentionally or accidentally. Does alignment fine-tuning yield have robust effects on models, or are its impacts merely superficial? In this work, we make the first exploration of this phenomenon from both theoretical and empirical perspectives. Empirically, we demonstrate the elasticity of post-alignment models, i.e., the tendency to revert to the behavior distribution formed during the pre-training phase upon further fine-tuning. Leveraging compression theory, we formally deduce that fine-tuning disproportionately undermines alignment relative to pre-training, potentially by orders of magnitude. We validate the presence of elasticity through experiments on models of varying types and scales. Specifically, we find that model performance declines rapidly before reverting to the pre-training distribution, after which the rate of decline drops significantly. Furthermore, we further reveal that elasticity positively correlates with the increased model size and the expansion of pre-training data. Our findings underscore the need to address the inherent elasticity of LLMs to mitigate their resistance to alignment.
Speaker: Huajian Zhang
Location: CS2311
Speaker: Huajian Zhang
Location: CS2311
Subject: RADIOLOGY GRAND ROUNDS CT Colonography: An Effective Test for Colorectal Cancer Screening- Judy Yee, M.D.
When: Wednesday, May 12, 2021 12:00 PM-1:00 PM (UTC-05:00) Eastern Time (US & Canada).
Where: JOIN ZOOM MEETING
Judy Yee, MD
Chair, Department of Radiology
Professor, Department of Radiology
Abdominal Imaging
Join Zoom Meeting
https://einsteinmed.zoom.us/j/
Meeting ID: 977 8219 0723
Passcode: 101083
https://stonybrook.zoom.us/j/
ID: 94414957054
Password: 094758
Speaker: Heather J. Lynch
Bio: Dr. Heather J. Lynch is an Associate Professor of Ecology & Evolution at Stony Brook University. Prior to Stony Brook, Dr. Lynch was an Adjunct Professor of Applied Math and Statistics at UC Santa Cruz and a Research Scientist in the Biology Department at the University Maryland. Dr. Lynch received her A.B. in Physics from Princeton University in 2000, an A.M. in Physics from Harvard University in 2004, and a Ph.D. in Organismal and Evolutionary Biology from Harvard University in 2006. Dr. Lynch's research is focused on spatial population dynamics of Antarctic penguins, with a particular focus on statistical and mathematical models to integrate patchy time series with remote sensing imagery. These data will allow Dr. Lynch and colleagues to develop mathematical models to explore how coloniality constrains the colonization and extinction of individual habitat patches and, ultimately, the metapopulation dynamics of colonial seabirds.
18th Annual Engineering Ball
Flowerfield, St. James, NY
Thursday April, 2nd, 7:00 to 10:00 pm
Pick up your tickets in 231 Engineering (Monday - Friday, 10:00 am to 4 pm)
Presenting Partner: L3Harris
The Natural Language Processing Reading Group at Stony Brook University meets weekly to discuss recent research papers in NLP and related fields.
Join the Google Group here.
Why LLM Safety Guardrails Collapse After Fine-tuning: A Similarity Analysis Between Alignment and Fine-tuning Datasets
Abstract: Recent advancements in large language models (LLMs) have underscored their vulnerability to safety alignment jailbreaks, particularly when subjected to downstream fine-tuning. However, existing mitigation strategies primarily focus on reactively addressing jailbreak incidents after safety guardrails have been compromised, removing harmful gradients during fine-tuning, or continuously reinforcing safety alignment throughout fine-tuning. As such, they tend to overlook a critical upstream factor: the role of the original safety-alignment data. This paper therefore investigates the degradation of safety guardrails through the lens of representation similarity between upstream alignment datasets and downstream fine-tuning tasks. Our experiments demonstrate that high similarity between these datasets significantly weakens safety guardrails, making models more susceptible to jailbreaks. Conversely, low similarity between these two types of datasets yields substantially more robust models and thus reduces harmfulness score by up to 10.33%. By highlighting the importance of upstream dataset design in the building of durable safety guardrails and reducing real-world vulnerability to jailbreak attacks, these findings offer actionable insights for fine-tuning service providers to prioritize upstream models with low jailbreak risk.
Location: NCS 220
Zoom Link: https://stonybrook.zoom.us/j/92942833294?pwd=927683
Join the Google Group here.
Why LLM Safety Guardrails Collapse After Fine-tuning: A Similarity Analysis Between Alignment and Fine-tuning Datasets
Abstract: Recent advancements in large language models (LLMs) have underscored their vulnerability to safety alignment jailbreaks, particularly when subjected to downstream fine-tuning. However, existing mitigation strategies primarily focus on reactively addressing jailbreak incidents after safety guardrails have been compromised, removing harmful gradients during fine-tuning, or continuously reinforcing safety alignment throughout fine-tuning. As such, they tend to overlook a critical upstream factor: the role of the original safety-alignment data. This paper therefore investigates the degradation of safety guardrails through the lens of representation similarity between upstream alignment datasets and downstream fine-tuning tasks. Our experiments demonstrate that high similarity between these datasets significantly weakens safety guardrails, making models more susceptible to jailbreaks. Conversely, low similarity between these two types of datasets yields substantially more robust models and thus reduces harmfulness score by up to 10.33%. By highlighting the importance of upstream dataset design in the building of durable safety guardrails and reducing real-world vulnerability to jailbreak attacks, these findings offer actionable insights for fine-tuning service providers to prioritize upstream models with low jailbreak risk.
Location: NCS 220
Zoom Link: https://stonybrook.zoom.us/j/92942833294?pwd=927683
Abstract: How is AI transforming pricing decisions, and what opportunities, challenges, and risks does this transformation bring? This book traces the evolution of pricing from traditional approaches to today's AI-powered strategies, showing how big data, machine learning, and generative AI are reshaping the way organizations set prices. Pricing decisions can now be adjusted in real time, personalized to individual consumers, and optimized at an unprecedented scale. The book examines the behavioral principles underlying modern pricing, explores how AI enables managers to automate and enhance pricing decisions, and discusses the ethical, legal, and societal challenges raised by these innovations, including concerns about competition, consumer protection, and discrimination. Ultimately, the book argues that AI is not merely changing how prices are set--it is redefining the role of pricing itself, requiring organizations to balance profitability, fairness, and consumer trust.
Speaker Bio: Maxime Cohen is the Scale AI Chair Professor and the Academic Director of the Bensadoun School of Retail Management at McGill University. He is the Chief of AI Strategy of the CIUSSS West-Central Montreal and the Scientific Director of the non-profit MyOpenCourt.org. He was a Visiting Professor of Operations Management and a Shubik Fellow at Yale School of Management. He has collaborated with many companies including Google, Microsoft, Meta, Uber, Waze, Amazon, Expedia, Oracle Retail, IBM Research, Via, Spotify, Aldo Group, Couche- Tard/Circle K, Loblaws, Canadian Tire, L'Oréal, and Staples and he serves on the advisory board of several startups. His research and teaching have received 40+ awards, including Poets&Quants Best 40-Under-40 MBA Professors, RETHINK Retail's Top Retail Influencers, MSOM Young Scholar Prize, and Best OM Paper in Management Science. Finally, he serves on the editorial board of several leading Operations journals (DE at M&SOM and POM, AE at Management Science, OR, and MBR).
Location: Harriman Hall 304
Speaker Bio: Maxime Cohen is the Scale AI Chair Professor and the Academic Director of the Bensadoun School of Retail Management at McGill University. He is the Chief of AI Strategy of the CIUSSS West-Central Montreal and the Scientific Director of the non-profit MyOpenCourt.org. He was a Visiting Professor of Operations Management and a Shubik Fellow at Yale School of Management. He has collaborated with many companies including Google, Microsoft, Meta, Uber, Waze, Amazon, Expedia, Oracle Retail, IBM Research, Via, Spotify, Aldo Group, Couche- Tard/Circle K, Loblaws, Canadian Tire, L'Oréal, and Staples and he serves on the advisory board of several startups. His research and teaching have received 40+ awards, including Poets&Quants Best 40-Under-40 MBA Professors, RETHINK Retail's Top Retail Influencers, MSOM Young Scholar Prize, and Best OM Paper in Management Science. Finally, he serves on the editorial board of several leading Operations journals (DE at M&SOM and POM, AE at Management Science, OR, and MBR).
Location: Harriman Hall 304
Topic: Exotic Neural Networks
Time: Mar 10, 2021 08:00 PM Eastern Time (US and Canada)
Join Zoom Meeting
https://stonybrook.zoom.us/j/
Meeting ID: 913 8893 8500
Passcode: 501725
Abstract: Generative visual models like Stable Diffusion and Sora generate photorealistic images and videos that are nearly indistinguishable from real ones to a naive observer. However, their grasp of the physical world remains an open question: Do they understand 3D geometry, light, and object interactions, or are they mere pixel parrots of their training data? Through systematic probing, I will demonstrate that these models surprisingly learn fundamental scene properties--intrinsic images such as surface normals, depth, albedo, and shading (à la Barrow & Tenenbaum, 1978)--without explicit supervision, which enables applications like image relighting. But I will also show that this knowledge is insufficient. Careful analysis reveals unexpected failures: inconsistent shadows, multiple vanishing points, and scenes that defy basic physics. All these findings suggest these models excel at local texture synthesis but struggle with global reasoning: a crucial gap between imitation and true understanding. I will then conclude by outlining a path toward generative world models that emulate global and counterfactual reasoning, causality, and physics.
Bio: Anand Bhattad is a Research Assistant Professor at the Toyota Technological Institute at Chicago. He earned his PhD from the University of Illinois Urbana-Champaign in 2024 under the mentorship of David Forsyth. His research interests lie at the intersection of computer vision and computer graphics, with a current focus on understanding the knowledge encoded in generative models. Anand has received Outstanding Reviewer honors at ICCV 2023 and CVPR 2021, and his CVPR 2022 paper was nominated for a Best Paper Award. He actively contributes to the research community by leading workshops at CVPR and ECCV, including Scholars and Big Models: How Can Academics Adapt? (CVPR 2023), CV 20/20: A Retrospective Vision (CVPR 2024), Knowledge in Generative Models (ECCV 2024), and How to Stand Out in the Crowd? (CVPR 2025). For more details, visit https://anandbhattad.github.