https://stonybrook.zoom.us/j/99820812332?pwd=c05BSTVLNmw3L04yZjdEcG5pem1OZz09 Speaker: Alexei Koulakov of Cold Spring Harbor Laboratory Brain evolution as a machine learning problem We have entered a golden age of artificial intelligence research, driven mainly by the advances in ANNs over the last decade or so. Applications of these techniques--to machine vision, speech recognition, autonomous vehicles, machine translation and many other domains--are coming so quickly that many observers predict that the long-elusive goal of Artificial General Intelligence (AGI) is within our grasp. However, we still cannot build a machine capable of building a nest, stalking prey, or loading a dishwasher. I will describe several projects, ranging from theories of evolution of neural development to the perception of smells, in which we are attempting to understand the algorithms that the nervous system is using to solve some of these challenging problems.
Abstract: Astronomers slowly made sense of the cosmos by following the stars night after night. I suggest we examine human identity in a similar way. Let's observe the words individuals use to describe themselves day after day. In this presentation, I will introduce ipseology - a new approach to studying human selves. Ipseology is the systematic, empirical study of ipseity: selfhood, individuality and the elements of identity. The primary idea is that we can learn a lot about people from their self-authored self-descriptions - especially if we follow their revisions over time. I will discuss results from sampling millions of social media bios over more than a decade and present new approaches for observation in the Post-API age.

Bio: Dr. Jason Jeffrey Jones is a computational social scientist whose expertise includes online experiments, social networks, high-throughput text analysis and machine learning. He is interested in humans' perceptions of themselves and the developing role of artificial intelligence in society.

Dr. Jones is the director of CSSERG (pronounced sea surge): the Computational Social Science of Emerging Realities Group. CSSERG is a team of scholars committed to cross-disciplinary collaboration, united by common computational methodologies and always with eyes on the near future. CSSERG has studied the effectiveness of virtual reality in evoking empathy, the dynamics of gender stereotypes in language over decades and temporal trends in personally expressed identity.

This seminar will take place in person and online (zoom link below):

Join Zoom Meeting
https://stonybrook.zoom.us/j/93686609778?pwd=KdHVyIbU3ymML6hTchXsm6JLYKLSru.1

Meeting ID: 936 8660 9778
Passcode: 638699
Join the Department of Computer Science as we welcome Lyle Ungar, University of Pennsylvania, who will be delivering a lecture on 'Measuring Cultural Variation using Natural Language Processing.' When: 11/08/24 @ 2:30 PM Where: New Computer Science Building, Room 120. Reception to follow. Abstract: Cultures vary widely in how they view the world, for example being more individualist or collectivist. Such cultural differences are, of course, reflected in the words that people use. We first show a variety of ways in which multilingual language models are not multicultural; they speak Hindi or Mandarin, but still think like Americans. In contrast, we then present a scalable method that uses embedding-derived lexica to successfully measure regional variation in culture. Bio: Lyle Ungar is a Professor of Computer and Information Science at the University of Pennsylvania, where he also holds secondary appointments in Psychology, Bioengineering, Genomics and Computational Biology, and Operations, Information and Decisions. His group uses natural language processing and explainable AI for psychological research, including analyzing social media and cell phone sensor data to better understand the drivers of physical and mental well-being. They are currently building socio-emotionally sensitive GPT-based tutors and coaches.
The Natural Language Processing Reading Group at Stony Brook University meets weekly to discuss recent research papers in NLP and related fields.
Join the Google Group here.

Memorization Dynamics in Knowledge Distillation for Language Models

Abstract: Knowledge Distillation (KD) is increasingly adopted to transfer capabilities from large language models to smaller ones, offering significant improvements in efficiency and utility while often surpassing standard fine-tuning. Beyond performance, KD is also explored as a privacy-preserving mechanism to mitigate the risk of training data leakage. While training data memorization has been extensively studied in standard pre-training and fine-tuning settings, its dynamics in a knowledge distillation setup remain poorly understood. In this work, we study memorization across the KD pipeline using three large language model (LLM) families (Pythia, OLMo-2, Qwen-3) and three datasets (FineWeb, Wikitext, Nemotron-CC-v2). We find: (1) distilled models memorize significantly less training data than standard fine-tuning (reducing memorization by more than 50%); (2) some examples are inherently easier to memorize and account for a large fraction of memorization during distillation (over ~95%); (3) student memorization is predictable prior to distillation using features based on zlib entropy, KL divergence, and perplexity; and (4) while soft and hard distillation have similar overall memorization rates, hard distillation poses a greater risk: it inherits 2.7 times more teacher-specific examples than soft distillation. Overall, we demonstrate that distillation can provide both improved generalization and reduced memorization risks compared to standard fine-tuning.

Location: NCS 220

Zoom Link: https://stonybrook.zoom.us/j/92942833294?pwd=927683
The Stony Brook Computing Society presents an exciting event featuring experts from Google (Danny Rosen - Technical Program Manager) and NVIDIA (Veer Mehta - Senior Solutions Architect), diving into the latest developments in generative AI. Learn how these industry leaders are shaping the future of technology and explore new ideas in a relaxed, engaging setting.

📍 Location: Frey 102
📅 Date: Monday, Nov 11
⏰ Time: 12 PM - 1:50 PM

Scan the QR code or register in the link.
Abstract: Recent progress in large language and vision models demonstrates how far we can go by scaling with vast internet-scale data. In contrast, physical AI, agents that perceive and act in the real world, still lags far behind. Today, both academia and industry primarily pursue generalizable physical AI by scaling up: collecting large-scale action-video datasets or training world models that enable interaction through learned environments. However, this paradigm is inherently inefficient and will soon reach a data ceiling. In this talk, I argue for a shift from scaling up to scaling out. I introduce reality world simulators, a new paradigm that converts real-world videos into diverse, interactive simulation environments. Instead of relying on more data collection, this approach expands data through structured reconstruction and recomposition, enabling both higher data efficiency and physically grounded interaction. I will present a three-pronged approach: 1) Scaling out via Digital Twins: reconstructing controllable, interactive environments from monocular videos to support diverse agent exploration. 2) Scaling out via Digital Cousins: disentangling scene structure into compositional elements to generate large-scale variations of real-world environments. 3) Scaling out via Embodied Humans: incorporating realistic human dynamics to improve safety and social compliance in robot learning. Finally, I will outline a roadmap toward building generalizable and safe physical AI systems for open-world deployment.

Bio: Dr. Wayne Wu is a postdoctoral researcher at UCLA Computer Science, working closely with Bolei Zhou, and collaborating with Trevor Darrell (UC Berkeley EECS) and Jiaqi Ma (UCLA CEE). He received his Ph.D. in Computer Science and Technology from Tsinghua University in June 2022 and was previously a visiting Ph.D. student at Nanyang Technological University. He also spent seven years in industry, where he led the research and development of products that reached more than 10 million end users worldwide. His research lies at the intersection of computer vision, robotics, and computer graphics. He focuses on developing infrastructure and methods to scale physical AI, enabling robots to work reliably and safely in the open world. He has published over 50 papers at top-tier venues including CVPR, ICCV, ICLR, NeurIPS, and ICRA, with over 9,500 citations and 10,000 GitHub stars. His work has received a CVPR Best Paper Candidate and multiple Oral, Spotlight, and Highlight presentations. He was also honored with the 2025 UCLA Chancellor's Award for Postdoctoral Research, recognizing the best postdocs at UCLA, and he was the only awardee from the School of Engineering. He serves as an Area Chair at CVPR 2026.

Location: NCS 120
Abstract: Large language models (LLMs) may exhibit unintended or undesirable behaviors. Recent works have concentrated on aligning LLMs to mitigate harmful outputs. Despite these efforts, some anomalies indicate that even a well-conducted alignment process can be easily circumvented, whether intentionally or accidentally. Does alignment fine-tuning yield have robust effects on models, or are its impacts merely superficial? In this work, we make the first exploration of this phenomenon from both theoretical and empirical perspectives. Empirically, we demonstrate the elasticity of post-alignment models, i.e., the tendency to revert to the behavior distribution formed during the pre-training phase upon further fine-tuning. Leveraging compression theory, we formally deduce that fine-tuning disproportionately undermines alignment relative to pre-training, potentially by orders of magnitude. We validate the presence of elasticity through experiments on models of varying types and scales. Specifically, we find that model performance declines rapidly before reverting to the pre-training distribution, after which the rate of decline drops significantly. Furthermore, we further reveal that elasticity positively correlates with the increased model size and the expansion of pre-training data. Our findings underscore the need to address the inherent elasticity of LLMs to mitigate their resistance to alignment.

Speaker: Huajian Zhang

Location: CS2311

Abstract: Traditional questionnaires remain the primary method for assessing psychological outcomes and beliefs, capturing individuals' and populations' inner states. This dissertation presents an alternative computational method that overcomes key limitations in current mental health monitoring, particularly in spatiotemporal resolution, responses to major events, and automatic belief identification. By analyzing ∼1 billion Tweets from 2 million geo-located users, we created a big data pipeline for estimating depression and anxiety at the county-week level. These Language-Based Mental Health Assessments (LBMHA) demonstrated higher reliability and validity than traditional survey measures. Our approach effectively captured mental health trends and highlighted significant increases in mental illness following major events. Using the LBMHA pipeline, we conducted quasi-experiments, research designs that simulate randomized control trials, to generate explanations for mental health changes due to COVID-19 incidence/death. Utilizing these time-series analyses, we conducted discontinuity forecasting for community-specific anxiety shifts using statistical learning via ensemble and contextual models. To likewise investigate individual internal states, we created a novel task and annotated dataset for self belief language identification. Our fine-tuned language model for self-belief classification, despite its relatively small scale, outperformed GPT-4o. The self belief topics identified by our model successfully predicted depression, anxiety, and stress, offering insights into the relationship between self-conceptualization and mental health. The adoption of scalable language-based assessments with modern distributed computation presents a promising avenue for advancing community and individual mental health research.

Speaker: Siddharth Mangalik

https://stonybrook.zoom.us/j/91251321639?pwd=faggV5jZ7ByFDCFmnLXD3HiYxjQ1Eb.1&jst=2