I will first present methods for learning unifying mechanisms from heterogeneous data, across domains and modalities, to enable reliable transfer and controllable generation. Next, I will show how structured concepts can be recovered even from seemingly unstructured data, by analyzing and improving self-supervised objectives (such as masking and diffusion) through hierarchical latent-variable models. These concept structures can then be used to interpret generative models and support targeted, multi-level edits. Finally, I connect these two threads to generalization beyond the training distribution. I will discuss natural conditions for extrapolation and a compositional generation framework that improves prompt following for novel concept combinations. I will conclude with a brief outlook on self-improving world models and AI-assisted scientific discovery.
Bio: Lingjing Kong is a Ph.D. candidate in the Computer Science Department at Carnegie Mellon University. His research focuses on Causal AI for transferable, interpretable, and controllable systems, with an emphasis on understanding and exploiting the structure of real-world data to make foundation models actionable and more reliable. He develops identification principles and scalable algorithms for learning unified models from heterogeneous data, uncovering hierarchical concept structures in unstructured data (e.g., images and text), and generalizing beyond training support through compositionality and extrapolation. His work has appeared in top ML venues including ICML, NeurIPS, CVPR, ICLR, and EMNLP and has been prototyped and applied in industry.
Location: NCS 120
The Della Pietra Lecture Series is pleased to present this lecture by Scott Aaronson:
Abstract: I'll survey some areas where I think theoretical computer science, math, and statistics can potentially contribute to the urgent quest to align powerful AI with humane values. These areas include: the watermarking of AI outputs, mechanistic interpretability (including Paul Christiano's No-Coincidence Principle, and succinct digests of the training process to aid interpretability), and theoretical guarantees for out-of-distribution generalization.
Speaker: Scott Aaronson is Schlumberger Chair of Computer Science at the University of Texas at Austin, and founding director of its Quantum Information Center. He received his bachelor's from Cornell University and his PhD from UC Berkeley. Aaronson's research has focused mainly on the capabilities and limits of quantum computers. His first book, Quantum Computing Since Democritus, was published in 2013 by Cambridge University Press. He received the National Science Foundation's Alan T. Waterman Award, the United States PECASE Award, the Tomassoni-Chisesi Prize in Physics, and the ACM Prize in Computing, and is a Fellow of the ACM and the AAAS and a member of the National Academy of Sciences. He blogs at Shtetl-Optimized, https://www.scottaaronson.com/blog.
Location: Della Pietra Family Auditorium (SCGP 103)
Topic: Exotic Neural Networks
Time: Mar 10, 2021 08:00 PM Eastern Time (US and Canada)
Join Zoom Meeting
https://stonybrook.zoom.us/j/
Meeting ID: 913 8893 8500
Passcode: 501725
Speaker: Qin Ren
Location: https://stonybrook.zoom.
In this workshop, we will take a historical journey through the ideas that eventually led to today's large language models, such as ChatGPT and Claude.
Modern LLMs can seem overwhelmingly complex, but they didn't appear all at once. Many of the basic ideas behind them have been developing for more than a century. By following this history step by step, we can better understand what language models do and how they evolved.
In part 1 of this workshop, we will begin with Andrey Markov's early work on sequences and probability in 1913 and Claude Shannon's influential work on information and language in 1948. From there, we will explore key ideas such as Markov chains, N-grams, and the next-word generation process using simple, accessible examples.
In part 2 of the workshop, which is not in this workshop, we will also look at how an artificial neuron can learn to predict the next word, an important step toward neural language models that later became the foundation of modern LLMs and we will see what perplexity is.
The workshop is designed for anyone who is curious about where today's AI came from and how a century of ideas about language, probability, and computation helped make modern language models possible. This workshop targets students of any level who are interested in language models or history. No background in computer science or artificial intelligence is required. The hardest part is converting counted numbers to probabilities.
Our historical journey will cover roughly the period from 1918 to 1970s.
Location: Melville Library, Central Reading Room, Lab A
Register for the event on SB Engaged.
Searching for Common Ground in Human-AI Interaction (What Gen AI Can't Do)
Abstract: For many of us, the alien invasion of LLMs has been unsettling, at best. Who knew that monstrous quantities of text and compute power could simulate human discourse processing? But despite claims about emergent theory-of-mind in reasoning models, LLMs remain stochastic parrots: they do not collaborate, project co-presence, form conceptual pacts, accurately signal commitment to what they say, or accrue common ground. This lack of pragmatic competence is clear from our recent studies of narrative and referential communication by AI vs. humans.
Speaker: Susan Brennan is SUNY Distinguished Professor at Stony Brook University in Cognitive Science within the Department of Psychology, with affiliations in Computer Science and Linguistics. Her research uses eye-tracking, experimental, behavioral, and observational methods to study the psycholinguistics of communication (verbal, nonverbal, and multimodal), adaptive language use, and the human use of technology, including spoken dialogue systems. She currently directs the NSF-funded Research Traineeship project, Detecting and Addressing Bias in Data, Humans, and Institutions, a collaboration among PhD. programs in Cognitive Science, Social & Health Psychology, Computer Science, Applied Math & Statistics, Linguistics, Political Science, Economics, Africana Studies, and Women's Studies. This traineeship supports students' AI and bias-related convergent research projects that include assessing potential for LLVMs (Large Language-Vision Models) to act as interlocutors in spontaneous dialogue as well as collaborating with the Innocence Network about AI tools. She holds a Ph.D. in cognitive psychology from Stanford University, an M.S. in visual studies from MIT (from what is now known as the MIT Media Lab), and a B.A. from Cornell University in cultural anthropology. She has worked in industry at Atari Research, Hewlett-Packard Labs, and Apple Computer, as well as at NSF as a program officer for the Graduate Research Funding Program.
Faculty Response Panel:
- Jiwon Hwang (Panel Chair; Assistant Professor, Asian and Asian American Studies)
- Rose Tirotta-Esposito (Director, Center for Excellence in Learning and Teaching)
- Lav Varshney (Director, AI Innovation Institute)
- Josh Zhang (Associate Professor, Sociology and Institute for Advanced Computational Science)
Location: Humanities 1006
Register here via Zoom.