Topic: AI Seminar: Owen Rambow
Time: Mar 17, 2021 10:00 AM Eastern Time (US and Canada)
Join Zoom Meeting

https://stonybrook.zoom.us/j/93614644178?pwd=MzJtVDJYYmU5T1dtMzJiUFMxb0x4dz09
Meeting ID: 936 1464 4178.    Passcode: 965936






Natural Language Understanding and Semantic Parsing

(Partly joint work with former colleagues at Elemental Cognition)

Semantic parsing refers to the task of determining the propositional content of language: who did what to whom.  It is part of the larger task of natural language understanding (NLU).  I will start out by discussing what full NLU means, and argue that we are still far away, as a field, from solving full NLU, or even from knowing how to evaluate it.

In the second part of the talk, I will situate semantic parsing in the context of several other NLU subtasks.  Typically, the target representation of semantic parsing uses an ontology (such as PropBank or FrameNet).  Semantic parsing includes the subtasks of word sense disambiguation, argument detection, and argument role labeling.  I will discuss choices among possible target ontologies.  I will justify why we created a new ontology, Hector, based on FrameNet and the lexical resource NOAD, and explain some of its characteristics.

In the third part of the talk, I will present experiments we performed using transformer models.  We obtain best results using a two-phase model, in which we first choose the frame, and then, given the frame, choose the arguments.  We encode the problem for both tasks using indices in the sentence.  While we develop the parser for our new ontology Hector, this approach also beats the state of the art for FrameNet and PropBank parsing.Biography:  I am a professor in the Department of Linguistics at Stony Brook University with a joint appointment in IACS.

Until recently, I was a research scientist at Elemental Cognition. Elemental Cognition is working on deep natural language understanding.

I got my PhD with Aravind Joshi at the University of Pennsylvania in 1994. I have worked at CoGenTex, and at AT&T Labs -- Research, and for many years I was a research scientist at Columbia University in the Center for Computational Learning Systems.
























A new virtual seminar series on Games, Decisions, and Networks will start this Friday. The series aims at bringing together researchers working on foundations and applications of games theory, decision theory, and networks from computer science, control, economics and operation research. 





The advisory board for the series comprises Asu Ozdaglar (MIT), Christos Papadimitriou (Columbia), Drew Fudenberg (MIT), Eva Tardos (Cornell), Matthew O. Jackson (Stanford), Ramesh Johari (Stanford), and Tamer Başar (UIUC). 
The first talk will be given by Costantinos Daskalakis (MIT) on January 22nd at noon ET, titled Equilibrium Computation and the Foundations of Deep Learning. Upcoming speakers include


- Rakesh Vohra (Upenn)
- Sanjeev Goyal (Cambridge)
- Aaron Roth (Upenn)
- Aislinn Bohren (Upenn)
- Jason Marden (UCSB)

and more to be added!
The Natural Language Processing Reading Group at Stony Brook University meets weekly to discuss recent research papers in NLP and related fields.
Join the Google Group here.

Title: Mechanistic Interpretability, Verbalizable Representations Form a Global Workspace in Language Models

Abstract, Mechanistic Interpretability: The rise of the term mechanistic interpretability has accompanied increasing interest in understanding neural models -- particularly language models. However, this jargon has also led to a fair amount of confusion. So, what does it mean to be mechanistic? We describe four uses of the term in interpretability research. The most narrow technical definition requires a claim of causality, while a broader technical definition allows for any exploration of a model's internals. However, the term also has a narrow cultural definition describing a cultural movement. To understand this semantic drift, we present a history of the NLP interpretability community and the formation of the separate, parallel mechanistic interpretability community. Finally, we discuss the broad cultural definition -- encompassing the entire field of interpretability -- and why the traditional NLP interpretability community has come to embrace it. We argue that the polysemy of mechanistic is the product of a critical divide within the interpretability community.

Abstract, Verbalizable Representations Form a Global Workspace in Language Models: If the mind is an ocean, we spend our lives floating at the surface. Beneath us, an enormous amount of processing takes place without our knowledge: our visual systems parsing the contours of a face, our motor circuits maintaining our posture. At any given moment, only a small fraction of this neural activity is accessible to us. Yet it is this privileged sliver of activity that we rely on to reason deliberately: to plan what ingredients to buy for a recipe, or to puzzle out why an engine won't start. Such thoughts can be articulated out loud, deliberately held in mind, and brought to bear on whatever task the moment demands. This distinction, between our accessible thoughts and our unconscious processing, is perhaps the most striking feature of human cognition.
In this paper, we present evidence that an analogous functional distinction has emerged in modern AI models. Specifically, we observe that language models maintain a privileged set of internal representations, available for report, modulation, and flexible internal reasoning, atop a much larger volume of automatic processing. We identify these representations using a new interpretability technique, which surfaces the concepts a model is poised to verbalize at any point in its processing. Measuring and intervening on these representations provides us a window into a model's thought processes, uncovering internal reasoning and reactions that do not appear in its output.

Location: NCS 220

Zoom Link: https://stonybrook.zoom.us/j/92942833294?pwd=ysKMaQx7Hp0lkq5nOb3zJ9ivNZVvLv.1&jst=2
The IEEE/CVF Conference on Computer Vision and Pattern Recognition 2025 will be held from June 11th to June 15th, 2025, at the Music City Center, Nashville, TN. The IEEE / CVF Computer Vision and Pattern Recognition Conference (CVPR) is the premier annual computer vision event comprising the main conference and several co-located workshops and short courses. With its high quality and low cost, it provides an exceptional value for students, academics and industry researchers. Register here.
Abstract: Artificial Intelligence (AI) is no longer a futuristic concept -- it is here, but its development, benefits, and risks remain unevenly distributed across industries, nations, and social groups. In this talk, Jieshu presents her research on the societal dimensions of AI from two perspectives: the forces shaping AI's development (backward-looking) and its current and potential impact on society (forward-looking). She first examines disparities in AI, including women's underrepresentation in AI patents and the geographic concentration of AI innovation, highlighting inequalities in who creates AI and who benefits from it. She then explores AI's societal impact, focusing on workforce transformation and the need for GenAI literacy. She will also discuss AI patents, AI's role in climate change mitigation and adaptation, potential environmental biases in LLMs, and gender-specific patterns in AI portrayals in science fiction.

Bio: Jieshu Wang is a Postdoctoral Research Scholar at Arizona State University (ASU), focusing on the social dimensions of artificial intelligence (AI). With a background in engineering, economics, communication, and science and technology studies, she examines how AI both shapes and is shaped by broader societal forces. Her research employs interdisciplinary methods to explore the social, political, and economic factors influencing AI development, as well as its role in innovation, the economy, the future of work, climate change mitigation, and popular culture. Jieshu holds a Ph.D. in Human and Social Dimensions of Science and Technology from ASU. She is also a science book translator and has translated six books.

Location: Old Computer Science, room 1310
The Future Histories Studio will host Young Maeng, an artist and professor at California State University, Fresno, for a talk exploring the intersection of artificial intelligence (AI) and traditional painting, examining how two seemingly disparate fields can converge to create new artistic expressions.

The lecture is part of the Future History Studio series at Stony Brook University, a platform dedicated to examining the evolving relationship between technology, art, and society.

Young will discuss her innovative approach to expanded painting, an integration of AI-generated images and traditional techniques such as Korean ink and acrylic painting. Through this fusion, she visualizes complex philosophical and ethical questions about the coexistence of humans, nature, and AI companion robots. The lecture will highlight the broader implications of AI in the art world, touching on how AI technologies challenge conventional notions of creativity and human-centric perspectives in art.

Speaker Bio:

Young Maeng is an artist and professor at California State University, Fresno, whose work explores the intersection of artificial intelligence (AI) and traditional painting techniques such as Korean ink and acrylic.

Maeng's innovative approach to expanded painting blends AI technology with traditional methods to visualize complex philosophical and ethical questions surrounding the coexistence of humans, nature, and AI companion robots.

Location: Future Histories Studio
Register here: https://www.eventbrite.ca/e/ai-and-painting-tickets-1021050809457?aff=oddtdtcreator

In this workshop, we will take a historical journey through the ideas that eventually led to today's large language models, such as ChatGPT and Claude.

Modern LLMs can seem overwhelmingly complex, but they didn't appear all at once. Many of the basic ideas behind them have been developing for more than a century. By following this history step by step, we can better understand what language models do and how they evolved.

In part 1 of this workshop, we will begin with Andrey Markov's early work on sequences and probability in 1913 and Claude Shannon's influential work on information and language in 1948. From there, we will explore key ideas such as Markov chains, N-grams, and the next-word generation process using simple, accessible examples.

In part 2 of the workshop, which is not in this workshop, we will also look at how an artificial neuron can learn to predict the next word, an important step toward neural language models that later became the foundation of modern LLMs and we will see what perplexity is.

The workshop is designed for anyone who is curious about where today's AI came from and how a century of ideas about language, probability, and computation helped make modern language models possible. This workshop targets students of any level who are interested in language models or history. No background in computer science or artificial intelligence is required. The hardest part is converting counted numbers to probabilities.

Our historical journey will cover roughly the period from 1918 to 1970s.

Location: Melville Library, Central Reading Room, Lab A

Register for the event on SB Engaged.

ChatGPT for Developing Interactional Competence: Affordances and Constraints
MIC Distinguished Lecture Series

Abstract: The advent of ChatGPT has redrawn the boundary of pedagogical discourse, where the dyadic configuration of teacher-student has, for many, become triadic -- one that includes AI as an relevant third party, not to be missed or dismissed. Within applied linguistics, AI-focused research has predominantly targeted the teaching and learning of writing (Fang & Han, 2025). The work on AI and speaking, on the other hand, has largely involved perception studies documenting its positive impact on learners' willingness to communicate (Goh & Aryadoust, 2025). In this talk, I explore the role of AI in the teaching and learning of speaking, and in particular, the development of interactional competence. Based on a corpus of learner-AI interactions, I demonstrate the ways in which ChatGPT excels and fails at acting as a useful conversation partner, with a view towards furthering our ongoing deliberation on the affordances and constraints of AI in language education.

Speaker: Hansun Zhang Waring is Professor of Linguistics and Education at Columbia University and founder The Language and Social Interaction Working Group (LANSI). As an applied linguist and a conversation analyst, Hansun is interested in all things interaction -- (second language) pedagogical interaction, communication with the public, parent-child interaction, and human-AI interaction (HAI). Her work has appeared in leading journals in applied linguistics and discourse analysis as well as numerous book volumes, some of which she (co-)authored or co-edited. She is on the editorial boards of Chinese Language and Discourse (CLD), Classroom Discourse (CD), and International Review of Applied Linguistics (IRAL).

Location: HUM 1006