Abstract: Computer vision seeks to extract semantic and geometric information from images and videos, serving as the perceptual foundation for intelligent systems such as robots and autonomous vehicles. Over the past decade, deep learning has driven remarkable progress in the field, advancing capabilities from 2D recognition to 3D reconstruction. However, the current purely data-driven paradigm faces fundamental challenges, including data inefficiency, curse of high dimensionality, and limited understanding of visual entities beyond individual objects.

In this talk, I will present my recent research on modeling and learning rich visual structures to address these challenges. First, I will introduce a novel framework that integrates explicit visual dependency modeling with deep learning for 2D and 3D dense prediction. Next, I will demonstrate how unfolding the manifold structure of visual data enables unsupervised semantic segmentation. Finally, I will present a recent project that represents, parses, and learns the geometric compositionality of 3D objects to facilitate self-supervised part-whole reconstruction. Through these efforts, I aim to bridge the gap between data-driven deep learning and visual structure modeling, paving the way for more efficient, generalizable, and interpretable computer vision models.

Bio: Dr. Wei Tang is an Assistant Professor in the Department of Computer Science at the University of Illinois Chicago (UIC). He obtained his Ph.D. in Electrical Engineering from Northwestern University, where his dissertation was honored with a Best Dissertation Award. His research interests include computer vision, digital image processing, and machine learning. Dr. Tang has served as an associate editor for several international journals, including Pattern Recognition and Machine Vision and Applications, and as an area chair for leading conferences, including CVPR, ICCV, and WACV. His research has been funded by the National Science Foundation (NSF) and industry partners such as Motorola and Wormpex AI Research.


Location: NCS 115

Zoom: https://stonybrook.zoom.us/j/4624091659?omn=95178138684&jst=3

As generative AI (GenAI) continues to reshape the educational landscape, educators must critically examine its implications for course design. How can we adapt our courses to ensure meaningful learning in a post-GenAI world? How can we harness its potential while mitigating risks to student learning? This seminar explores the evolving role of GenAI in higher education, emphasizing learner-centered teaching practices--such as backward design, transparency, and active learning--as essential strategies for navigating both the opportunities and challenges posed by GenAI. We will examine how GenAI disrupts traditional models of teaching and assessment, highlighting course design choices that intentionally promote deep learning and critical thinking in this new era.

Speaker Bio: Dr. Lourdes Alemán is an Associate Director at MIT's Teaching and Learning Lab (TLL). She earned her Ph.D. in Biology from MIT, studying RNA interference (RNAi) with Professor Phil Sharp. She later completed a postdoc in curriculum innovation with Professor Graham Walker's HHMI MIT Education Group. As a postdoc and research scientist, she helped develop software tools for teaching experimental design and data analysis, including collaborations with the MIT-Haiti Initiative. Before joining TLL, she worked at MIT's Open Learning, supporting MIT faculty in blended and online education. At TLL, Lourdes trains graduate students and postdocs in college-level teaching, advises faculty on classroom innovation, and previously designed and taught a hands-on biology module on novel antibiotic discovery for first-year students. She has served on university committees focused on mentoring and advising. Drawing from her experiences as a Cuban immigrant student, she developed MIT's first curriculum on growth mindset and co-founded Flipping Failure, a campus-wide initiative for students to share their stories of academic challenges and the strategies they have used to overcome them.

Abstract: Retrieval-augmented generation (RAG) systems empower large language models (LLMs) to access external knowledge during inference. Recent advances have enabled LLMs to act as search agents via reinforcement learning (RL), improving information acquisition through multi-turn interactions with retrieval engines. However, existing approaches either optimize retrieval using search-only metrics (e.g., NDCG) that ignore downstream utility or fine-tune the entire LLM to jointly reason and retrieve--entangling retrieval with generation and limiting the real search utility and compatibility with frozen or proprietary models. In this work, we propose s3, a lightweight, model-agnostic framework that decouples the searcher from the generator and trains the searcher using a Gain Beyond RAG reward: the improvement in generation accuracy over naïve RAG. s3 requires only 2.4k training samples to outperform baselines trained on over 70 × more data, consistently delivering stronger downstream performance across six general QA and five medical QA benchmarks.

Speaker: Peter Zeng

Location: CS2311
Join the Department of Computer Science as we welcome Lyle Ungar, University of Pennsylvania, who will be delivering a lecture on 'Measuring Cultural Variation using Natural Language Processing.' When: 11/08/24 @ 2:30 PM Where: New Computer Science Building, Room 120. Reception to follow. Abstract: Cultures vary widely in how they view the world, for example being more individualist or collectivist. Such cultural differences are, of course, reflected in the words that people use. We first show a variety of ways in which multilingual language models are not multicultural; they speak Hindi or Mandarin, but still think like Americans. In contrast, we then present a scalable method that uses embedding-derived lexica to successfully measure regional variation in culture. Bio: Lyle Ungar is a Professor of Computer and Information Science at the University of Pennsylvania, where he also holds secondary appointments in Psychology, Bioengineering, Genomics and Computational Biology, and Operations, Information and Decisions. His group uses natural language processing and explainable AI for psychological research, including analyzing social media and cell phone sensor data to better understand the drivers of physical and mental well-being. They are currently building socio-emotionally sensitive GPT-based tutors and coaches.

Over the past decade, researchers in neuroscience, psychology and artificial intelligence have come together to build advanced computer models that mimic how our brain processes what we see. These models are designed to closely copy the brain's visual system, all the way to a key area called the inferior temporal cortex, which plays an important role in recognizing objects.

Because these computer models can be fully observed, scientists can use them to make detailed predictions about how the brain works -- something older, more theoretical models could not do.

Dr. James DiCarlo's work explores whether these computer digital twin models of the brain could help guide safe, non- invasive ways to infl uence brain activity. In his talk, he explains how such a model could be used to design specific patterns of light. When this carefully designed light is added to what the eye naturally sees, it can precisely influence activity in groups of neurons in the inferior temporal cortex.

Since neural activity in this visual brain area may be connected to emotional states like anxiety, this research could eventually open the door to non-invasive approaches that may benefit mental well-being in the future.

Speaker: James J. DiCarlo, MD, PhD, Peter de Florez Professor, MIT Brain and Cognitive Sciences, and Director, MIT Siegel Family Quest for Intelligence

Location: Staller Center Main Stage

The event will be livestreamed at stonybrook.edu/live

https://stonybrook.zoom.us/j/99820812332?pwd=c05BSTVLNmw3L04yZjdEcG5pem1OZz09 Speaker: Alexei Koulakov of Cold Spring Harbor Laboratory Brain evolution as a machine learning problem We have entered a golden age of artificial intelligence research, driven mainly by the advances in ANNs over the last decade or so. Applications of these techniques--to machine vision, speech recognition, autonomous vehicles, machine translation and many other domains--are coming so quickly that many observers predict that the long-elusive goal of Artificial General Intelligence (AGI) is within our grasp. However, we still cannot build a machine capable of building a nest, stalking prey, or loading a dishwasher. I will describe several projects, ranging from theories of evolution of neural development to the perception of smells, in which we are attempting to understand the algorithms that the nervous system is using to solve some of these challenging problems.
Do Natural Language Understanding Systems Learn to Understand or to
Find Shortcuts? (Naoya Inoue, http://naoya-i.github.io/)

ABSTRACT: Recent studies have suggested that natural language understanding (NLU) systems learn to exploit superficial, task-unrelated cues (a.k.a. annotation artifacts) in current datasets. This prevents the community from reliably measuring the progress of NLU systems. In this talk, I will discuss two latest studies from our research team: (i) analysis of annotation artifacts in commonsense causal reasoning and (ii) creation of benchmark for evaluating NLU systems' internal reasoning.
---------------------------------------------------------------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------------------------------------------------------------
Learning graph-structured sparse models (Baojian Zhou, https://baojianzhou.github.io/) 

ABSTRACT: Learning graph-structured sparse models has recently received significant attention thanks to their broad applicability to many important real-world problems. However, such models, of more effective and stronger interpretability compared with their counterparts, are difficult to learn due to optimization challenges. In this talk, we will discuss how to learn graph-structured sparse models under stochastic and online learning settings. Some interesting related problems will also be discussed.


The International Conference on Learning Representations (ICLR) is the premier gathering of professionals dedicated to the advancement of the branch of artificial intelligence called representation learning, but generally referred to as deep learning.



ICLR is globally renowned for presenting and publishing cutting-edge research on all aspects of deep learning used in the fields of artificial intelligence, statistics and data science, as well as important application areas such as machine vision, computational biology, speech recognition, text understanding, gaming, and robotics.

ICLR is one of the fastest growing artificial intelligence conferences in the world. Participants at ICLR span a wide range of backgrounds, from academic and industrial researchers, to entrepreneurs and engineers, to graduate students and postdocs.

The rapidly developing field of deep learning is concerned with questions surrounding how we can best learn meaningful and useful representations of data. ICLR takes a broad view of the field and includes topics such as feature learning, metric learning, compositional modeling, structured prediction, reinforcement learning, and issues regarding large-scale learning and non-convex optimization.

A non-exhaustive list of relevant topics explored at the conference include:



  • Unsupervised, Semi-supervised, and Supervised Representation Learning
  • Representation Learning for Planning and Reinforcement Learning
  • Metric Learning and Kernel Learning
  • Sparse Coding and Dimensionality Expansion
  • Hierarchical Models

  • Optimization for Representation Learning
  • Learning Representations of Outputs or States
  • Implementation Issues, Parallelization, Software Platforms, Hardware
  • Applications in Vision, Audio, Speech, Natural Language Processing, Robotics, Neuroscience, or Any Other Field


For more information or registration, please visit the official website.

Abstract: Human gaze behavior is a fundamental cue for understanding social intent, human-machine interaction, and cognitive processes. This dissertation addresses the challenges of gaze target estimation (GTE), also known as gaze following, by developing a holistic understanding of gaze across complex environments.
First, we improve GTE performance through Patch-level Distribution Prediction (PDP). Unlike traditional pixel-wise regression, PDP models gaze as a spatial distribution over patches, better accounting for annotation variance and regularizing the pixel-wise heatmap prediction through multi-scale modeling. Second, to mitigate the high cost of data labeling, we present GCDR, the first semi-supervised method for gaze following. By prompting large Visual Question Answering (VQA) models to generate initial Grad-CAM heatmaps and refining them via a diffusion model, GCDR achieves robust performance with minimal human annotation. Third, we expand the applicability of GTE to multi-camera environments. By introducing the Multi-View Gaze Target (MVGT) dataset, along with two novel frameworks for integrating information and predicting gaze targets across views, we explore a new direction that overcomes single-view limitations such as face occlusion and out-of-view targets. Finally, we propose OmniGF, a multi-person gaze following model built on Vision-Language Models (VLMs) that enriches gaze target localization with semantic and social reasoning. By leveraging the semantic capabilities of VLMs alongside structural innovations to ground the model with fine-grained cues for each individual, OmniGF achieves state-of-the-art performance across three gaze following tasks.
Collectively, by tackling the gaze following problem through the distinct yet complementary perspectives of probabilistic modeling, geometric reasoning, and multimodal learning, this dissertation builds a holistic understanding of human gaze, paving the way for more intuitive artificial intelligence systems in downstream applications.

Speaker: Qiaomu Miao

Location: NCS 220
University Libraries Presents:
Join librarian Christine Fena for an interactive workshop that invites you to explore AI tools first hand, not just as users, but as critical investigators.
Through playful experimentation and collaborative discovery, you'll uncover inherent biases, probe algorithmic flaws, and gain a deeper understanding of AI's limitations and societal impacts.

RSVP on SBEngaged

Location: Melville Library, Central Reading Room, Lab B