Abstract: Human gaze behavior is a fundamental cue for understanding social intent, human-machine interaction, and cognitive processes. This dissertation addresses the challenges of gaze target estimation (GTE), also known as gaze following, by developing a holistic understanding of gaze across complex environments.
First, we improve GTE performance through Patch-level Distribution Prediction (PDP). Unlike traditional pixel-wise regression, PDP models gaze as a spatial distribution over patches, better accounting for annotation variance and regularizing the pixel-wise heatmap prediction through multi-scale modeling. Second, to mitigate the high cost of data labeling, we present GCDR, the first semi-supervised method for gaze following. By prompting large Visual Question Answering (VQA) models to generate initial Grad-CAM heatmaps and refining them via a diffusion model, GCDR achieves robust performance with minimal human annotation. Third, we expand the applicability of GTE to multi-camera environments. By introducing the Multi-View Gaze Target (MVGT) dataset, along with two novel frameworks for integrating information and predicting gaze targets across views, we explore a new direction that overcomes single-view limitations such as face occlusion and out-of-view targets. Finally, we propose OmniGF, a multi-person gaze following model built on Vision-Language Models (VLMs) that enriches gaze target localization with semantic and social reasoning. By leveraging the semantic capabilities of VLMs alongside structural innovations to ground the model with fine-grained cues for each individual, OmniGF achieves state-of-the-art performance across three gaze following tasks.
Collectively, by tackling the gaze following problem through the distinct yet complementary perspectives of probabilistic modeling, geometric reasoning, and multimodal learning, this dissertation builds a holistic understanding of human gaze, paving the way for more intuitive artificial intelligence systems in downstream applications.

Speaker: Qiaomu Miao

Location: NCS 220
The Natural Language Processing Reading Group at Stony Brook University meets weekly to discuss recent research papers in NLP and related fields.
Join the Google Group here.

IdeaLens: Detecting AI Ideas in Long-form Writing

Abstract: While modern AI detectors identify who wrote the words, emerging policies on AI use increasingly hinge on a different question: who came up with the ideas? We introduce IdeaLens, a detector that identifies whether a document's ideas came from a human or AI (idea provenance), regardless of who wrote its words. To focus IdeaLens on ideas rather than prose, we represent documents as outlines: lists of items that each pair a discourse role with a brief, paraphrased description of the content, minimizing word-level overlap with the raw text. We train IdeaLens on 1M FineWeb documents with silver labels from Pangram, a prose provenance detector. Since the outlines are largely stripped of surface-level information, the labels must be fit mainly through the ideas. In a controlled study, IdeaLens's AI flag rate drops from 95% to 7% as models write from increasingly detailed human plans, while Pangram 4 still flags 92%; from AI-derived plans, IdeaLens stays above 96%. Conversely, on a new dataset of 50 stories that human authors wrote from AI-generated plans, IdeaLens flags 68% of the stories as AI, compared to 8% for Pangram 4. On a comprehensive suite of 19 existing detection benchmarks, we show that IdeaLens maintains strong detection rates at low false positive rates, suggesting that ideas themselves provide a powerful discriminative signal, and its performance holds across domains, formats, and languages. Finally, we examine 90K predictions from IdeaLens to characterize systematic differences between human and AI ideation. We release our models and labeled datasets to facilitate future research on idea provenance detection.

Location: NCS 220

Zoom Link: https://stonybrook.zoom.us/j/92942833294?pwd=927683
Abstract: Retrieval-augmented generation (RAG) systems empower large language models (LLMs) to access external knowledge during inference. Recent advances have enabled LLMs to act as search agents via reinforcement learning (RL), improving information acquisition through multi-turn interactions with retrieval engines. However, existing approaches either optimize retrieval using search-only metrics (e.g., NDCG) that ignore downstream utility or fine-tune the entire LLM to jointly reason and retrieve--entangling retrieval with generation and limiting the real search utility and compatibility with frozen or proprietary models. In this work, we propose s3, a lightweight, model-agnostic framework that decouples the searcher from the generator and trains the searcher using a Gain Beyond RAG reward: the improvement in generation accuracy over naïve RAG. s3 requires only 2.4k training samples to outperform baselines trained on over 70 × more data, consistently delivering stronger downstream performance across six general QA and five medical QA benchmarks.

Speaker: Peter Zeng

Location: CS2311
The Empirical Methods in Natural Language Processing (EMNLP) conference is a premier international academic conference in the field of artificial intelligence and natural language processing (NLP). Organized annually by the Association for Computational Linguistics (ACL) special interest group on linguistic data (SIGDAT), it focuses on research that uses empirical methods to solve language processing problems.

For more information, and registration, visit the official website.

Google Docs has been updated to integration Gemini AI. Learn about the integrations that gives you more features within Google Docs. There are also AI Automations that will keep you updated when things happen with documents.

In this session, you will:

  1. Learn about the AI features within Google Docs
  2. Learn about workflow to be notified of changes to Docs
  3. Understand the features that allow AI to work

Register here.

The AI Phenomenon is here to stay. Will it help or hurt your career path? Industry leaders will share their insights and help us understand how AI will affect your ability to manage your career path. Technology will always evolve and add layers to the process. Don't run from it - Embrace it.

Presented by: SBU Career Center and Co-hosted with Alumni Relations

Register to attend.

Educational objectives:
  1. Techniques involving data driven inferences from multi-omics data
  2. Interdisciplinary approaches to uncover aspects of tumor biology
  3. Computational genomics method development, benchmarking, and validation


Speaker: Dr. Subhajyoti De, PhDProfessor, Cancer SystemsBiology and Pathology,Rutgers Cancer Institute,The State University of New Jersey

Location: MART Building, Room 7M-0602 (7th Floor)

Remote Access: https://stonybrook.zoom.us/j/95617197636?pwd=KytzZ2pVRG9SZGpKZUtpNXJISjNjZz09
Meeting ID: 95617197636 | Passcode: 924293
AI3, SBU Libraries and IACS present
at International Love Data Week
sponsored by The Office of the Provost and
Educational and Institutional Effectiveness (EIE)

Special Talk and Panel Discussion

How I Learned to Stop Worrying and Love AI (For Now)


with Paul Fain from The Job and Work Shift

A reporter's take on what we know--and what we don't know--about AI's emerging impacts on the labor market. The discussion will include the latest research from economists and the AI labs themselves about how workers are using AI, and current thinking among experts on how the tech's rapid deployment will play out across job roles, industries, and regions.

Panel discussion to follow with:

  • Lav Varshney, Della Pietra Infinity Professor and inaugural director of the AI Innovation Institute
  • Nicholas Johnson, Director of AI, SBU Libraries
  • Marianna Savoca, Associate Vice President for Career Readiness and Experiential Education
Paul Fain is co-founder of Work Shift, editor of the must-read newsletter, The Job, and host of The Cusp podcast. A veteran higher education reporter, Paul is perhaps the nation's top journalist focused on connections between education and work. He started Work Shift after a decade as a senior reporter and then news editor at Inside Higher Ed, where he led the outlet's coverage of low-income and first-generation students, college completion, community colleges, federal policy, and emerging models of higher education. He also was the founding host of the successful podcast, The Key with Inside Higher Ed, and has contributed chapters for books on innovation in higher education, published by the Harvard University Press and the Stanford University Press. Earlier in his career, Paul was a senior reporter at The Chronicle of Higher Education.

Limited Seats!

Registration is required.
Abstract: Autonomous systems, whether on Earth or in space, rely on 3D perception to understand and interact with the world around them. Yet traditional techniques for 3D understanding often depend on human designed features, fixed sensors, and conventional imaging modalities. This constrained approach can limit every stage of perception, from sensing to interpretation to decision making.
In this talk, we'll explore an alternative paradigm for imaging: physically based neural representations for 3D scenes and 3D sensing systems. We will discuss how recent advances in large scale learned representations can be used to jointly optimize both 3D scene models and the design of sensing systems for 3D capture, with the goal of enabling task specific perception systems.
Unlike modern AI models trained on internet scale datasets, these specialized 3D representations typically operate in data sparse regimes and therefore require a different kind of prior. We'll examine how grounding these learned representations in the physics of light transport can improve our understanding of scene structure, and inform imaging system design even with limited data. By connecting physical insights with learned representations, we'll highlight new possibilities for robust, efficient, and adaptive perception in challenging environments.

Speaker: Nikhil Behari is a graduate student in the Camera Culture group at the MIT Media Lab, advised by Professor Ramesh Raskar. His research interests include computational imaging, 3D scene understanding, and multi-agent decision-making under uncertainty, with a focus on automating imaging system design for 3D perception in human and planetary health. His research is supported by the NASA Space Technology Graduate Research Fellowship. He received his bachelor's in Computer Science and Statistics from Harvard University in 2022.
The Artificial Intelligence Innovation Institute (AI^3), with administrative support from the Office of the Vice President for Research (OVPR), invites applications to a seed grant program for collaborative projects in artificial intelligence, along three distinct tracks: Collaborative Research in AI, Technical Support for Discipline-Centric Research, and Seed Grants for AI Education and Service.

The program will fund projects for up to a one-year period, depending on the availability of funds. AI^3 anticipates making at least six awards on this call. A one-year, no-cost extension can be requested in the final 6 months of a project with approval subject to progress towards project goals and active participation in research themes.

Competitive applications will actively incorporate modern AI technologies into the work; integrate students; document significant potential for future funding or other growth-oriented outcomes; and highlight innovations.

The 2024 application deadline will be October 15, at 11:59 PM EST. Recipients will be notified by December 20, and projects are anticipated to commence at the start of the Spring 2025 semester.