Abstract: Human gaze behavior is a fundamental cue for understanding social intent, human-machine interaction, and cognitive processes. This thesis addresses the challenges of gaze target estimation (GTE), also known as gaze following, by developing a holistic understanding of gaze in complex environments.

The first part of this work improves GTE performance by introducing Patch-level Distribution Prediction (PDP). Unlike traditional models that rely on strict pixel-wise regression, PDP models gaze as a distribution over patches, which better accounts for annotation variance and bridges the gap between target location and in/out-of-frame prediction. To address the laborious nature of data labeling, the second part presents GCDR, the first semi-supervised method for gaze following. By prompting large Visual Question Answering (VQA) models to generate initial Grad-CAM heatmaps and refining them with a diffusion model, this method achieves high performance with significantly fewer human annotations. The third part expands the applicability of GTE to multi-camera environments. By introducing the Multi-View Gaze Target (MVGT) dataset, along with two novel frameworks for integrating information between multiple views and predicting the gaze target across views, we explore a new direction that overcomes single-view limitations such as face occlusion and out-of-view targets.

Building on these foundations, the final part of this thesis proposes a new direction toward semantic social gaze understanding using next-generation multimodal Large Language Models (LLMs). Rather than focusing solely on geometric gaze target localization, we aim to enrich gaze prediction with semantic and relational interpretation in complex social scenes. To this end, we will leverage existing gaze following datasets to derive social gaze supervision, including mutual gaze and shared attention, and obtain aligned language descriptions of scene-level gaze behaviors. This proposed work will enable the model to not only locate gaze targets but also predict structured social gaze relations among individuals, meanwhile generating a concise natural-language summary describing the dominant gaze interactions. By integrating spatial gaze estimation, social relation reasoning, and language-based scene understanding within a unified multimodal model, this work takes an important step toward a holistic understanding of human gaze behavior in real-world environments.

Speaker: Qiaomu Miao
Abstract: AI is reaching children faster than the rules meant to protect them. Schools are pushed toward early AI exposure, chatbot safety rules lag behind, and families of young people with autism and intellectual and developmental disabilities (IDD) often wait longest for support. This talk asks who gets a say in the AI built for young people and their families, and when. I examine three settings, and each one brings families in earlier. The first is platform governance for young children. When YouTube implemented children's privacy law through a Made for Kids label, content creators had to classify their own videos, and parents stepped in mostly for the harms they could see. Creators and parents carried the costs after the rules were set. The second is safety by design for teenagers. We built MOSafely, a teen-centered online risk detection tool, and asked 33 online safety experts to review it before it reached teens. The experts found that to protect teens' messages, the tool must read them. The third is AI support for families with special needs. With caregivers of young adults with autism and IDD, we co-designed AI support for the workplace and found that such support can make workers too visible to employers. Across the three settings, families move from carrying the costs of technology to shaping its design. I close with a question for my next five years. Who decides when AI is ready for a family's home?

Bio: Renkai Ma is an Assistant Professor in the School of Information Technology at the University of Cincinnati. He designs, builds, and evaluates various AI technologies that support the safety and well-being of children and adolescents, parents and caregivers, and autistic people and people with intellectual and developmental disabilities (IDD). His research spans platform governance for young children, safety by design for teenagers, and AI support for families with special needs. His work appears at ACM CHI, CSCW, WWW, UIST, IDC, and DIS. In 2026, he was one of eight recipients of the University of Cincinnati's University Research Council Faculty Scholars Research Award. He received his Ph.D. in Informatics from Penn State, was a postdoctoral scholar at Vanderbilt University, and conducted user experience research at Meta and SiriusXM.

Location: Seminar Room 1310, Old Computer Science Building, Stony Brook University
Predicting Subjective Attributes in Visual Data - Zijun Wei

ABSTRACT: Recent progress in deep neural networks has revolutionized many computer vision tasks such as image classification, detection and segmentation. However, in addition to excelling in tasks that predict well-defined objective information, human-centered artificial intelligence systems should also be able to model subjective attributes, as defined by human perceptual behavior, that goes beyond the pure physical content of visual data. Example subjective tasks are the prediction of spatial or temporal regions that are interesting to humans (e.g., attract attention or are visually pleasing) and the recognition of subjective attributes (e.g., visually elicited sentiments). Better models for these tasks will improve the human-computer interaction experience in various applications. This thesis investigates several approaches to address the challenges in predicting those subjective attributes in visual data over a diverse set of tasks. I first present a novel framework for real-time automatic photo composition. The framework consists of a cost-effective data collection workflow, an efficient model training pipeline and a lightweight module to account for personalized preferences. Then I develop a novel and general algorithm to detect interesting segments in sequential data, which can be naturally applied to video summarization tasks. Furthermore, I propose methods that learn to represent sentiments elicited by images, in an unsupervised manner, using linguistic features extracted from large scale Web data. To conclude this thesis, I introduce a human-vision-inspired image classification algorithm that also predicts spatial visual attention even though no attention data was used for training it.  
Abstract:
Artificial intelligence (AI)-based methods and computational materials science continue to make inroads into accelerated materials design and development. I will review Al-enabled advances made in the subfield of polymer informatics, with a particular focus on the design of application-specific practical polymeric materials. I will describe exemplar design attempts within a few critical and emerging application spaces, including materials designs for storing, producing, and conserving energy, and those that can prepare us for a sustainable economy powered by recyclable and/or biodegradable polymers. Al- powered workflows help efficiently search the staggeringly large chemical and configurational space of materials, using modern machine-learning (ML) algorithms to solve forward and inverse materials design problems. A practical informatics-based design protocol involves creating a set of application-specific target property criteria, building ML model predictors for those relevant target properties, enumerating or generating a tangible population of viable polymers, and selecting candidates that meet design recommendations. The protocol will be demonstrated for several energy and sustainability-related applications. Finally, I will offer an outlook on the lingering obstacles that must be overcome to achieve widespread adoption of informatics-driven protocols in industrial-scale materials development.

Speaker Bio:
Prof. Ramprasad is the Regents' Entrepreneur, Michael E. Tennenbaum Family Chair and Georgia Research Alliance Eminent Scholar in the School of Materials Science & Engineering at the Georgia Institute of Technology. He is also the CEO and co-founder of Matmerize, Inc. His area of expertise is the development and application of computational and machine learning tools to accelerate sustainable materials development aimed at energy production, storage and utilization. Prof. Ramprasad received his B. Tech. in Metallurgical Engineering at the Indian Institute of Technology, Madras, India, an M.S. degree in Materials Science & Engineering at the Washington State University, and a Ph.D. degree also in Materials Science & Engineering at the University of Illinois, Urbana-Champaign.
Prof. Ramprasad is a Fellow of the Materials Research Society, a Fellow of the American Physical Society, an elected member of the Connecticut Academy of Science and Engineering, and the recipient of the Alexander von Humboldt Fellowship and the Max Planck Society Fellowship for Distinguished Scientists. He has authored or co-authored over 300 peer-reviewed journal articles, 8 book chapters and 8 patents, and has delivered over 300 invited talks at Universities and Conferences worldwide. He is a member of the Editorial Advisory Boards of npj Computational Materials, ACS Materials Letters and Journal of Physical Chemistry A/B/C. He created and chaired the inaugural 2022 Gordon Research Conference on Computational Materials Science and Engineering.

Location: Room 301, Engineering Building

Abstract: Modern large language models (LLMs) increasingly operate not on single inputs, but over sets of information--collections of generated statements, retrieved documents, or intermediate reasoning steps. However, existing frameworks largely treat these elements independently, overlooking the fact that correctness and reliability often emerge at the set level, where interactions such as contradictions, redundancy, or joint relevance become critical. In this talk, I argue that reasoning over sets is a fundamental challenge for building reliable and controllable LLM systems, and present a unified perspective centered on set-level verification, retrieval, and editing.

Bio: Jay-Yoon Lee is an associate professor in the Graduate School of Data Science at
Seoul National University. The goal of his research is injecting knowledge/constraints into neural models, primarily for natural language processing (NLP) tasks. He is broadly interested in structured prediction, multi-task learning, logical reasoning, and better representation learning for the aforementioned topics. Currently, he is exploring these topics at the Graduate School of Data Science (GSDS), Seoul National University (SNU), with SKI-ML lab members. Prior to joining SNU he was a postdoc associate at IESL with Professor Andrew McCallum and during his Ph.D. with his advisor Jaime Carbonell.

Location: NCS 120

You are cordially invited to Stony Brook University's

FinTech Workshop

A conversation bringing together academic research and industry leaders

in financial technology


Friday, November 6, 2026

1 PM to 4:30 PM

First Floor Auditorium

New Computer Science Building


1 pm

Welcome & Keynote Introduction

Andrew C. Singer, Dean of the College of Engineering and Applied Sciences (CEAS), Stony Brook University

1:05 pm

Keynote Address:

Robert J. Frey, Founder and Director of Stony Brook's Quantitative Finance Program, renowned quantitative finance expert and investor

1:40 pm

Academic Session

  • 1:20 pm: Jeffrey T. Ludwig, Mathematics Department and Director of Jump Labs, University of California, Irvine

  • 1:30 pm: Lav Varshney, AI Innovation Institute (AI3), Stony Brook University

  • 1:40 pm: Pawel Polak, Applied Math & Statistics (AMS), Stony Brook University

2:10 pm

Industry Session

  • 2:00 PM: Roger Burkhardt, Broadridge

  • 2:10 PM: Jill Scher, CBIZ Advisors

  • 2:20 PM: Milind Sharma, QuantZ and QWAFAxNEW

2:40 pm

Industry-Academic Panel (Moderator: Yvette Wohn, Chair of Department of Technology, AI & Society)

  • 2:40 PM: Panel discussion

  • 3:00 PM: Audience Q&A

3:10 pm

Broadridge Innovation Lab

Samir Das, Chair of the Computer Science Department

3:15 pm

Coffee & Networking

Open floor until 4:30 PM

Please RSVP by Friday, October 30

Register Here

Join us at the Center for Excellence in Learning and Teaching (CELT) for an interactive Zoom workshop on Generative AI designed for faculty and staff interested in enhancing teaching and assessment practices, increasing student engagement, and navigating the rapidly evolving landscape of AI tools. Participants will be introduced to common AI tools, explore potential instructional uses, and discuss key considerations such as academic integrity, transparency, and equity.

Register now: https://stonybrook.zoom.us/meeting/register/6js1eP64T1ys8tyU57EJ7Q#/registration