Postmortem Program Analysis from a Conventional Program Analysis Method to an AI-assisted Approach

Abstract: Despite the best efforts of developers, software inevitably contains flaws that may be leveraged as security vulnerabilities. Modern operating systems integrate various security mechanisms to prevent software faults from being exploited. To bypass these defenses and hijack program execution, an attacker needs to constantly mutate an exploit and make many attempts. While in their attempts, the exploit triggers a security vulnerability and makes the running process abnormally terminate.

After a program has crashed and abnormally terminated, it typically leaves behind a snapshot of its crashing state in the form of a core dump. While a core dump carries a large amount of information, which has long been used for software debugging, it barely serves as informative debugging aids in locating software faults, particularly memory corruption vulnerabilities. As such, previous research mainly seeks fully reproducible execution tracing to identify software vulnerabilities in crashes. However, such techniques are usually impractical for complex programs. Even for simple programs, the overhead of fully reproducible tracing may only be acceptable at the time of in-house testing.

In this talk, I will discuss how we tackle this issue by bridging program analysis with artificial intelligence (AI). More specifically, I will first talk about the history of postmortem program analysis, characterizing and disclosing their limitations. Second, I will introduce how we design a new reverse-execution approach for postmortem program analysis. Third, I will discuss how we integrate AI into our reverse-execution method to escalate its analysis efficiency and accuracy. Last but not least, as part of this talk, I will demonstrate the effectiveness of this AI-assisted postmortem program analysis framework by using massive amounts of real-world programs.

Bio: Dr. Xinyu Xing is an Assistant Professor at Pennsylvania State University. His research interests include exploring, designing and developing new program analysis and AI techniques to automate vulnerability discovery, failure reproduction, vulnerability diagnosis (and triage), exploit and security patch generation. His past research has been featured by many mainstream media and received the best paper awards from ACM CCS and ACSAC. Going beyond academic research, he also actively participates and hosts many world-class cybersecurity competitions (such as HITB and XCTF). As the founder of JD-OMEGA, his team has been selected for DEFCON/GeekPwn AI challenge grand final at Las Vegas. Currently, his research is mainly supported by NSF, ONR, NSA and industry partners.
The overall purpose of this seminar is to bring together people with interests in Computer Vision theory and techniques and to examine current research issues. This course will be appropriate for people who already took a Computer Vision graduate course or already had research experience in Computer Vision. To enroll in this course, you must either: (1) be in the PhD program or (2) receive permission from the instructors.

Each seminar will consist of multiple short talks (around 10 minutes) by multiple people. Students can register for 1 credit for CSE 656. Registered students must attend and present a minimum of 2 or 3 talks. Everyone else is welcome to attend. Fill in https://forms.gle/pCVXovgfMfQwGqG38 to subscribe to our mailing list for further announcement.
CSE 600 Seminar Series | Fall 2025


Abstract: Vision-language models that see and describe the world are now part of our daily lives, from internet search and accessibility tools to content generation and automatic moderation. However, as these models grow and become more widely used, their limitations have also become increasingly visible. In particular, it has been shown that these models are unable to reliably perform complex tasks that require abstraction and compositional reasoning. For example, they struggle to decompose an image or text into entities, attributes, and relations, and then reason over new combinations of these elements. As a result, we see generated content full of hallucinations, privacy leaks in images, and different types of biases in the model outputs.In this talk, I will outline a research agenda that aims to build trustworthy vision-language models in the age of generative AI. I will begin with compositional reasoning: how natural language inference can be used to decompose complex instructions and captions into atomic, verifiable statements, improving both evaluation and model behavior on tasks that require multi-step reasoning. I will then discuss how synthetic data and simulated environments can be used to train more reliable models, and how they can also stress-test models beyond standard benchmarks, revealing when models drop attributes, break object relations, or fail under distribution shifts. I will also share recent work on using hallucination correction as a signal to improve video-language alignment, and on privacy-preserving image understanding for blind and low-vision users. I will conclude with possible ways we can systematically probe, debug, and repair these models, turning synthetic perception into something we can trust in real-world deployments.



Speaker: Paola Cascante-Bonilla is a tenure-track Assistant Professor in the Department of Computer Science at Stony Brook University (SUNY). Before that, she was a Postdoctoral Associate at the University of Maryland Institute for Advanced Computer Studies (UMIACS), developing methods and metrics related to trustworthy machine learning. She received her Ph.D. in Computer Science at Rice University in 2024, working on Computer Vision, Natural Language Processing, and Machine Learning.Her research focuses on developing systems that enable compositional reasoning and common-sense inference through vision and language, while tackling issues such as cultural biases, data distribution, explainability, and trustworthy AI. Additionally, Cascante-Bonilla creates simulated environments for embodied agents to learn in a safe, controlled setting, aiming to facilitate effective collaboration and problem-solving for complex tasks by leveraging the implicit knowledge of large-scale pre-trained deep learning models.
Cascante-Bonilla is the recipient of the Ken Kennedy Institute SLB Graduate Fellowship (2022/23), she was selected as a Future Faculty Fellow by Rice's George R. Brown School of Engineering (2023) and as a Rising Star in EECS (2023).
Location: NCS 120
AI3, SBU Libraries and IACS present
at International Love Data Week
sponsored by The Office of the Provost and
Educational and Institutional Effectiveness (EIE)

Special Talk and Panel Discussion

How I Learned to Stop Worrying and Love AI (For Now)

with Paul Fain from The Job and Work Shift

A reporter's take on what we know--and what we don't know--about AI's emerging impacts on the labor market. The discussion will include the latest research from economists and the AI labs themselves about how workers are using AI, and current thinking among experts on how the tech's rapid deployment will play out across job roles, industries, and regions.

Panel discussion to follow with:

  • Lav Varshney, Della Pietra Infinity Professor and inaugural director of the AI Innovation Institute
  • Nicholas Johnson, Director of AI, SBU Libraries
  • Marianna Savoca, Associate Vice President for Career Readiness and Experiential Education
Paul Fain is co-founder of Work Shift, editor of the must-read newsletter, The Job, and host of The Cusp podcast. A veteran higher education reporter, Paul is perhaps the nation's top journalist focused on connections between education and work. He started Work Shift after a decade as a senior reporter and then news editor at Inside Higher Ed, where he led the outlet's coverage of low-income and first-generation students, college completion, community colleges, federal policy, and emerging models of higher education. He also was the founding host of the successful podcast, The Key with Inside Higher Ed, and has contributed chapters for books on innovation in higher education, published by the Harvard University Press and the Stanford University Press. Earlier in his career, Paul was a senior reporter at The Chronicle of Higher Education.

Limited Seats!

Registration is required.


Abstract: In high-dimensional data spaces, vast empty regions often exist where no known data points are present. These empty spaces are not merely gaps but hold untapped potential for discovering novel configurations, optimizing parameters, and improving decision-making processes. However, traditional exploration techniques struggle to identify and leverage these regions due to the curse of dimensionality. To address this, we introduce the Empty Space Search Algorithm (ESA), a scalable, physics-inspired method that systematically identifies and explores these uncharted voids. ESA operates by modeling the data space as a dynamic system, using a repulsion-attraction mechanism to locate optimal empty space configurations (ESCs) without requiring exhaustive search. Building upon ESA, we present GapMiner, a visual analytics system that integrates human-in-the-loop AI to iteratively refine and validate ESCs. GapMiner combines parallel coordinate visualization, interactive optimization, and deep learning-based predictive modeling to enhance the efficiency of empty space exploration. This methodology has broad applications, including accelerating convergence in evolutionary algorithms through a more diverse initial population, optimizing adversarial learning strategies, and discovering novel parameter configurations in reinforcement learning. Our approach demonstrates that empty space is not just an absence of data but a frontier for new possibilities in high-dimensional problem-solving.
Bio: Xinyu Zhang received his B.E. in Computer Science from Shandong University, Taishan College, in 2019. He is currently a final-year Ph.D. candidate in the Department of Computer Science at Stony Brook University, advised by Prof. Klaus Mueller. His research focuses on multivariate data analysis, scientific visualization, and reinforcement learning. He has published multiple papers in top-tier journals and conferences, including IEEE TVCG and NeurIPS.
*this seminar will be held in person (food provided on a first come, first serve basis), and online (zoom link below)!
Topic: IACS Student Seminar Speaker: Xinyu Zhang
Time: Feb 26, 2025 12:00 PM Eastern Time (US and Canada)
Join Zoom Meeting
https://stonybrook.zoom.us/j/91848218975?pwd=lfITFa61GaXZ2Wsa1B1OnbLQMmXvOE.1

Meeting ID: 918 4821 8975
Passcode: 027337
Zoom Like a Pro! Unlock Whiteboard, Polls, AI Companion, and more to supercharge student participation. This hands-on workshop explores innovative ways to use Zoom's built-in tools to enhance active learning activities in your classes. Learn how to utilize the Whiteboard feature to make collaborative work more engaging, use Polling and Quizzes for instant feedback, AI Companion for summary, and Breakout Sessions for group activities. Register here: https://stonybrook.zoom.us/meeting/register/tJckf--rpj4pGdRV0ItgTW8Lk7gn_RuykByO#/registration
Abstract: Human gaze behavior is a fundamental cue for understanding social intent, human-machine interaction, and cognitive processes. This thesis addresses the challenges of gaze target estimation (GTE), also known as gaze following, by developing a holistic understanding of gaze in complex environments.

The first part of this work improves GTE performance by introducing Patch-level Distribution Prediction (PDP). Unlike traditional models that rely on strict pixel-wise regression, PDP models gaze as a distribution over patches, which better accounts for annotation variance and bridges the gap between target location and in/out-of-frame prediction. To address the laborious nature of data labeling, the second part presents GCDR, the first semi-supervised method for gaze following. By prompting large Visual Question Answering (VQA) models to generate initial Grad-CAM heatmaps and refining them with a diffusion model, this method achieves high performance with significantly fewer human annotations. The third part expands the applicability of GTE to multi-camera environments. By introducing the Multi-View Gaze Target (MVGT) dataset, along with two novel frameworks for integrating information between multiple views and predicting the gaze target across views, we explore a new direction that overcomes single-view limitations such as face occlusion and out-of-view targets.

Building on these foundations, the final part of this thesis proposes a new direction toward semantic social gaze understanding using next-generation multimodal Large Language Models (LLMs). Rather than focusing solely on geometric gaze target localization, we aim to enrich gaze prediction with semantic and relational interpretation in complex social scenes. To this end, we will leverage existing gaze following datasets to derive social gaze supervision, including mutual gaze and shared attention, and obtain aligned language descriptions of scene-level gaze behaviors. This proposed work will enable the model to not only locate gaze targets but also predict structured social gaze relations among individuals, meanwhile generating a concise natural-language summary describing the dominant gaze interactions. By integrating spatial gaze estimation, social relation reasoning, and language-based scene understanding within a unified multimodal model, this work takes an important step toward a holistic understanding of human gaze behavior in real-world environments.

Speaker: Qiaomu Miao
https://meetings.cshl.edu/meetings.aspx?meet=naisys&year=20  


November 9 - 12, 2020 Virtual
Abstract Deadline: August 14, 2020


Organizers:

Raia Hadsell, DeepMind, United Kingdom
Blake Richards, Mila, Québec AI Institute, Canada
Anthony Zador, Cold Spring Harbor Laboratory

*********

The current COVID-19 situation is challenging and difficult for all of us - we hope this virtual conference will allow colleagues to share and discuss their latest research, while under travel and stay-at-home restrictions.

Because of the ongoing COVID-19/SARS-CoV-2 outbreak, CSHL and the organizers have now reached the difficult decision to restructure the meeting on From Neuroscience to Artificially Intelligent Systems into a virtual meeting scheduled November 9-12, 2020.  NAISys will begin at 10 am (EDT)  on Monday, November 9 and ending with an afternoon session on Thursday, November 12, 2020. Oral sessions will be confined to later morning and afternoon sessions EST to maximize access by participants from around the world. 

*********

Artificial intelligence (AI) and neural networks have long drawn on neuroscience for inspiration. However, in spite of tremendous recent advances in AI, natural intelligence is still far more adept at interacting with the real world in real-time, adapting to changes, and doing so under significant physical and energetic constraints. The goal of this meeting is to bring together researchers at the intersection of AI and neuroscience, and to identify insights from neuroscience that can help catalyze the development of next-generation artificial systems.

Abstracts are welcomed on all scientific topics related to how principles and insights from neuroscience can lead to better artificial intelligence. Topics of interest include but are not limited to network architectures, computing with spiking networks, learning algorithms, active perception, inductive bias, meta-learning, and online learning. Please note that abstracts should be ONE page (~2900 characters).   




Keynote speakers (pending reconfirmation):Yoshua Bengio, Université de Montréal
Yann Lecun, NYU/Facebook


Invited Speakers (pending reconfirmation):Kwabena Boahen, Stanford University
Dmitri Chklovskii, Simons Foundation
Anne Churchland, Cold Spring Harbor Laboratory
Claudia Clopath, Imperial College London, UK
Jim DiCarlo, MIT
Chelsea Finn, Stanford University
Surya Ganguli, Stanford University
Jeff Hawkins, Numenta
Konrad Kording, University of Pennsylvania
Timothy Lillicrap, DeepMind
Yael Niv, Princeton University
Bruno Olshausen, UC Berkeley
Cristina Savin, New York University
Terry Sejnowski, Salk Institute for Biological Studies
Sebastian Seung, Princeton University
Eero Simoncelli, New York University
Sara A. Solla, Northwestern University
David Sussillo, Google Brain
Andreas Tolias, Baylor College of Medicine


New and revised abstracts should be submitted by the resubmission deadline, Friday, August 14. Individuals originally selected for talks should assume they will still be speaking, but we may select additional talks based on the number of invited and selected speakers who cannot reconfirm.

Abstracts should contain only new and unpublished material and must be submitted electronically by the abstract deadline. Selection of material for oral and poster presentation will be made by the organizers and individual session chairs. Status (talk/poster) of abstracts will be posted on our web site as soon as decisions have been made by the organizers.

We are eager to have as many students and postdocs as possible to attend since they are likely to benefit most from this meeting. We have applied for funds from industry and foundations to partially support graduate students and postdocs. Apply in writing stating need for financial support to Catie Carr at carr@cshl.edu. Preference is given to those submitting abstracts. 

All questions pertaining to registration, fees, abstract submission or any other matters may be directed to Catie Carr at carr@cshl.edu.

We anticipate the following support :

National Science Foundation

Social Media:

The designated hashtag for this meeting is #cshlNeuroAI. Note that you must obtain permission from an individual presenter before live-tweeting or discussing his/her talk, poster, or research results on social media. Click the Policies tab above to see our full Confidentiality & Reporting Policy.


Pricing:

Virtual Academic Package: $285
Virtual Graduate Student Package: $175
Virtual Corporate Package: $425

Lab Group Discounts (not departmental or institutional discounts):

Labs registering 4 people: 20% discount off applicable fees
Labs registering 5 people: 25% discount off applicable fees
Labs registering 6 people: 30% discount off applicable fees

To be eligible for lab group discounts, please submit a list of lab members planning to attend in advance of registration to Catie Carr  to establish appropriate discounted fees. Please include a link to your lab web page for verification purposes. Prior payments will be included in the group discount calculation.

IBRO/International Brain Research Organization are generously supporting the participation of a limited number of individuals from US/Canadian Minority Serving Institutions (check eligibility): $25
Limit: 65 attendees / limit per institution: 5 (contact Catie Carr  to confirm eligibility/availability prior to registering) 

Reduced Pricing for Individuals from US/Canadian Minority Serving Institutions (check eligibility): $50

Abstract: This dissertation addresses the methodological disconnect between Natural Language Processing (NLP) and human-centric analysis by shifting the unit of analysis from document to human behavior in two broad respects: (i) time-ordering: modeling documents as sequential person-indexed behavioral observations, and (ii) person-level semantics: evaluation and explainability of models by their latent structure of psychological constructs rather than just its predictive accuracy against narrow proxy measures. First, we consider the most basic implication of language as a person's behaviors when measuring their psychological constructs: relationship between language sample size and model's predictive performance. We empirically show that the state-of-the-art transformers are often over-parameterized for typical NLP dataset sizes and can be reduced in dimensionality without performance loss. Establishing the author as the unit of analysis naturally allows us to treat their behavior as a time-ordered sequence. Second, we introduce a longitudinal evaluation framework that establishes ecologically valid evaluation settings, namely, cross-sectional and prospective generalization, and separates error measurement of the model into within-person dynamics and between-person differences. We demonstrate that traditional NLP evaluations based on random document splits can yield reversed conclusions under ecologically valid generalization settings. To address this, we develop models that capture the trajectory of mental states (e.g., mood shifts) rather than static traits. Third, moving into person-level semantics, we evaluate the latent structure of large language models using a novel machine behavior analytic framework. We find that while GPT-4 achieves high predictive correlation with self-reports, its latent symptoms structure diverges from clinical understanding. Finally, we propose a method for modeling multidimensional behaviors, embedding concurrent behavioral signals alongside language to predict future states. Taken together, this work suggests that operationalizing language as behavior advances NLP methods into a rigorous instrument for valid psychological inquiry.

Speaker: Adithya Ganesan

Location: Join Zoom Meeting (ID: 99021939129, Passcode: 569493)