Abstract: Pretraining vision encoders with self-supervision (SSL) leads to stronger representations that excel across diverse downstream tasks. One of the key factors enabling self-supervision is extracting multiple views of the same scene to formulate either: 1) View-invariant pretraining (DINO, SimCLR, iBOT), where the objective is predicting the same representation for different views of the scene; or 2) Cross-view pretraining (cross-view Masked Autoencoders), where the objective is predicting missing parts of one view using other views. For extracting multiple views, view-invariant methods rely on a combination of handcrafted augmentations (random cropping, color jittering, gaussian blur, etc.) of the same image, whereas cross-view pretraining methods rely on image cropping or video frames. In this work, we present methods to effectively incorporate synthetic views from diffusion models into SSL training.
For view-invariant pretraining, we introduce Gen-SIS, a method that leverages the ability of diffusion models to generate interpolated images through interpolation in conditioning space. We introduce a disentanglement pretext task: disentangling two source images from an interpolated synthetic image. This disentanglement task, in addition to vanilla single-source generative augmentation for view extraction, improves visual pretraining of various view-invariant methods (DINO, SimCLR, iBOT).
For cross-view pretraining, we introduce CDG-MAE, a novel cross-view masked autoencoder (MAE) based method that uses diverse synthetic views generated from static images via an image-conditioned diffusion model to learn dense correspondences. We present a quantitative method to evaluate the local and global consistency of the generated views to choose the right diffusion model for cross-view pretraining. These generated views exhibit substantial changes in pose and perspective, providing a rich training signal that overcomes the limitations of video (expensive) and crop-based (less variation) methods. CDG-MAE substantially narrows the gap to video-based MAE methods on video label propagation tasks while maintaining the data advantages of image-only MAEs.

Speaker: Varun Belagali

Location: NCS 120
Zoom: https://stonybrook.zoom.us/j/93647452432?pwd=hZaX7LXCAD8KPHWYE1Afw2sDI3owpv.1

The Provost's Office is excited to invite you to join in responding to an extraordinary opportunity to enhance our academic and research capabilities in AI at Stony Brook. SUNY recently made funding available to support the creation of departments of AI and Society at its universities. Stony Brook is well-positioned to seize this opportunity to build upon our interdisciplinary strengths in AI.

The office is hosting a forum on Friday, Nov. 15, from 11:30 a.m. to 1:30 p.m., in Ballroom A, SAC. You are invited to attend to learn more about this opportunity and to help us generate ideas to build a compelling proposal for Stony Brook to submit to SUNY. Lunch will be provided.

Please click here to RSVP as soon as possible.

This funding will support innovation in our curriculum, allowing us to create programs that explore the social and societal impact of AI alongside the technological advancements led by researchers in engineering and scientific disciplines.

We believe we can make a significant impact through this SUNY program and look forward to your participation in this initiative.

Zoom Link: https://stonybrook.zoom.us/j/98533029054?pwd=5FXO6lWGTJssCADEYkYbA7sjaacPRX.1

Meeting ID: 985 3302 9054

Passcode: 436997

Abstract:

Semantic segmentation, the task of assigning a semantic label to each pixel in an image, is a fundamental problem in the field of Computer Vision. with crucial applications in domains like autonomous driving, drone imagery and medical image analysis. Despite advancements in deep learning architectures, state-of-the-art models still heavily depend on large-scale pixel-level annotations, which are costly and time-consuming to acquire. To address this issue, Semi-Supervised Segmentation (SSS) has emerged as a promising solution, leveraging a small set of labeled images alongside a larger corpus of unlabeled data to reduce the annotation burden. In this proposal, I aim to investigate the challenges of SSS and propose approaches to address them. Existing SSS methods rely on a teacher-student framework to generate pseudo-labels for unlabeled images, which are then used for model training. However, this approach presents two major challenges. Pixel-level consistency fails to effectively capture contextual information, and pseudo-labels are noisy, especially in the early stages of training. To address the challenge of noisy pseudo-labels, existing methods rely on confidence-based thresholding to identify reliable pseudo-labels. However, during early training phases, when the model is poorly calibrated, this approach can select high-confidence but noisy pseudo-labels. To address this, we propose a novel approach that reduces reliance on model confidence to select reliable pseudo-labels. Our method employs an ensemble of a segmentation model and an object detection model to select more reliable pseudo-labels, which are then used to weight pseudo-labels using rank statistics, reducing the influence of noisy labels in training. Next, to address both the challenge of capturing contextual information and noisy pseudo-labels I introduce a novel Multi-scale Patch-based Multi-label Classifier (MPMC), which incorporates patch-level contextual information and reduces the impact of noisy pixel pseudo-labels by using the predictions of the patch-level Multi-label classifier to detect noisy labels, enhancing overall segmentation performance. While my work so far has focused on effectively utilizing unlabeled data to improve segmentation performance, as part of our future work, I will explore the use of textual information, such as category descriptions, for segmentation tasks. In limited labeled data scenarios it is more challenging to align visual features with textual features from large language models (LLMs).

Abstract: Artificial Intelligence (AI) is no longer a futuristic concept -- it is here, but its development, benefits, and risks remain unevenly distributed across industries, nations, and social groups. In this talk, Jieshu presents her research on the societal dimensions of AI from two perspectives: the forces shaping AI's development (backward-looking) and its current and potential impact on society (forward-looking). She first examines disparities in AI, including women's underrepresentation in AI patents and the geographic concentration of AI innovation, highlighting inequalities in who creates AI and who benefits from it. She then explores AI's societal impact, focusing on workforce transformation and the need for GenAI literacy. She will also discuss AI patents, AI's role in climate change mitigation and adaptation, potential environmental biases in LLMs, and gender-specific patterns in AI portrayals in science fiction.

Bio: Jieshu Wang is a Postdoctoral Research Scholar at Arizona State University (ASU), focusing on the social dimensions of artificial intelligence (AI). With a background in engineering, economics, communication, and science and technology studies, she examines how AI both shapes and is shaped by broader societal forces. Her research employs interdisciplinary methods to explore the social, political, and economic factors influencing AI development, as well as its role in innovation, the economy, the future of work, climate change mitigation, and popular culture. Jieshu holds a Ph.D. in Human and Social Dimensions of Science and Technology from ASU. She is also a science book translator and has translated six books.

Location: Old Computer Science, room 1310
Abstract: Jailbreak attacks circumvent LLMs' built-in safeguards by concealing harmful queries within adversarial prompts. While most existing defenses attempt to mitigate the effects of adversarial prompts, they often prove inadequate as adversarial prompts can take arbitrary, adaptive forms. This paper introduces RobustKV, a novel jailbreak defense that takes a fundamentally different approach by selectively removing critical tokens of harmful queries from key-value (KV) caches. Intuitively, for an adversarial prompt to be effective, its tokens must achieve sufficient `importance' (measured by attention scores), which consequently lowers the importance of tokens in the concealed harmful query. Therefore, by carefully evicting the KVs of low-ranked tokens, RobustKV minimizes the harmful query's presence in the KV cache, thus preventing the LLM from generating informative responses. Extensive evaluation using benchmark datasets and models demonstrates that RobustKV effectively counters state-of-the-art jailbreak attacks while maintaining the LLM's performance on benign queries. Notably, RobustKV creates an interesting effectiveness-evasiveness dilemma for the adversary, leading to its robustness against adaptive attacks.

Speaker: Tanqiu Jiang

Where: NCS 220 and Zoom (https://stonybrook.zoom.us/j/6406956411)
Join librarian Christine Fena for an interactive workshop that invites you to explore AI tools firsthand, not just as users, but as critical investigators. Through playful experimentation and collaborative discovery, you'll uncover inherent biases, probe algorithmic flaws, and gain a deeper understanding of AI's limitations and societal impacts.

Register for the Zoom workshop here.
The Division of Educational & Institutional Effectiveness is excited to host International Love Data Week at SBU, February 9-13, 2026!
Join us for a mix of 30-minute virtual sessions, an in-person kickoff on Monday, and a student-focused event on Wednesday celebrating data and data-informed decision-making.
Wrap up the week at the Love Data Week Open House on Friday, 2/13 with light refreshments, data-themed swag, photos with Wolfie, and time to connect with presenters.
Learn more and register on https://www.stonybrook.edu/commcms/oee/recognition/Love%20Data%20Week%202026%20Save%20the%20Date%20Placeholder.php
Abstract: Visual generation is a fundamental problem in computer vision and graphics, with applications ranging from 3D capture to image and video synthesis. Despite rapid progress in neural rendering and generative models, efficiency remains a key bottleneck: high-quality 3D reconstruction often relies on dense multi-view supervision; high-fidelity 3D synthesis requires costly optimization, training, and rendering; and modern image and video generators require substantial computation as the number of tokens grows rapidly for high-resolution generation. This dissertation focuses on efficient visual generation by improving sample efficiency in 3D reconstruction, representation efficiency in 3D generation, and computational efficiency in image and video synthesis. First, we improve the sample efficiency of neural implicit surface reconstruction. We integrate multi-view stereo probability volumes as a geometric regularizer, enabling high-quality sparse-view reconstruction from as few as three input images. Next, we introduce an explicit 3D representation for 3D generation, built from multi-view depth and RGB images with 3D Gaussian features. This design allows the model to use 2D generative priors while enforcing multi-view consistency through epipolar attention. We then address the computational bottleneck in image and video synthesis with importance-based token merging. Our method uses importance signals available during generation to preserve critical information while merging redundant tokens. Finally, we enable efficient mixed-resolution diffusion transformers via phase-aligned attention. This approach stabilizes attention under mixed-resolution token grids and unlocks high-fidelity image and video generation at reduced cost. Taken together, these contributions reduce the data requirements, representational overhead, and computational demands, thereby providing a foundation for high-quality, scalable, and efficient visual generation.

Speaker: Haoyu Wu

Location: NCS 120