AI Innovation Institute

Abstract: Computer vision seeks to extract semantic and geometric information from images and videos, serving as the perceptual foundation for intelligent systems such as robots and autonomous vehicles. Over the past decade, deep learning has driven remarkable progress in the field, advancing capabilities from 2D recognition to 3D reconstruction. However, the current purely data-driven paradigm faces fundamental challenges, including data inefficiency, curse of high dimensionality, and limited understanding of visual entities beyond individual objects.

In this talk, I will present my recent research on modeling and learning rich visual structures to address these challenges. First, I will introduce a novel framework that integrates explicit visual dependency modeling with deep learning for 2D and 3D dense prediction. Next, I will demonstrate how unfolding the manifold structure of visual data enables unsupervised semantic segmentation. Finally, I will present a recent project that represents, parses, and learns the geometric compositionality of 3D objects to facilitate self-supervised part-whole reconstruction. Through these efforts, I aim to bridge the gap between data-driven deep learning and visual structure modeling, paving the way for more efficient, generalizable, and interpretable computer vision models.

Bio: Dr. Wei Tang is an Assistant Professor in the Department of Computer Science at the University of Illinois Chicago (UIC). He obtained his Ph.D. in Electrical Engineering from Northwestern University, where his dissertation was honored with a Best Dissertation Award. His research interests include computer vision, digital image processing, and machine learning. Dr. Tang has served as an associate editor for several international journals, including Pattern Recognition and Machine Vision and Applications, and as an area chair for leading conferences, including CVPR, ICCV, and WACV. His research has been funded by the National Science Foundation (NSF) and industry partners such as Motorola and Wormpex AI Research.


Location: NCS 115

Zoom: https://stonybrook.zoom.us/j/4624091659?omn=95178138684&jst=3

Abstract:

Photorealistic editing of human facial expressions and head articulations remains a long-standing topic in the computer graphics and computer vision community. Methods enabling such control have great potential in AR/VR applications where a 3D immersive experience is valuable, especially when this control extends to novel views of the scene in which the human subject appears. Traditionally, 3D Morphable Face Models (3DMMs) have been used to control the facial expressions and head pose of a human head. However, the PCA-based shape and expression spaces of 3DMMs lack the expressivity. They cannot model essential elements of the human head such as hair, skin details, and accessories such as glasses that are paramount for realistic reanimation. In this thesis, we present a set of methods that enables facial reanimation, starting from editing expressions in still face images to creating fully controllable neural 3D portraits with control over facial expressions, head pose, and viewing direction of the scene using only casually captured monocular videos from a smartphone to finally achieving studio-like quality from the said monocular captures.
First, we propose a method for editing facial expressions in near-frontal facial images through the unsupervised disentangling of expression-induced deformations and texture changes. Next, we extend facial expression editing to human subjects in 3D scenes. We represent the scene and the subject in it using a semantically guided neural field. This enables control over the subject's facial expressions and the viewing direction of the scene they're in. We then present a method that learns, in an unsupervised manner, to deform static 3D neural fields using facial expression and head-pose dependent deformations, enabling control over facial expressions and head pose of the subject along with the viewing direction of the 3D scene they're in. Next, we propose a method that makes the learning of the aforementioned deformation field robust to strong illumination effects, which adversely impact the registration of the deformation. We then propose an extension of this unsupervised deformation model to 3D Gaussian splatting by constraining it using a 3D morphable model, resulting in a rendering speed of 18 FPS--a 100x speed improvement over prior work. Finally, we propose a method that bridges the quality gap between 3D portraits created using in-the-wild monocular data and multi-view studio capture data. We accomplish this using a two-stage method. First, we train a StyleGAN to relight and inpaint in-the-wild face texture maps (with strong illumination effects and incompletely captured regions). Next, we both reconstruct and generate identity-specific facial details that may be poorly captured in the in-the-wild captures. Once trained, we can generate studio-like complete avatars from monocular phone captures.

Speaker: Shahrukh Athar

Zoom Link:
https://stonybrook.zoom.us/j/94228500743?pwd=RqOBgG6tbJkKaFBlWFwBkYFX0VRovV.1

Meeting ID: 94228500743
Passcode: 661599

Abstract: This talk shows how machine learning can address challenges in Astrophysics. We specifically focus on black hole simulations and supernova observations. First, we present a super-resolution technique for black hole simulations that avoids the need for high-resolution labels by leveraging the Hamiltonian and momentum constraints from general relativity. This method reduces constraint violations by one to two orders of magnitude. Next, we introduce Maven, a multimodal foundation model for supernova science. Using contrastive learning to align photometric and spectroscopic data, Maven achieves state-of-the-art results in classification and redshift estimation by pre-training on synthetic data and fine-tuning on real observations.

Bio: Thomas Helfer is a computational physicist specializing in deep learning and physics. Currently based at the Institute for Advanced Computational Science at Stony Brook University, Thomas was previously a postdoctoral fellow at Johns Hopkins and did his PhD with Eugene Lim at King's College in London. In his work, he looks to bridge topics; in his PhD, he bridged theoretical particle physics and gravitational waves. Now, in his postdoctoral work, he aims to find novel applications of deep learning in astrophysics.

*please note: this seminar will be held in a hybrid format*


Location: IACS Seminar Room OR Join Zoom Meeting
https://stonybrook.zoom.us/j/98617630652?pwd=tb4hplPgb3bTTifPCJTCcsn3P9vX8y.1

Meeting ID: 986 1763 0652
Passcode: 882994

The Natural Language Processing Reading Group at Stony Brook University meets weekly to discuss recent research papers in NLP and related fields.
Join the Google Group here.

AI can help you write, you hear. AI can save you time, leverage your skills, enhance your productivity. . . . But you also hear: AI output is not reliable, not adequate for advanced tasks/learning, not ethical to use -- you could get in deep trouble for using AI tools without adequate mastery and caution. Which way is it?
Come join this hands-on workshop where you will explore AI tools and their affordances. Engage in writing tasks to learn how to use AI tools effectively and responsibly.
Sign up for a seat now: https://docs.google.com/forms/d/e/1FAIpQLSd0iDTKkTYnkxFd4LkgqbtP97zQSS4FI_MiPVm7p6IY5SGwSg/viewform

This symposium will highlight how artificial intelligence (AI) can assist in dementia detection, research and clinical care. For example, the use of robotics to assist with dementia care therapy is truly inspirational and cutting-edge for clinicians, trainees and the community at large, including assisted living facilities. The symposium will also focus on the role of AI in early detection of dementia and in identifying characteristics associated with future cognitive decline.

Learn more and register at https://cme.stonybrookmedicine.edu/continuing-medical-education/conferences/233/alzheimers-symposium-ai-the-future-of-dementia-care-2024/11/15/2024

Engineering Job & Internship Fair hosted by the Career Center Learn more and RSVP on Handshake.