AI Innovation Institute

I will be holding an informal 2-week short optimization course, to try
to cover a few important proofs in the field. The goal will be depth
over breadth, with focus on:

 - convergence proofs for gradient descent and stochastic gradient descent
 - energy functions and continuous time optimization
 - estimate sequences and Nesterov acceleration

and, time permitting, additional topics like variance reduction,
quasi-Newton methods, and Frank-Wolfe methods. If we go super fast, we
can spend a few days at the end brainstorming interesting research
project ideas.

Details: NCS 220 6:15pm-7:45pm, Monday-Friday, Feb 7-Feb 18.

In person only, since I plan to use the whiteboard (but may be recorded)

More details will be uploaded here (notes, specific schedule):
https://sites.google.com/view/optimization-short-course/home

I will be holding an informal 2-week short optimization course, to try
to cover a few important proofs in the field. The goal will be depth
over breadth, with focus on:

 - convergence proofs for gradient descent and stochastic gradient descent
 - energy functions and continuous time optimization
 - estimate sequences and Nesterov acceleration

and, time permitting, additional topics like variance reduction,
quasi-Newton methods, and Frank-Wolfe methods. If we go super fast, we
can spend a few days at the end brainstorming interesting research
project ideas.

Details: NCS 220 6:15pm-7:45pm, Monday-Friday, Feb 7-Feb 18.

In person only, since I plan to use the whiteboard (but may be recorded)

More details will be uploaded here (notes, specific schedule):
https://sites.google.com/view/optimization-short-course/home

Join us at the Center for Excellence in Learning and Teaching (CELT) for an interactive Zoom workshop on Generative AI designed for faculty and staff interested in enhancing teaching and assessment practices, increasing student engagement, and navigating the rapidly evolving landscape of AI tools. Participants will be introduced to common AI tools, explore potential instructional uses, and discuss key considerations such as academic integrity, transparency, and equity.

Register now: https://stonybrook.zoom.us/meeting/register/6js1eP64T1ys8tyU57EJ7Q#/registration

Speaker TBD Refreshments will be provided

The overall purpose of this seminar is to bring together people with interests in Computer Vision theory and techniques and to examine current research issues. This course will be appropriate for people who already took a Computer Vision graduate course or already had research experience in Computer Vision. To enroll in this course, you must either: (1) be in the PhD program or (2) receive permission from the instructors.

Each seminar will consist of multiple short talks (around 10 minutes) by multiple people. Students can register for 1 credit for CSE 656. Registered students must attend and present a minimum of 2 or 3 talks. Everyone else is welcome to attend. Fill in https://forms.gle/pCVXovgfMfQwGqG38 to subscribe to our mailing list for further announcement.

Abstract:

Photorealistic editing of human facial expressions and head articulations remains a long-standing topic in the computer graphics and computer vision community. Methods enabling such control have great potential in AR/VR applications where a 3D immersive experience is valuable, especially when this control extends to novel views of the scene in which the human subject appears. Traditionally, 3D Morphable Face Models (3DMMs) have been used to control the facial expressions and head pose of a human head. However, the PCA-based shape and expression spaces of 3DMMs lack the expressivity. They cannot model essential elements of the human head such as hair, skin details, and accessories such as glasses that are paramount for realistic reanimation. In this thesis, we present a set of methods that enables facial reanimation, starting from editing expressions in still face images to creating fully controllable neural 3D portraits with control over facial expressions, head pose, and viewing direction of the scene using only casually captured monocular videos from a smartphone to finally achieving studio-like quality from the said monocular captures.
First, we propose a method for editing facial expressions in near-frontal facial images through the unsupervised disentangling of expression-induced deformations and texture changes. Next, we extend facial expression editing to human subjects in 3D scenes. We represent the scene and the subject in it using a semantically guided neural field. This enables control over the subject's facial expressions and the viewing direction of the scene they're in. We then present a method that learns, in an unsupervised manner, to deform static 3D neural fields using facial expression and head-pose dependent deformations, enabling control over facial expressions and head pose of the subject along with the viewing direction of the 3D scene they're in. Next, we propose a method that makes the learning of the aforementioned deformation field robust to strong illumination effects, which adversely impact the registration of the deformation. We then propose an extension of this unsupervised deformation model to 3D Gaussian splatting by constraining it using a 3D morphable model, resulting in a rendering speed of 18 FPS--a 100x speed improvement over prior work. Finally, we propose a method that bridges the quality gap between 3D portraits created using in-the-wild monocular data and multi-view studio capture data. We accomplish this using a two-stage method. First, we train a StyleGAN to relight and inpaint in-the-wild face texture maps (with strong illumination effects and incompletely captured regions). Next, we both reconstruct and generate identity-specific facial details that may be poorly captured in the in-the-wild captures. Once trained, we can generate studio-like complete avatars from monocular phone captures.

Speaker: Shahrukh Athar

Zoom Link:
https://stonybrook.zoom.us/j/94228500743?pwd=RqOBgG6tbJkKaFBlWFwBkYFX0VRovV.1

Meeting ID: 94228500743
Passcode: 661599

The next AI Institute seminar speaker will be Chao Chen of Biomedical Informatics, on Monday November 29 at noon via zoom:

https://stonybrook.zoom.us/j/96233844681?pwd=aVVsUnIzMWJDMHRqVXcrQU5HMjFVQT09

He will be talking on the Detection of Trojan Attacks to Deep Neural Networks - A Topological Perspective, with his abstract and bio below.


Abstract: Deep neural networks are known to have security issues. One particular threat is the Trojan attack. It occurs when the attackers stealthily manipulate the model's behavior through Trojaned training samples, i.e., samples with special trigger injected and labels altered. To identify a Trojaned model at deployment is challenging, due to limited access to the training data. We propose to identify Trojaned neural networks using methods from topological data analysis. In particular, we propose to (1) inspect high-order topological features of the neuron interactions and (2) reverse engineer the injected triggers using a topological loss. These approaches take different angles and reveal insights into the behavior of neural networks when their strong memorialization power is exploited maliciously. The work has been accepted to NeurIPS'21. I will also briefly mention other research directions from my group, including incorporating topological information into deep image analysis, topology-inspired graph neural networks, and robust training of neural networks with label noise. These works have been published in ICLR, ICML, NeurIPS, ECCV, ICCV and AAAI in recent years.
Bio: Dr. Chao Chen is an assistant professor of Biomedical Informatics at Stony Brook University. His research interests span topological data analysis (TDA), machine learning and biomedical image analysis. He develops principled learning methods inspired by the theory from TDA, such as persistent homology and discrete Morse theory. These methods address problems in biomedical image analysis, robust machine learning, and graph neural networks from a unique topological view. His research results have been published in major machine learning, computer vision, and medical image analysis conferences. He is serving as an area chair for MICCAI, AAAI, CVPR and NeurIPS.