AI Innovation Institute

Abstract:

Photorealistic editing of human facial expressions and head articulations remains a long-standing topic in the computer graphics and computer vision community. Methods enabling such control have great potential in AR/VR applications where a 3D immersive experience is valuable, especially when this control extends to novel views of the scene in which the human subject appears. Traditionally, 3D Morphable Face Models (3DMMs) have been used to control the facial expressions and head pose of a human head. However, the PCA-based shape and expression spaces of 3DMMs lack the expressivity. They cannot model essential elements of the human head such as hair, skin details, and accessories such as glasses that are paramount for realistic reanimation. In this thesis, we present a set of methods that enables facial reanimation, starting from editing expressions in still face images to creating fully controllable neural 3D portraits with control over facial expressions, head pose, and viewing direction of the scene using only casually captured monocular videos from a smartphone to finally achieving studio-like quality from the said monocular captures.
First, we propose a method for editing facial expressions in near-frontal facial images through the unsupervised disentangling of expression-induced deformations and texture changes. Next, we extend facial expression editing to human subjects in 3D scenes. We represent the scene and the subject in it using a semantically guided neural field. This enables control over the subject's facial expressions and the viewing direction of the scene they're in. We then present a method that learns, in an unsupervised manner, to deform static 3D neural fields using facial expression and head-pose dependent deformations, enabling control over facial expressions and head pose of the subject along with the viewing direction of the 3D scene they're in. Next, we propose a method that makes the learning of the aforementioned deformation field robust to strong illumination effects, which adversely impact the registration of the deformation. We then propose an extension of this unsupervised deformation model to 3D Gaussian splatting by constraining it using a 3D morphable model, resulting in a rendering speed of 18 FPS--a 100x speed improvement over prior work. Finally, we propose a method that bridges the quality gap between 3D portraits created using in-the-wild monocular data and multi-view studio capture data. We accomplish this using a two-stage method. First, we train a StyleGAN to relight and inpaint in-the-wild face texture maps (with strong illumination effects and incompletely captured regions). Next, we both reconstruct and generate identity-specific facial details that may be poorly captured in the in-the-wild captures. Once trained, we can generate studio-like complete avatars from monocular phone captures.

Speaker: Shahrukh Athar

Zoom Link:
https://stonybrook.zoom.us/j/94228500743?pwd=RqOBgG6tbJkKaFBlWFwBkYFX0VRovV.1

Meeting ID: 94228500743
Passcode: 661599

AI Seminar: Computational Pathology: Deep Learning, Classification and
Predicting the Future  - Joel Saltz

Abstract:  Pathologists have been looking at tissue through microscopes since the 1800s.  During each pathologist's career,  he or she views slides having  roughly 1,000,000,000,000 cells. Deep learning methods are rapidly being developed to assimilate the huge amount of information walked inside of tissue images and to use this information to predict outcomes and responses to treatments.

Stony Brook is a leader in this type of multi-disciplinary work. I will provide an overview of Stony Brook computational Pathology efforts and articulate how these have the potential to create biomedical advances as well as to drive development of new computer science. 


Bio: Dr. Joel Saltz is a leader in research on advanced information technologies for large scale data science and biomedical/scientific research. He has developed innovative pathology informatics methods, including: the first published whole slide virtual microscope system; pioneering pathology computer-aided diagnosis techniques; and methods for decomposing pathology images into features and linking those features to cancer omics, response to treatment and outcome. He has broken new ground in big data through development of the filter-stream based DataCutter system, the map-reduce style Active Data Repository and the inspector-executor runtime compiler framework. He has also been an active contributor in clinical informatics, having developed
predictive models for hospital readmissions, point of care laboratory testing quality assurance systems, decision support systems for electrophoresis interpretation and graphical user interfaces to support clinical data warehouse queries. Dr. Saltz has been a pioneer in establishing the field of biomedical informatics; he founded and built two highly successful departments of biomedical informatics, one at Ohio State University and one at Emory University. In 2013, he came to Stony Brook as Vice President for Clinical Informatics and Founding Department Chair of Biomedical Informatics - to create a living laboratory for biomedical informatics and to create a third unique biomedical informatics department dually housed in the School of Medicine and the College of Engineering. Dr. Saltz is trained both as a computer scientist and as a physician through the MSTP program at Duke University. He has deep experience in computer science, having served on the computer science faculties at Yale University and the University of Maryland. He completed his residency in clinical
pathology at Johns Hopkins University and he is a practicing, board-certified clinical pathologist. 

The Pittsburgh Supercomputing Center is pleased to present a Machine Learning and Big Data workshop.

This workshop will focus on topics including big data analytics and machine learning with Spark, as well as deep learning.

This will be an IN PERSON event hosted by various satellite sites, there WILL NOT be a direct to desktop option for this event. SBU's Institute for Advanced Computational Science (IACS) is one of those satellite sites!

Location: IACS Conference Room #2

Interested applicants must first have an ACCESS ID. If you don't have the ID, please visit this page to create one: ACCESS USER REGISTRATION.

Once you have an ACCESS ID, please login (see top right here) then register here.

This event brings together people with interests in Computer Vision theory and techniques and examines current research issues in the field.

Each seminar consists of multiple short talks (around 15 minutes) by several students.

Join Zoom Meeting:

https://stonybrook.zoom.us/j/93547152068?pwd=WVpoRVgzelBXeloxdXVEakNSb2M5UT09

Meeting ID: 935 4715 2068 | Passcode: 481832

An interactive session to discover how to create ALT text tags from images and create high-impact visuals, from identification to communicating ideas with images.

Discover how to use AI to create ALT text from images as well as identify objects in your environment, and build relatable visuals for high-impact presentations. Images communicate ideas as a way to understand concepts. AI-generated images have helped allow anyone to create these.

In this session, you will

1. Creating image ALT Tags
2. Transform ideas into images that are visually appealing
3. Identify objects from visuals

Register here.

Join your friends at DoIT for a workshop on Zoom's AI Companion

AI Companion is a meeting summary tool that can capture and summarize what is said in a Zoom meeting transcript, eliminating the need for a notetaker. In addition, meeting participants can use AI Companion to ask questions to a chat bot to get clarity on information within a meeting, and they can also use it to create stylistic virtual backgrounds.

Register here for the online session

A new virtual seminar series on Games, Decisions, and Networks will start this Friday. The series aims at bringing together researchers working on foundations and applications of games theory, decision theory, and networks from computer science, control, economics and operation research. 





The advisory board for the series comprises Asu Ozdaglar (MIT), Christos Papadimitriou (Columbia), Drew Fudenberg (MIT), Eva Tardos (Cornell), Matthew O. Jackson (Stanford), Ramesh Johari (Stanford), and Tamer Başar (UIUC). 
The first talk will be given by Costantinos Daskalakis (MIT) on January 22nd at noon ET, titled Equilibrium Computation and the Foundations of Deep Learning. Upcoming speakers include


- Rakesh Vohra (Upenn)
- Sanjeev Goyal (Cambridge)
- Aaron Roth (Upenn)
- Aislinn Bohren (Upenn)
- Jason Marden (UCSB)

and more to be added!

The annual conference on Neural Information Processing Systems is a multi-track interdisciplinary annual meeting that includes invited talks, demonstrations, symposia, and oral and poster presentations of refereed papers. Along with the conference is a professional exposition focusing on machine learning in practice, a series of tutorials, and topical workshops that provide a less formal setting for the exchange of ideas.

For more information and registration, visit the official website.