In this dissertation, we address the problem of learning neural representations of humans in a holistic way. Given that the video data in the real world include multiple modalities (e.g., audio and video) and multiple identities, we develop multi-modal and multi-identity representations. First, we propose to reconstruct the 4D face geometry of humans by leveraging both audio and video information. In this way, the network produces accurate lip shapes and is robust to cases when either modality is insufficient. Next, we introduce a NeRF-based representation for audio-driven human face animation that achieves high-quality lip synchronization for cinematic content. Since humans communicate with their full body, combining body pose, hand gestures, and facial expressions, we extend the network to capture full-body human motion for multiple identities simultaneously. In order to better disentangle identity and non-identity specific information, we subsequently study non-linear interactions between latent factors of variation, and propose a specific multiplicative module. In this way, we learn a multi-identity NeRF that robustly animates human faces under novel expressions and achieves a significant decrease in the total training time. Similarly, we propose a multi-identity Gaussian splatting representation for human bodies, by constructing a high-order tensor. Assuming a low-rank structure, we learn a tensor decomposition that leads to a significant decrease in the total number of learnable parameters, as well as to a robust animation under novel poses. Last but not least, we propose to jointly synthesize audio and visual outputs from just text input. Given the recent rise of large language models, coupling text with natural-looking avatars can enhance the overall interaction between a human and an AI system.
Location: NCS 220 or Zoom
passcode: 045476
1. Explain the clinical radiology workflow, and highlight how AI is currently in use to impact each step
2. Describe how radiologists interact with the currently available tools, highlighting both positive andnegative examples
3. Offer a brief description of how these tools are approved, validated, and reimbursed
4. Explore the utility of cutting edge AI techniques in diagnostic radiology
Speaker:
Dr. David Payne, MD Neuroradiologist and Assistant Professor, Rush University Medical Centre
Remote Access:
Zoom: https://stonybrook.zoom.us/j/95617197636?pwd=KytzZ2pVRG9SZGpKZUtpNXJISjNjZz09
Meeting ID: 95617197636
Passcode: 924293
Learn how to summarize docs with AI, output a PowerPoint from AI, & Create professional visuals
Unlock greater efficiency and impact in your university role with AI productivity tools. This workshop is your introduction to a few ways that I have found to make our daily tasks more efficient. Discover how easily you can create presentations (that outputs to a PowerPoint format), summarize content using AI, and get information from images. These AI tool tips are invaluable resources designed to streamline your work processes. Start working smarter today!
In this session, you will
- Summarize docs with AI
- Output a PowerPoint from AI
- Gather information from visuals
Register here.
The SUNY AI Symposium brings together AI experts from across the state, in Western New York and around the country.
This two-day event showcases AI thought leaders, SUNY researchers, students and companies of all sizes who leverage AI to produce positive outcomes--with scientific discovery, business innovation and economic impact. Come curious, explore the fascinating world of AI and leave with connections to those at the forefront of innovation.
Get hands-on with data cleaning techniques using Python and AI tools. Join SBU Libraries' Data Literacies Lead, Ahmad Pratama, to learn how to identify and rectify errors, handle missing data, and prepare your dataset for analysis. This workshop introduces you to powerful yet easy-to-use tools and techniques that make data cleaning efficient and effective, turning chaotic data into valuable insights.
Please register for the Data Cleaning with Python and AI here.
Abstract: Spectral decompositions such as singular value decompositions (SVDs) and eigenvalue decompositions (EVDs) are central tools across a vast swath of scientific computing and machine learning, with abundant engineering applications. Yet many modern methods for learning such decompositions in high dimensions struggle with instability, bias, and poor scalability, even when approximation power is not the limiting factor. I argue that these difficulties are not intrinsic to spectral problems, but instead arise from a shared reliance on Rayleigh-quotient-based constrained optimization, which forces explicit orthogonality handling through penalties, normalization, or whitening.
To address these challenges, I present a reformulation based on unconstrained variational objectives that implicitly encode spectral structure, eliminating the need for orthogonalization and ad-hoc regularization. This perspective leads to a conceptually simpler and scalable parametric framework for learning ordered spectral representations via nested optimization. The resulting framework is well matched to diverse settings in science and engineering. As examples, I demonstrate its effectiveness on eigenvalue problems for linear PDEs such as the Schrödinger equation, spectral (Koopman) analysis of nonlinear dynamical systems such as molecular dynamics, and structured representation learning with deep neural nets. Collectively, these examples illustrate how abandoning Rayleigh-quotient-based formulations resolves long-standing optimization pathologies across domains.
Bio: Jongha (Jon) Ryu is a postdoctoral associate at MIT EECS. He received his Ph.D. in Electrical and Computer Engineering from UC San Diego. His research develops statistical and mathematical foundations for scientific machine learning, with a focus on scalable spectral methods, efficient generative modeling, and reliable uncertainty quantification for scientific and engineering systems.
Location: NCS 120
The overall purpose of this seminar is to bring together people with interests in Computer Vision theory and techniques and to examine current research issues. This course will be appropriate for people who already took a Computer Vision graduate course or already had research experience in Computer Vision. To enroll in this course, you must either: (1) be in the PhD program or (2) receive permission from the instructors.
Each seminar will consist of multiple short talks (around 10 minutes) by multiple people. Students can register for 1 credit for CSE656. Registered students must attend and present a minimum of 2 or 3 talks. Everyone else is welcome to attend. Fill in https://forms.gle/pCVXovgfMfQwGqG38 to subscribe to our mailing list for further announcement.
Stony Brook University Libraries invites students, faculty, & staff to join a conversation about how AI is transforming the private sector workforce. As AI tools move from experimentation to everyday business use, companies are rethinking roles, skill sets, leadership, and long-term strategy. This discussion-based event will focus on the fast-paced changes and directions at tech companies and their possible impact. This event will be particularly relevant for students preparing for an AI influenced job market and how to position themselves for opportunities in a rapidly evolving professional landscape.
The discussion will be led by Tariq Khan, Senior Director of Private Cloud Solutions at Hewlett Packard Enterprise. Tariq is a technology leader and architect with experience across private cloud, hybrid cloud, and data center platforms. He is responsible for shaping the technology architecture and strategic direction of HPE's Private Cloud offerings across on premises and cloud integrated environments.
Light refreshments will be served.
Location: Melville Library, NRR, Learning Lab