Join the Office of Educational Effectiveness' upcoming workshop on the transformative potential of AI tools to enhance program assessment. Learn how to leverage AI to create targeted learning objectives, detailed rubrics, and precise benchmarks that will elevate the quality and effectiveness of your program assessment process. Join in-person on Oct. 17 at 10:30 am or virtually on Oct. 21 at 12 pm.

Register in advance: https://calendar.stonybrook.edu/site/office-educational-effectiveness/event/leveraging-ai-in-assessment-zoom/
Abstract: Capturing the spatio-temporal (4D) dynamics of humans has been a long standing research problem in computer vision and graphics. Synthesizing photorealistic human avatars has broad applications, ranging from immersive telepresence in AR/VR and the movie industry, to enriching the education and healthcare systems. Earlier approaches relied on hand-engineered models that use a small amount of data from one or more subjects. With the advent of neural networks, training on large datasets enhanced the output visual quality. Currently, the combination of neural networks with graphics techniques has achieved natural-looking human animation. However, most approaches are identity-specific, trained only on a single identity, and use only one modality.

In this dissertation, we address the problem of learning neural representations of humans in a holistic way. Given that the video data in the real world include multiple modalities (e.g., audio and video) and multiple identities, we develop multi-modal and multi-identity representations. First, we propose to reconstruct the 4D face geometry of humans by leveraging both audio and video information. In this way, the network produces accurate lip shapes and is robust to cases when either modality is insufficient. Next, we introduce a NeRF-based representation for audio-driven human face animation that achieves high-quality lip synchronization for cinematic content. Since humans communicate with their full body, combining body pose, hand gestures, and facial expressions, we extend the network to capture full-body human motion for multiple identities simultaneously. In order to better disentangle identity and non-identity specific information, we subsequently study non-linear interactions between latent factors of variation, and propose a specific multiplicative module. In this way, we learn a multi-identity NeRF that robustly animates human faces under novel expressions and achieves a significant decrease in the total training time. Similarly, we propose a multi-identity Gaussian splatting representation for human bodies, by constructing a high-order tensor. Assuming a low-rank structure, we learn a tensor decomposition that leads to a significant decrease in the total number of learnable parameters, as well as to a robust animation under novel poses. Last but not least, we propose to jointly synthesize audio and visual outputs from just text input. Given the recent rise of large language models, coupling text with natural-looking avatars can enhance the overall interaction between a human and an AI system.

Location: NCS 220 or Zoom

CSE 600 Seminar Series | Fall 2025


Abstract: Virtual worlds are prevalent in applications ranging from entertainment, healthcare, retail, to workforce training. With the demand for virtual content growing exponentially, the market for such content is valued at over $200 Billion, which is accelerating the need for advanced computational solutions. In this talk, I will focus on a key challenge in virtual content creation: simulating autonomous agents.
I begin by overviewing this problem domain, through the lens of a physics-based dynamics simulation, which enables the simulation of thousands of agents at interactive rates with GPU programming, achieving a level of performance previously unattainable.
Next, I'll present our recent results in Deep Reinforcement Learning for multi-agent navigation, which enable refined, reward-based strategies to control agent movement. We demonstrate how these techniques can simulate realistic crowds, with broad applications in pedestrians, robots, and swarms. Lastly, I conclude my talk by discussing our lab's work-at-large and the wide range of research opportunities in this emerging area.

Speaker: Tomer Weiss is a professor with New Jersey Institute of Technology since 2020. He received the best student, presentation, and best paper awards in various ACM SIGGRAPH conferences for his work on simulating multi-agent crowds. He was also a finalist in both ACM SIGGRAPH Thesis Fast Forward, and the ACM SIGGRAPH Asia Doctoral Symposium in 2018. He received his PhD in computer science from UCLA in 2018. His research interests include multi-agent dynamics, scene understanding, and interactive visual computing.
Title: Sustainable NLP

Time: Friday 4/29, 2:40 PM

Location: NCS 120

Abstract:


Natural language processing (NLP) technology has supercharged many real-world applications ranging from intelligent personal assistants (like Alexa, Siri, and Google Assistant) to commercial search engines such as Google and Bing. But current NLP applications use extremely large neural models, making them (i) expensive to deploy on servers, requiring large amounts of compute resources and power, and (ii) impossible to run on mobile devices, making on-device, privacy-preserving applications impractical.

In the first part of the talk, I will describe systems optimizations we have developed that significantly reduce the compute and memory requirement of NLP models. The optimizations we developed can be applied broadly and results in over 10x reduction in latency when deployed on mobile devices. In the second part of the talk, I will describe our recent work on predicting energy consumption of NLP models. Existing energy prediction approaches are not accurate, making it difficult for developers and practitioners to reason about their models in terms of power. We use a multi-level regression approach that produces highly accurate and interpretable energy predictions.



Bio:
Aruna Balasubramanian is an Associate Professor at Stony Brook University. She received her Ph.D from the University of Massachusetts Amherst, where her dissertation won the UMass outstanding dissertation award and was the SIGCOMM dissertation award runner up. She works in the area of networked systems. Her current work consists of two threads: (1) significantly improving Quality of Experience of Internet applications, and (2) improving the usability, accessibility, and privacy of mobile systems. She is the recipient of the SIGMobile Rockstar award, a Ubicomp best paper award, a Computing Innovation Fellowship, a VMWare Early Career award, several Google research awards, an
Nam Nguyen

4-5pm, Dec 17 2020

https://stonybrook.zoom.us/j/94214254415?pwd=K1VoQml4cFdlVW51VW41dWtid2tJdz09



The molecular mechanisms and functions in complex biological systems
currently remain elusive. Recent high-throughput techniques, such as
next-generation sequencing, have generated a wide variety of
multiomics datasets that enable the identification of biological
functions and mechanisms via multiple facets. However, integrating
these large-scale multiomics data and discovering functional insights
are, nevertheless, challenging tasks. To address these challenges,
machine learning has been broadly applied to analyze multiomics. In
particular, multiview learning is more effective than previous
integrative methods for learning data's heterogeneity and revealing
cross-talk patterns. Although it has been applied to various contexts,
such as computer vision and speech recognition, multiview learning has
not yet been widely applied to biological data--specifically,
multiomics data. Therefore, we have developed a framework called
multiview empirical risk minimization (MV-ERM) for unifying multiview
learning methods (Nguyen, et al., PLoS Computational Biology, 2020).
MV-ERM enables potential applications to understand multiomics
including genomics, transcriptomics, and epigenomics, in an aim to
discover the functional and mechanistic interpretations across omics.
Based on MV-ERM, we have developed the following methods:
ManiNetCluster, Varmole and ECMarker.



(1) ManiNetCluster (Nguyen, et al., BMC Genomics, 2019) is a manifold
learning method which simultaneously aligns and clusters gene networks
(e.g., co-expression) to systematically reveal the links of genomic
function between different phenotypes. Specifically, ManiNetCluster
employs manifold alignment to uncover and match local and non-linear
structures among networks, and identifies cross-network functional
links. We demonstrated that ManiNetCluster better aligns the
orthologous genes from their developmental expression profiles across
model organisms than state-of-the-art methods. This indicates the
potential non-linear interactions of evolutionarily conserved genes
across species in development. Furthermore, we applied ManiNetCluster
to time series transcriptome data measured in the green alga
Chlamydomonas reinhardtii to discover the genomic functions linking
various metabolic processes between the light and dark periods of a
diurnally cycling culture;



(2) Varmole (Nguyen, et al., Bioinformatics, 2020) is an interpretable
deep learning method that simultaneously reveals genomic functions and
mechanisms while predicting phenotype from genotype. In particular,
Varmole embeds multi-omic networks into a deep neural network
architecture and prioritizes variants, genes and regulatory linkages
via biological drop-connect without needing prior feature selections.
With an application to schizophonia, we demonstrate that Varmole
provides an effective alternative for recent statistical methods that
associate functional omic data (e.g. gene expression) with genotype
and phenotype and that link variants to individual genes in population
studies such as genome-wide association study;



(3) ECMarker (Jin*, Nguyen*, et al., Bioinformatics, 2020) is an
interpretable and scalable machine learning model that predicts gene
expression biomarkers for disease phenotypes and simultaneously
reveals underlying regulatory mechanisms. Particularly, ECMarker is
built on the integration of semi- and discriminative- restricted
Boltzmann machines, a neural network model for classification allowing
lateral connections at the input gene layer. With application to the
gene expression data of non-small cell lung cancer (NSCLC) patients,
we found that ECMarker not only achieved a relatively high accuracy
for predicting cancer stages but also identified the biomarker genes
and gene networks implying the regulatory mechanisms in lung cancer
development.



Finally, we propose a novel multiview learning method, Malignomics, to
predict phenotypes from heterogeneous multi-omic features. Malignomics
will first align multi-omic features by deep manifold alignment onto a
common latent space, better predicting nonlinear relationships across
omics. This deep alignment aims to preserve both global consistency
and local smoothness across omics and reveal higher-order nonlinear
interactions (i.e., manifolds) among cross-omic features. Second, it
uses these manifold structures to regularize the classifiers for
predicting phenotypes. This manifold-regularization allows
highlighting cross-omic feature manifolds and prioritizing the
features and interactions for the phenotypes. The prioritized
multi-omic features will further reveal underlying phenotypic
functions and mechanisms and thus enhance the biological
interpretation of Malignomics. We will apply Malignomics to
multi-omics data in neuropsychiatric disorders, and prioritize gene
regulatory networks linking risk variants, regulatory elements, and
genes for the disorders. We will also compare Malignomics with the
state-of-the-arts, and investigate how the manifold regulation will
potentially improve understanding of multi-omics functions and
predicting diseases.

Description:

Curious about what AI image generation tools are out there and how they work? Come down to the library Galleria space (outside the Central Reading Room) to see some demonstrations and learn more about them.

Librarians Chris Kretz and Ahmad Pratama, along with David Ecker of DoIT, will be hosting Explore AI demos from Monday - Wednesday this week on different topics. Whether you're new to AI or an experienced user, stop by and take a look!

Location: Library Galleria

University Libraries Present: Analyzing quantitative data can feel overwhelming without the right tools. In this workshop, SBU Libraries' Data Literacies Lead, Ahmad Pratama will show you how to master the basics of exploratory data analysis for quantitative data using Python. This workshop covers several techniques to help you uncover patterns and insights in your datasets.

Online RSVP via link: https://stonybrook.zoom.us/meeting/register/vEPycmDrQoGjFqkmsYHgxw
The International Neuroethics Society (INS) Speaker Series on AI & Consciousness

Abstract: Colln Allen and I noted in our 2008 book Moral Machines: Teaching Robots Right From Wrong, that consciousness, a theory of mind, sociability, situational awareness and embodiment are all supra-rational (beyond reason) capabilities that contribute to making ethical decision Whether any of these can be fully instantiated in machines remains an open question. Nevertheless, moral decision making in the digital age will require an evolution in and refinement of specific skills for both humans and for AI. I call one of these evolutions in moral decision making capabilities tradeoff ethics and another a silent ethics. Aspects of this social, and not just technological evolution, will require research by neuroscientists.

Speaker Bio: Wendell Wallach has an international reputation as an expert on the ethics and governance of emerging technologies, particularly artificial intelligence and biotechnologies. He is also senior advisor to The Hastings Center and a scholar at the Yale University Interdisciplinary Center for Bioethics where he chaired Technology and Ethics studies for eleven years. Wallach's latest book, a primer on emerging technologies, is entitled, A Dangerous Master: How to keep technology from slipping beyond our control. He co-authored (with Colin Allen) Moral Machines: Teaching Robots Right From Wrong. Wallach has been referred to as, a Godfather of AI Ethics.

Pre-register here (required): https://umaryland.zoom.us/meeting/register/sPpiR_drR4-9JYDhI2NhJg

The Association for Computational Linguistics is the international scientific and professional society for people working on problems involving natural language and computation. Membership includes the ACL quarterly journals, Computational Linguistics and Transactions of the ACL, reduced registration at most ACL-sponsored conferences, discounts on ACL-sponsored publications, and participation in ACL Special Interest Groups.

An annual meeting is held each summer in locations where significant computational linguistics research is carried out.

For more information and registration, visit the official website.