Abstract: Artificial Intelligence (AI) is no longer a futuristic concept -- it is here, but its development, benefits, and risks remain unevenly distributed across industries, nations, and social groups. In this talk, Jieshu presents her research on the societal dimensions of AI from two perspectives: the forces shaping AI's development (backward-looking) and its current and potential impact on society (forward-looking). She first examines disparities in AI, including women's underrepresentation in AI patents and the geographic concentration of AI innovation, highlighting inequalities in who creates AI and who benefits from it. She then explores AI's societal impact, focusing on workforce transformation and the need for GenAI literacy. She will also discuss AI patents, AI's role in climate change mitigation and adaptation, potential environmental biases in LLMs, and gender-specific patterns in AI portrayals in science fiction.

Bio: Jieshu Wang is a Postdoctoral Research Scholar at Arizona State University (ASU), focusing on the social dimensions of artificial intelligence (AI). With a background in engineering, economics, communication, and science and technology studies, she examines how AI both shapes and is shaped by broader societal forces. Her research employs interdisciplinary methods to explore the social, political, and economic factors influencing AI development, as well as its role in innovation, the economy, the future of work, climate change mitigation, and popular culture. Jieshu holds a Ph.D. in Human and Social Dimensions of Science and Technology from ASU. She is also a science book translator and has translated six books.

Location: Old Computer Science, room 1310
Abstract:

Photorealistic editing of human facial expressions and head articulations remains a long-standing topic in the computer graphics and computer vision community. Methods enabling such control have great potential in AR/VR applications where a 3D immersive experience is valuable, especially when this control extends to novel views of the scene in which the human subject appears. Traditionally, 3D Morphable Face Models (3DMMs) have been used to control the facial expressions and head pose of a human head. However, the PCA-based shape and expression spaces of 3DMMs lack the expressivity. They cannot model essential elements of the human head such as hair, skin details, and accessories such as glasses that are paramount for realistic reanimation. In this thesis, we present a set of methods that enables facial reanimation, starting from editing expressions in still face images to creating fully controllable neural 3D portraits with control over facial expressions, head pose, and viewing direction of the scene using only casually captured monocular videos from a smartphone to finally achieving studio-like quality from the said monocular captures.
First, we propose a method for editing facial expressions in near-frontal facial images through the unsupervised disentangling of expression-induced deformations and texture changes. Next, we extend facial expression editing to human subjects in 3D scenes. We represent the scene and the subject in it using a semantically guided neural field. This enables control over the subject's facial expressions and the viewing direction of the scene they're in. We then present a method that learns, in an unsupervised manner, to deform static 3D neural fields using facial expression and head-pose dependent deformations, enabling control over facial expressions and head pose of the subject along with the viewing direction of the 3D scene they're in. Next, we propose a method that makes the learning of the aforementioned deformation field robust to strong illumination effects, which adversely impact the registration of the deformation. We then propose an extension of this unsupervised deformation model to 3D Gaussian splatting by constraining it using a 3D morphable model, resulting in a rendering speed of 18 FPS--a 100x speed improvement over prior work. Finally, we propose a method that bridges the quality gap between 3D portraits created using in-the-wild monocular data and multi-view studio capture data. We accomplish this using a two-stage method. First, we train a StyleGAN to relight and inpaint in-the-wild face texture maps (with strong illumination effects and incompletely captured regions). Next, we both reconstruct and generate identity-specific facial details that may be poorly captured in the in-the-wild captures. Once trained, we can generate studio-like complete avatars from monocular phone captures.

Speaker: Shahrukh Athar

Zoom Link:
https://stonybrook.zoom.us/j/94228500743?pwd=RqOBgG6tbJkKaFBlWFwBkYFX0VRovV.1

Meeting ID: 94228500743
Passcode: 661599
Learn how these two AI tools will help you this year. AI has been all over, but figuring out the tools that we may use is critical. Background remover of images and a replacement for Google Search may disrupt the industry this year. Learn and refresh your knowledge about these tools.
Abstract: Capturing the spatio-temporal (4D) dynamics of humans has been a long standing research problem in computer vision and graphics. Synthesizing photorealistic human avatars has broad applications, ranging from immersive telepresence in AR/VR and the movie industry, to enriching the education and healthcare systems. Earlier approaches relied on hand-engineered models that use a small amount of data from one or more subjects. With the advent of neural networks, training on large datasets enhanced the output visual quality. Currently, the combination of neural networks with graphics techniques has achieved natural-looking human animation. However, most approaches are identity-specific, trained only on a single identity, and use only one modality.

In this dissertation, we address the problem of learning neural representations of humans in a holistic way. Given that the video data in the real world include multiple modalities (e.g., audio and video) and multiple identities, we develop multi-modal and multi-identity representations. First, we propose to reconstruct the 4D face geometry of humans by leveraging both audio and video information. In this way, the network produces accurate lip shapes and is robust to cases when either modality is insufficient. Next, we introduce a NeRF-based representation for audio-driven human face animation that achieves high-quality lip synchronization for cinematic content. Since humans communicate with their full body, combining body pose, hand gestures, and facial expressions, we extend the network to capture full-body human motion for multiple identities simultaneously. In order to better disentangle identity and non-identity specific information, we subsequently study non-linear interactions between latent factors of variation, and propose a specific multiplicative module. In this way, we learn a multi-identity NeRF that robustly animates human faces under novel expressions and achieves a significant decrease in the total training time. Similarly, we propose a multi-identity Gaussian splatting representation for human bodies, by constructing a high-order tensor. Assuming a low-rank structure, we learn a tensor decomposition that leads to a significant decrease in the total number of learnable parameters, as well as to a robust animation under novel poses. Last but not least, we propose to jointly synthesize audio and visual outputs from just text input. Given the recent rise of large language models, coupling text with natural-looking avatars can enhance the overall interaction between a human and an AI system.

Location: NCS 220 or Zoom

Speaker Petar Djuric Refreshments will be provided Deep Gaussian processes: Theory and applications Petar M. Djurić Department of Electrical and Computer Engineering Stony Brook University Abstract: Gaussian processes are an infinite-dimensional generalization of multivariate normal distributions. They provide a principled approach to learning with kernel machines and they have found wide applications in many fields. More recently, with the advance of deep learning, the concept of deep Gaussian processes has emerged. Deep Gaussian processes can be viewed as multilayer hierarchical organizations of Gaussian processes that are equivalent to infinitely wide multiple layer neural networks. Deep Gaussian processes have improved capacity for prediction and classification over standard Gaussian processes, while models based on them continue to allow for full Bayesian treatment and for applications when the amount of available data is limited. The theory of recent progress in deep Gaussian processes will be presented and some applications will be provided. Biosketch: Petar M. Djurić received the B.S. and M.S. degrees in electrical engineering from the University of Belgrade, Belgrade, Yugoslavia, respectively, and the Ph.D. degree in electrical engineering from the University of Rhode Island, Kingston, RI, USA. He is a SUNY Distinguished Professor and currently, he is a Chair of the Department of Electrical and Computer Engineering, Stony Brook University, Stony Brook, NY, USA. Djurić was a recipient of the IEEE Signal Processing Magazine Best Paper Award in 2007 and the EURASIP Technical Achievement Award in 2012. From 2008 to 2009, he was a Distinguished Lecturer of the IEEE Signal Processing Society. He was the Editor-in-Chief of the IEEE Transactions on Signal and Information Processing over Networks (2015-2018). Djurić is a Fellow of IEEE and EURASIP


Dates: 

Wednesday, March 3, 2021 - 6:00pm to 7:30pm

Location: 

Zoom - contact events@cs.stonybrook.edu for Zoom info.

Event Description: 

Women in Computer Science (WiCS), the Society of Women Engineers (SWE), and the Stony Brook Robotics Team (SBRT) are collaborating to host an event called Inspiring Women in STEM Academia: A Community Dialogue to address the lack of female representation in STEM academia. 
 

All are invited to attend so they may gain a better understanding of the challenges faced by their female colleagues and hear perspectives on how they can offer support in the workplace. Given the shockingly disproportionate number of female professionals in STEM academia, we feel that this event would be extremely beneficial for male faculty to listen to and amplify their voices.

It will begin with a discussion panel consisting of Stony Brook professors and faculty who will provide valuable insight into the issue. From there, we will split into smaller discussion groups where student and faculty attendees will be able to voice their opinions, hear about the thoughts/experiences of others, and participate in an engaging discussion with panelists.

The event will be held on March 3rd from 6:00 - 7:30 PM on Zoom.
 

The following Stony Brook faculty will be panelists:

Dr. Aruna Balasubramanian - Computer Science Professor, WiCS Advisor, WPhD Advisor

Dr. Xinwei Mao - Civil Engineering Assistant Professor

Urszula Zalewski - Director of Experiential Learning, Career Center Advisor (Healthcare)

Dr. Heather Lynch - Ecology and Evolution Professor, Lynch Lab for Quantitative Ecology

Karen Kernan - URECA Director, Simons Summer Research Program Director

Dr. Eszter Boros - Chemistry Assistant Professor, Boros Lab

Dr. Maria Nagan - Chemistry Lecturer, Nagan Research Lab

Join us to share your thoughts about teaching, learning, and AI!

The landscape of higher education is rapidly evolving with the integration of Artificial Intelligence (AI). Through the Institute on AI, Pedagogy, and the Curriculum with AAC&U, we are exploring ways that we can better address AI in teaching and learning. We want to hear your experiences, your concerns, and your ideas.

This is an open discussion for all faculty and staff to share their perspectives on the opportunities and challenges AI presents in our academic environment.

We'll be exploring critical questions like:
  • In the age of AI, what are the opportunities you see for enriching the classroom and curriculum? How can it enhance student learning or your professional practice?
  • What are the most significant challenges and concerns that AI raises for you regarding academics, student integrity, or your workload?
  • What resources (tools, training, technical support, policy guidance, etc.) do you need to feel confident and successful in the age of AI?
Date: Monday, December 1st
Time: 12:30pm-1:45pm
Location: West Campus - Melville Library, Special Collections Seminar Room (the room is to the left at the top of the first flight of stairs from the Melville lobby)
or
Date: Wednesday, December 3rd
Time: 10:30am-11:45am
Location: East Campus - HSC 2-154

Please register in advance so we can confirm the room.

Note: Videos will not be shared publicly and comments will only be shared in aggregate.

Your input is vital. From pedagogy to assessment, your insights will be critical. We look forward to a thoughtful and productive conversation!
  • Dr. Rose Tirotta-Esposito (Assistant Provost; Director of CELT)
  • Dr. Elizabeth Hewitt (Associate Professor in the Department of Technology and Society (DTS) in the College of Engineering and Applied Sciences)
  • Chris Kretz (Associate Librarian and Head of Academic Engagement at SBU Libraries)
  • Prof. Rajiv Lajmi (Assistant Professor in the School of Health Professions and Chair of Applied Health Informatics)
  • Dr. Matthew Salzano (Assistant Professor in the Department of Communication in the School of Communication and Journalism)
https://stonybrook.zoom.us/j/99820812332?pwd=c05BSTVLNmw3L04yZjdEcG5pem1OZz09 Speaker: Alexei Koulakov of Cold Spring Harbor Laboratory Brain evolution as a machine learning problem We have entered a golden age of artificial intelligence research, driven mainly by the advances in ANNs over the last decade or so. Applications of these techniques--to machine vision, speech recognition, autonomous vehicles, machine translation and many other domains--are coming so quickly that many observers predict that the long-elusive goal of Artificial General Intelligence (AGI) is within our grasp. However, we still cannot build a machine capable of building a nest, stalking prey, or loading a dishwasher. I will describe several projects, ranging from theories of evolution of neural development to the perception of smells, in which we are attempting to understand the algorithms that the nervous system is using to solve some of these challenging problems.
Abstract: Jailbreak attacks circumvent LLMs' built-in safeguards by concealing harmful queries within adversarial prompts. While most existing defenses attempt to mitigate the effects of adversarial prompts, they often prove inadequate as adversarial prompts can take arbitrary, adaptive forms. This paper introduces RobustKV, a novel jailbreak defense that takes a fundamentally different approach by selectively removing critical tokens of harmful queries from key-value (KV) caches. Intuitively, for an adversarial prompt to be effective, its tokens must achieve sufficient `importance' (measured by attention scores), which consequently lowers the importance of tokens in the concealed harmful query. Therefore, by carefully evicting the KVs of low-ranked tokens, RobustKV minimizes the harmful query's presence in the KV cache, thus preventing the LLM from generating informative responses. Extensive evaluation using benchmark datasets and models demonstrates that RobustKV effectively counters state-of-the-art jailbreak attacks while maintaining the LLM's performance on benign queries. Notably, RobustKV creates an interesting effectiveness-evasiveness dilemma for the adversary, leading to its robustness against adaptive attacks.

Speaker: Tanqiu Jiang

Where: NCS 220 and Zoom (https://stonybrook.zoom.us/j/6406956411)
The overall purpose of this seminar is to bring together people with interests in Computer Vision theory and techniques and to examine current research issues. This course will be appropriate for people who already took a Computer Vision graduate course or already had research experience in Computer Vision. To enroll in this course, you must either: (1) be in the PhD program or (2) receive permission from the instructors.

Each seminar will consist of multiple short talks (around 10 minutes) by multiple people. Students can register for 1 credit for CSE 656. Registered students must attend and present a minimum of 2 or 3 talks. Everyone else is welcome to attend. Fill in https://forms.gle/pCVXovgfMfQwGqG38 to subscribe to our mailing list for further announcement.