Topic: Exotic Neural Networks
Time: Mar 10, 2021 08:00 PM Eastern Time (US and Canada)
Join Zoom Meeting
https://stonybrook.zoom.us/j/
Meeting ID: 913 8893 8500
Passcode: 501725
Stony Brook University Libraries invites students, faculty, & staff to join a conversation about how AI is transforming the private sector workforce. As AI tools move from experimentation to everyday business use, companies are rethinking roles, skill sets, leadership, and long-term strategy. This discussion-based event will focus on the fast-paced changes and directions at tech companies and their possible impact. This event will be particularly relevant for students preparing for an AI influenced job market and how to position themselves for opportunities in a rapidly evolving professional landscape.
The discussion will be led by Tariq Khan, Senior Director of Private Cloud Solutions at Hewlett Packard Enterprise. Tariq is a technology leader and architect with experience across private cloud, hybrid cloud, and data center platforms. He is responsible for shaping the technology architecture and strategic direction of HPE's Private Cloud offerings across on premises and cloud integrated environments.
Light refreshments will be served.
Location: Melville Library, NRR, Learning Lab
Unlock the power of AI in your job search! Join the Head of Indeed Job Search Academy and AI experts as they explore how to leverage cutting-edge AI tools to optimize your job search activities, enhance your resume, prepare for interviews, and conduct thorough
career research, as well as answer all your AI-related questions.
This virtual watch party session will equip you with the knowledge to stand out in today's competitive market.
https://forms.gle/
career research, as well as answer all your AI-related questions.
This virtual watch party session will equip you with the knowledge to stand out in today's competitive market.
https://forms.gle/
Postmortem Program Analysis from a Conventional Program Analysis Method to an AI-assisted Approach
Abstract: Despite the best efforts of developers, software inevitably contains flaws that may be leveraged as security vulnerabilities. Modern operating systems integrate various security mechanisms to prevent software faults from being exploited. To bypass these defenses and hijack program execution, an attacker needs to constantly mutate an exploit and make many attempts. While in their attempts, the exploit triggers a security vulnerability and makes the running process abnormally terminate.
After a program has crashed and abnormally terminated, it typically leaves behind a snapshot of its crashing state in the form of a core dump. While a core dump carries a large amount of information, which has long been used for software debugging, it barely serves as informative debugging aids in locating software faults, particularly memory corruption vulnerabilities. As such, previous research mainly seeks fully reproducible execution tracing to identify software vulnerabilities in crashes. However, such techniques are usually impractical for complex programs. Even for simple programs, the overhead of fully reproducible tracing may only be acceptable at the time of in-house testing.
In this talk, I will discuss how we tackle this issue by bridging program analysis with artificial intelligence (AI). More specifically, I will first talk about the history of postmortem program analysis, characterizing and disclosing their limitations. Second, I will introduce how we design a new reverse-execution approach for postmortem program analysis. Third, I will discuss how we integrate AI into our reverse-execution method to escalate its analysis efficiency and accuracy. Last but not least, as part of this talk, I will demonstrate the effectiveness of this AI-assisted postmortem program analysis framework by using massive amounts of real-world programs.
Bio: Dr. Xinyu Xing is an Assistant Professor at Pennsylvania State University. His research interests include exploring, designing and developing new program analysis and AI techniques to automate vulnerability discovery, failure reproduction, vulnerability diagnosis (and triage), exploit and security patch generation. His past research has been featured by many mainstream media and received the best paper awards from ACM CCS and ACSAC. Going beyond academic research, he also actively participates and hosts many world-class cybersecurity competitions (such as HITB and XCTF). As the founder of JD-OMEGA, his team has been selected for DEFCON/GeekPwn AI challenge grand final at Las Vegas. Currently, his research is mainly supported by NSF, ONR, NSA and industry partners.
Abstract: Despite the best efforts of developers, software inevitably contains flaws that may be leveraged as security vulnerabilities. Modern operating systems integrate various security mechanisms to prevent software faults from being exploited. To bypass these defenses and hijack program execution, an attacker needs to constantly mutate an exploit and make many attempts. While in their attempts, the exploit triggers a security vulnerability and makes the running process abnormally terminate.
After a program has crashed and abnormally terminated, it typically leaves behind a snapshot of its crashing state in the form of a core dump. While a core dump carries a large amount of information, which has long been used for software debugging, it barely serves as informative debugging aids in locating software faults, particularly memory corruption vulnerabilities. As such, previous research mainly seeks fully reproducible execution tracing to identify software vulnerabilities in crashes. However, such techniques are usually impractical for complex programs. Even for simple programs, the overhead of fully reproducible tracing may only be acceptable at the time of in-house testing.
In this talk, I will discuss how we tackle this issue by bridging program analysis with artificial intelligence (AI). More specifically, I will first talk about the history of postmortem program analysis, characterizing and disclosing their limitations. Second, I will introduce how we design a new reverse-execution approach for postmortem program analysis. Third, I will discuss how we integrate AI into our reverse-execution method to escalate its analysis efficiency and accuracy. Last but not least, as part of this talk, I will demonstrate the effectiveness of this AI-assisted postmortem program analysis framework by using massive amounts of real-world programs.
Bio: Dr. Xinyu Xing is an Assistant Professor at Pennsylvania State University. His research interests include exploring, designing and developing new program analysis and AI techniques to automate vulnerability discovery, failure reproduction, vulnerability diagnosis (and triage), exploit and security patch generation. His past research has been featured by many mainstream media and received the best paper awards from ACM CCS and ACSAC. Going beyond academic research, he also actively participates and hosts many world-class cybersecurity competitions (such as HITB and XCTF). As the founder of JD-OMEGA, his team has been selected for DEFCON/GeekPwn AI challenge grand final at Las Vegas. Currently, his research is mainly supported by NSF, ONR, NSA and industry partners.
The overall purpose of this seminar is to bring together people with interests in Computer Vision theory and techniques and to examine current research issues. This course will be appropriate for people who already took a Computer Vision graduate course or already had research experience in Computer Vision. To enroll in this course, you must either: (1) be in the PhD program or (2) receive permission from the instructors.
Each seminar will consist of multiple short talks (around 10 minutes) by multiple people. Students can register for 1 credit for CSE 656. Registered students must attend and present a minimum of 2 or 3 talks. Everyone else is welcome to attend. Fill in https://forms.gle/pCVXovgfMfQwGqG38 to subscribe to our mailing list for further announcement.
Each seminar will consist of multiple short talks (around 10 minutes) by multiple people. Students can register for 1 credit for CSE 656. Registered students must attend and present a minimum of 2 or 3 talks. Everyone else is welcome to attend. Fill in https://forms.gle/pCVXovgfMfQwGqG38 to subscribe to our mailing list for further announcement.
Abstract: Autonomous systems, whether on Earth or in space, rely on 3D perception to understand and interact with the world around them. Yet traditional techniques for 3D understanding often depend on human designed features, fixed sensors, and conventional imaging modalities. This constrained approach can limit every stage of perception, from sensing to interpretation to decision making.
In this talk, we'll explore an alternative paradigm for imaging: physically based neural representations for 3D scenes and 3D sensing systems. We will discuss how recent advances in large scale learned representations can be used to jointly optimize both 3D scene models and the design of sensing systems for 3D capture, with the goal of enabling task specific perception systems.
Unlike modern AI models trained on internet scale datasets, these specialized 3D representations typically operate in data sparse regimes and therefore require a different kind of prior. We'll examine how grounding these learned representations in the physics of light transport can improve our understanding of scene structure, and inform imaging system design even with limited data. By connecting physical insights with learned representations, we'll highlight new possibilities for robust, efficient, and adaptive perception in challenging environments.
Speaker: Nikhil Behari is a graduate student in the Camera Culture group at the MIT Media Lab, advised by Professor Ramesh Raskar. His research interests include computational imaging, 3D scene understanding, and multi-agent decision-making under uncertainty, with a focus on automating imaging system design for 3D perception in human and planetary health. His research is supported by the NASA Space Technology Graduate Research Fellowship. He received his bachelor's in Computer Science and Statistics from Harvard University in 2022.
In this talk, we'll explore an alternative paradigm for imaging: physically based neural representations for 3D scenes and 3D sensing systems. We will discuss how recent advances in large scale learned representations can be used to jointly optimize both 3D scene models and the design of sensing systems for 3D capture, with the goal of enabling task specific perception systems.
Unlike modern AI models trained on internet scale datasets, these specialized 3D representations typically operate in data sparse regimes and therefore require a different kind of prior. We'll examine how grounding these learned representations in the physics of light transport can improve our understanding of scene structure, and inform imaging system design even with limited data. By connecting physical insights with learned representations, we'll highlight new possibilities for robust, efficient, and adaptive perception in challenging environments.
Speaker: Nikhil Behari is a graduate student in the Camera Culture group at the MIT Media Lab, advised by Professor Ramesh Raskar. His research interests include computational imaging, 3D scene understanding, and multi-agent decision-making under uncertainty, with a focus on automating imaging system design for 3D perception in human and planetary health. His research is supported by the NASA Space Technology Graduate Research Fellowship. He received his bachelor's in Computer Science and Statistics from Harvard University in 2022.
Abstract:
Capturing the spatio-temporal (4D) dynamics of humans has been a long standing research problem in computer vision and graphics. Synthesizing photorealistic human avatars has broad applications, ranging from immersive telepresence in AR/VR and the movie industry, to enriching the education and healthcare systems. Earlier approaches relied on hand-engineered models that use a small amount of data from one or more subjects. With the advent of neural networks, training on large datasets enhanced the output visual quality. Currently, the combination of neural networks with graphics techniques has achieved natural-looking human animation. However, most approaches are identity-specific, trained only on a single identity, and use only one modality.
In this thesis, we address the problem of learning neural representations of humans in a holistic way. Given that the video data in the real world include multiple modalities (audio and video) and multiple identities, we develop multi-modal and multi-identity representations. First, we propose to reconstruct the 4D face geometry of humans by leveraging both audio and video information. In this way, the network produces accurate lip shapes and is robust to cases when either modality is insufficient. Next, we introduce a NeRF-based representation for audio-driven human face animation that achieves high-quality lip synchronization for cinematic content. Since humans communicate with their full body, combining body pose, hand gestures, and facial expressions, we extend our network to capture the full-body human motion for multiple identities simultaneously. In order to better disentangle identity and non-identity specific information, we subsequently study non-linear interactions between latent factors of variation, and propose a specific multiplicative module. In this way, we learn a multi-identity NeRF that robustly animates human faces under novel expressions and achieves a significant decrease in the total training time. Similarly, we propose a multi-identity gaussian splatting representation for human bodies, by constructing a high-order tensor. Assuming a low-rank structure, we learn a tensor decomposition that leads to a significant decrease in the total number of learnable parameters, as well as to a robust animation under novel poses. In the future, we propose to jointly synthesize audio and visual outputs from just text input. Given the recent rise of large language models, coupling text with natural-looking avatars can enhance the overall interaction between a human and an AI system.
Speaker: Aggelina Chatziagapi
Where: NCS, Room 220
Zoom link: https://stonybrook.zoom.
ID: 98775312249
Passcode: 505777
Capturing the spatio-temporal (4D) dynamics of humans has been a long standing research problem in computer vision and graphics. Synthesizing photorealistic human avatars has broad applications, ranging from immersive telepresence in AR/VR and the movie industry, to enriching the education and healthcare systems. Earlier approaches relied on hand-engineered models that use a small amount of data from one or more subjects. With the advent of neural networks, training on large datasets enhanced the output visual quality. Currently, the combination of neural networks with graphics techniques has achieved natural-looking human animation. However, most approaches are identity-specific, trained only on a single identity, and use only one modality.
In this thesis, we address the problem of learning neural representations of humans in a holistic way. Given that the video data in the real world include multiple modalities (audio and video) and multiple identities, we develop multi-modal and multi-identity representations. First, we propose to reconstruct the 4D face geometry of humans by leveraging both audio and video information. In this way, the network produces accurate lip shapes and is robust to cases when either modality is insufficient. Next, we introduce a NeRF-based representation for audio-driven human face animation that achieves high-quality lip synchronization for cinematic content. Since humans communicate with their full body, combining body pose, hand gestures, and facial expressions, we extend our network to capture the full-body human motion for multiple identities simultaneously. In order to better disentangle identity and non-identity specific information, we subsequently study non-linear interactions between latent factors of variation, and propose a specific multiplicative module. In this way, we learn a multi-identity NeRF that robustly animates human faces under novel expressions and achieves a significant decrease in the total training time. Similarly, we propose a multi-identity gaussian splatting representation for human bodies, by constructing a high-order tensor. Assuming a low-rank structure, we learn a tensor decomposition that leads to a significant decrease in the total number of learnable parameters, as well as to a robust animation under novel poses. In the future, we propose to jointly synthesize audio and visual outputs from just text input. Given the recent rise of large language models, coupling text with natural-looking avatars can enhance the overall interaction between a human and an AI system.
Speaker: Aggelina Chatziagapi
Where: NCS, Room 220
Zoom link: https://stonybrook.zoom.
ID: 98775312249
Passcode: 505777
AI can help you write, you hear. AI can save you time, leverage your skills, enhance your productivity. . . . But you also hear: AI output is not reliable, not adequate for advanced tasks/learning, not ethical to use -- you could get in deep trouble for using AI tools without adequate mastery and caution. Which way is it?
Come join this hands-on workshop where you will explore AI tools and their affordances. Engage in writing tasks to learn how to use AI tools effectively and responsibly.
Sign up for a seat now: https://docs.google.com/forms/d/e/1FAIpQLSd0iDTKkTYnkxFd4LkgqbtP97zQSS4FI_MiPVm7p6IY5SGwSg/viewform
Come join this hands-on workshop where you will explore AI tools and their affordances. Engage in writing tasks to learn how to use AI tools effectively and responsibly.
Sign up for a seat now: https://docs.google.com/forms/d/e/1FAIpQLSd0iDTKkTYnkxFd4LkgqbtP97zQSS4FI_MiPVm7p6IY5SGwSg/viewform
Join a faculty development program to support instructors across campus with navigating/integrating AI in their courses. We're inviting interested faculty to participate in the grant project called Fostering Writing-to-Learn Skills with Critical AI Literacy: A Faculty Development and Student Support Program (funded through the AI3 Institute).
https://docs.google.com/document/d/1b51tvfK0HSOkCW7cwYq2nyyeeHtvBZYC7_XHv7Av8wQ/edit?tab=t.0
Time commitment and completion requirements :
Attend four sessions and a final symposium on the following dates/times:
Friday, September 12 from 11am - 12:30pm over Zoom
Friday, September 26 from 11am - 12:30pm over Zoom
Friday, October 10 from 11am - 12:30pm over Zoom
Friday, October 24 from 11am - 12:30pm over Zoom
Friday, November 14 from 10am - 1pm in Wang 201 - please note that this is an in person session only
Engage with online materials in Brightspace prior to each of the sessions (mainly to update a syllabus, assignment, or teaching strategy that you can share and discuss at the workshop)
https://docs.google.com/document/d/1b51tvfK0HSOkCW7cwYq2nyyeeHtvBZYC7_XHv7Av8wQ/edit?tab=t.0
Abstract: Neurosymbolic AI, as the combination of deep learning and knowledge representation/reasoning methods, is currently receiving significant attention due to its promise to both overcome apparent limitations of pure deep learning systems, and to address the knowledge acquisition bottleneck. This presentation will focus on some recent reseach advances made at Kansas State University related to Semantic Web / knowledge graphs and neurosymbolic AI, with particular emphasis on the use of symbolic AI methods for understanding hidden neuron activations, and on using large language models for knowledge graph and ontology engineering.
Short bio: Pascal Hitzler is University Distinguished Professor and endowed Lloyd T. Smith Creativity in Engineering Chair at the Department of Computer Science at Kansas State University, one of the Directors of the Institute for Digital Agriculture and Advanced Analytics (ID3A), and Director of the Center for Artificial Intelligence and Data Science (CAIDS). His research record lists over 400 publications in such diverse areas as neurosymbolic artificial intelligence, semantic web, knowledge graphs, knowledge representation and reasoning, denotational semantics, and set-theoretic topology. He was founding Editor-in-chief of the Semantic Web journal, the leading journal in the field, and is founding Editor-in-chief of the new Neurosymbolic Artificial Intelligence journal. He is co-author of the W3C Recommendation OWL 2 Primer, and of the book Foundations of Semantic Web Technologies by CRC Press, 2010, which was named as one out of seven Outstanding Academic Titles 2010 in Information and Computer Science by the American Library Association's Choice Magazine, and has translations into German and Chinese. He is a founding steering committee member of the Neural-Symbolic Learning and Reasoning Association that runs the annual International Conference on Neurosymbolic Learning and Reasoning (NeSy). For more information about him, see http://www.pascal-hitzler.de.
Location: NCS 120
Zoom: https://stonybrook.zoom.us/j/94657223635?pwd=FHtIvIupDAbQTCKzpDF3kTg6xRZOtj.1&jst=2