AI Seminar: Video Architecture Search - Michael Ryoo
Abstract: Video understanding is a challenging problem. Because a video contains spatio-temporal data, its feature representation is required to abstract both appearance and motion information. This is not only essential for automated understanding of the semantic content of videos, such as Web-video classification or sport activity recognition, but is also crucial for robot perception and learning. Previously, convolutional neural networks (CNNs) for videos were normally built by manually extending known 2D architectures such as Inception and ResNet to 3D or by carefully designing two-stream CNN architectures that fuse together both appearance and motion information. However, designing an optimal video architecture to best take advantage of spatio-temporal information in videos still remains an open problem. In this talk, we discuss recent progress in neural architecture search for videos, obtaining more optimal network architectures for video understanding.
Abstract:
Quantifying similarity is a central notion in science and data analysis, pervading everything from phylogenetic trees to the foundation of clustering. Unfortunately, despite being examined and applied for decades, traditional similarity and distance metrics have fundamental drawbacks. The key problem is that all of them are only defined over pairs of objects, so they scale quadratically when one tries to compare N objects. The present explosion in the amount of data available to us requires new ways to process information, and while some current algorithms can handle millions of points, we need alternatives applicable to billions. This is what motivated us to develop a new framework that can compare any number of objects at the same time. With this, we achieve an unprecedented linear scaling when comparing multiple objects. Here we will discuss the main properties of this formalism, along with its applications in drug design and to the analysis of Molecular Dynamics (MD) simulations. Our indices have proven to be incredibly versatile when applied to chemical space exploration and visualization, allowing us to rigorously quantify the chemical diversity of very large molecular libraries. This has led to the creation of several algorithms to sample important regions in chemical space, including a more efficient way of identifying the prevalence of activity cliffs. Additionally, our indices provide a convenient route to sample complex MD trajectories, allowing to identify representative structures very efficiently. Moreover, we can also cluster biological ensembles in a more robust way than with standard algorithms, which has led to our group's work on MDANCE, a very flexible and efficient open-source clustering module. Drop by if you want to know how we clustered one billion molecules!
Speaker:
Assistant Professor, Department of Chemistry and Quantum Theory Project
University of Florida, Gainesville
Website: https://quintana.chem.ufl.edu/
Location:
Laufer Center Lecture Hall 101
Topic: AI Seminar: Stanley Bak
Time: Monday Nov 1, 2021 12:00 PM Eastern Time (US and Canada)
Join Zoom Meeting
https://stonybrook.zoom.us/j/
Abstract: The field of formal verification has traditionally looked at proving properties about finite state machines or software programs. The surge in deep learning has been accompanied by a surge of progress in trying to apply mathematical and algorithmic techniques to prove things about the function being computed by a neural network.
This talk formalizes the neural network verification problem and describes technical methods for neural network verification based on reachability analysis. Improvements to analysis efficiency will be given, as well as research directions for further exploration. We also include an objective comparison performed this last summer trying to evaluate the best existing verification methods in terms of speed and network size. The competition was performed on common hardware and involved the participation of twelve international teams (the tool authors) on a common set of benchmarks.
Biography: Stanley Bak is an assistant professor in the Department of Computer Science at Stony Brook University investigating the verification of autonomy, cyber-physical systems, and neural networks. He strives to develop practical formal methods that are both scalable and useful, which demands developing new theory, programming efficient tools and building experimental systems.
Stanley Bak received a Bachelor's degree in Computer Science from Rensselaer Polytechnic Institute (RPI) in 2007 (summa cum laude), and a Master's degree in Computer Science from the University of Illinois at Urbana-Champaign (UIUC) in 2009. He completed his PhD from the Department of Computer Science at UIUC in 2013. He received the Founders Award of Excellence for his undergraduate research at RPI in 2004, the Debra and Ira Cohen Graduate Fellowship from UIUC twice, in 2008 and 2009, and was awarded the Science, Mathematics and Research for Transformation (SMART) Scholarship from 2009 to 2013. From 2013 to 2018, Stanley was a Research Computer Scientist at the US Air Force Research Lab (AFRL), both in the Information Directorate in Rome, NY, and in the Aerospace Systems Directorate in Dayton, OH. He helped run Safe Sky Analytics, a research consulting company investigating verification and autonomous systems, and performed teaching at Georgetown University before joining Stony Brook University as an assistant professor in Fall 2020.
The Fortieth AAAI Conference on Artificial Intelligence (AAAI-26), which will be held in Singapore EXPO from January 20 to January 27, 2026.
The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines. AAAI-26 will feature technical paper presentations, special tracks, invited speakers, workshops, tutorials, poster sessions, senior member presentations, competitions, and exhibit programs, and a range of other activities to be announced.
For more information and registration, please visit the official website.
Abstract: Computer vision seeks to extract semantic and geometric information from images and videos, serving as the perceptual foundation for intelligent systems such as robots and autonomous vehicles. Over the past decade, deep learning has driven remarkable progress in the field, advancing capabilities from 2D recognition to 3D reconstruction. However, the current purely data-driven paradigm faces fundamental challenges, including data inefficiency, curse of high dimensionality, and limited understanding of visual entities beyond individual objects.
In this talk, I will present my recent research on modeling and learning rich visual structures to address these challenges. First, I will introduce a novel framework that integrates explicit visual dependency modeling with deep learning for 2D and 3D dense prediction. Next, I will demonstrate how unfolding the manifold structure of visual data enables unsupervised semantic segmentation. Finally, I will present a recent project that represents, parses, and learns the geometric compositionality of 3D objects to facilitate self-supervised part-whole reconstruction. Through these efforts, I aim to bridge the gap between data-driven deep learning and visual structure modeling, paving the way for more efficient, generalizable, and interpretable computer vision models.
Bio: Dr. Wei Tang is an Assistant Professor in the Department of Computer Science at the University of Illinois Chicago (UIC). He obtained his Ph.D. in Electrical Engineering from Northwestern University, where his dissertation was honored with a Best Dissertation Award. His research interests include computer vision, digital image processing, and machine learning. Dr. Tang has served as an associate editor for several international journals, including Pattern Recognition and Machine Vision and Applications, and as an area chair for leading conferences, including CVPR, ICCV, and WACV. His research has been funded by the National Science Foundation (NSF) and industry partners such as Motorola and Wormpex AI Research.
Location: NCS 115
Zoom: https://stonybrook.zoom.
Subject: RADIOLOGY GRAND ROUNDS CT Colonography: An Effective Test for Colorectal Cancer Screening- Judy Yee, M.D.
When: Wednesday, May 12, 2021 12:00 PM-1:00 PM (UTC-05:00) Eastern Time (US & Canada).
Where: JOIN ZOOM MEETING
Judy Yee, MD
Chair, Department of Radiology
Professor, Department of Radiology
Abdominal Imaging
Join Zoom Meeting
https://einsteinmed.zoom.us/j/
Meeting ID: 977 8219 0723
Passcode: 101083
https://stonybrook.zoom.us/j/9
Meeting ID: 917 7572 9097
Passcode: 555459
Abstract: As the saying goes, there are many ways to skin a cat.
While we don't want to go around skinning cats, the world of
optimization is rich with different problems, problem formulations,
and methods and approaches, each with different guarantees and
computational benefits. In this talk we will take a tour down the
problem of structured sparsity in sensing to see how one simple
problem can inspire a wide range of analysis and tools. First, I will
present the optimality conditions for a generalized structured sparse
problem, which can be geometrically visualized as alignment of vectors
and matrices. Then I will introduce three approximation methods for
the problem of phase retrieval, which are a twist on stochastic
gradient and coordinate descent methods. These methods leverage
fundamental numerical linear algebra concepts to give fast approximate
solutions to large-scale problems, which then after postprocessing can
produce more reliable sensing results.
Bio: Yifan Sun received her PhD in Electrical Engineering from the
University of California Los Angeles in 2015, with research focusing
on convex optimization and semidefinite programming. She was then
Technicolor Research and Innovation, focusing on machine learning and
data science applications. More recently, she completed two postdocs,
at the University of British Columbia in Vancouver, Canada and
L'Institut National de Recherche en Informatique et Automatique
(INRIA) in Paris, France.
Abstract: Language offers a uniquely powerful lens for understanding the mind: one that can access latent psychological realities often missed by traditional measurement tools. However, as language models expand their ability to capture semantics through context length, expansion into deeper levels of semantics is less explored, especially with respect to understanding cognitive patterns of authors. This dissertation proposes that we can uncover deeper cognitive and affective patterns that reflect more accurate underlying mental states by analyzing language at higher levels of discourse semantics and by modeling latent states.
First, the dissertation focuses on uncovering cognitive styles or thinking patterns manifesting in language. We demonstrate that modeling language at deeper semantic levels such as discourse relations, can unveil latent psychological states and traits, including cognitive styles that influence both mental health and behavior. Introducing a novel blend of transfer and active learning, we efficiently curated a new set of linguistic data on cognitive styles like dissonance. This approach allows for more precise measurement when dealing with rare-classes and low-resource tasks. As a second contribution, effective validation methods are introduced to language-based assessments of the underlying cognitive styles. Controlled behavioral experiments and online studies show that cognitive styles detected through linguistic signals reliably predict real-world behaviors such as decision-making and engagement with extremist communities, both at the individual and community levels, sometimes months in advance
The research further moves beyond traditional measurement tools like questionnaires and expert judgments, which rely on Classical Test Theory, by establishing that language-based assessments more closely approximate true psychological states. The mechanisms by which these assessments outperform standard tools are explained, highlighting their predictive power for behaviors linked to underlying traits. Finally, a more sophisticated approach is explored by modeling psychological outcomes with Item Response Theory (IRT), an improvement over Classical Test Theory. Adaptive language-based assessments are introduced, showing that targeted, adaptive testing based on latent IRT scores can efficiently and accurately capture multiple psychological dimensions.
Taken together, these contributions argue for a shift towards language-based psychological assessments. By integrating deeper discourse-level semantics with measurement theory, this dissertation charts a path towards truer scores of mental states: ones that are more precise, and reflective of the complexity of human cognition and emotions.
Speaker: Vasudha Varadarajan
https://stonybrook.zoom.us/j/
First, the dissertation focuses on uncovering cognitive styles or thinking patterns manifesting in language. We demonstrate that modeling language at deeper semantic levels such as discourse relations, can unveil latent psychological states and traits, including cognitive styles that influence both mental health and behavior. Introducing a novel blend of transfer and active learning, we efficiently curated a new set of linguistic data on cognitive styles like dissonance. This approach allows for more precise measurement when dealing with rare-classes and low-resource tasks. As a second contribution, effective validation methods are introduced to language-based assessments of the underlying cognitive styles. Controlled behavioral experiments and online studies show that cognitive styles detected through linguistic signals reliably predict real-world behaviors such as decision-making and engagement with extremist communities, both at the individual and community levels, sometimes months in advance
The research further moves beyond traditional measurement tools like questionnaires and expert judgments, which rely on Classical Test Theory, by establishing that language-based assessments more closely approximate true psychological states. The mechanisms by which these assessments outperform standard tools are explained, highlighting their predictive power for behaviors linked to underlying traits. Finally, a more sophisticated approach is explored by modeling psychological outcomes with Item Response Theory (IRT), an improvement over Classical Test Theory. Adaptive language-based assessments are introduced, showing that targeted, adaptive testing based on latent IRT scores can efficiently and accurately capture multiple psychological dimensions.
Taken together, these contributions argue for a shift towards language-based psychological assessments. By integrating deeper discourse-level semantics with measurement theory, this dissertation charts a path towards truer scores of mental states: ones that are more precise, and reflective of the complexity of human cognition and emotions.
Speaker: Vasudha Varadarajan
https://stonybrook.zoom.us/j/
Join us for an engaging panel discussion featuring researchers who participated in our inaugural AI JAM session on February 26th. Our panelists will share their firsthand experiences using large language models to tackle complex scientific problems, with a special focus on prompt engineering strategies, discussing both breakthroughs and challenges encountered during this collaborative initiative. Learn how these cutting-edge AI tools are being applied to real-world research questions and discover insights that could inform your own scientific endeavors. Attendees are encouraged to come prepared with questions about prompt engineering for the panel discussion.
Moderator: Adolfy Hoisie, Deputy Director, Computing and Data Sciences
Kevin Yager, Group Leader, AI-Accelerated Nanoscience, Center for Functional Nanomaterials
Lingda Li, Associate Computational Scientist, Systems, Architecture and Computing Technologies, Computing and Data Sciences
Liguo Wang, Director of Scientific Operations, Laboratory for BioMolecular Structure (LBMS), National Synchrotron Light Source II
Weiguo Yin, Physicist, Condensed Matter Theory, Condensed Matter Physics and Materials Science Department
Location: CDS, Bldg. 725, Training Room
Join ZoomGov Meeting: https://bnl.zoomgov.com/j/1606837837?pwd=Tc0mwQqLXpDfYOIaoaurmpLD2mMlzS.1 (Meeting ID)
Passcode: 822553
Title:Deep Contextual Modeling for Natural Language Understanding, Generation, and Grounding
Zoom instructions:
Join Zoom Meeting
https://stonybrook.zoom.us/j/645050299?pwd=TVJVRkc3dlhxdDF5d00xWGlDQkovZz09
Meeting ID: 645 050 299
Password: 810247
One tap mobile
+16468769923,,645050299#,,#,810247# US (New York)
+13126266799,,645050299#,,#,810247# US (Chicago)
Dial by your location
+1 646 876 9923 US (New York)
+1 312 626 6799 US (Chicago)
+1 301 715 8592 US
+1 346 248 7799 US (Houston)
+1 408 638 0968 US (San Jose)
+1 669 900 6833 US (San Jose)
+1 253 215 8782 US
Meeting ID: 645 050 299
Password: 810247
Find your local number: https://stonybrook.zoom.us/u/aemTiJMXu6
Abstract:
Natural language is a fundamental form of information and communication. In both human-human and human-computer communication, people reason about the context of text and world state to understand language and produce language response. In this talk, I present several deep neural network based systems that first understand the meaning of language grounded in various contexts where the language is used, and then generate effective language responses in different forms for information access and human-computer communication. First, I will introduce Speaker Interaction RNNs for addressee and response selection in multi-party conversations based on explicit representations for different discourse participants. Then, I will present a text summarization approach for generating email subject lines by optimizing quality scores in a reinforcement learning framework. Finally, I will show an editing-based multi-turn SQL query generation system towards intelligent natural language interfaces to databases.
Bio:Rui Zhang is a final year Ph.D. student at Yale University advised by Professor Dragomir Radev. His research interest lies in various natural language processing problems in understanding, generation, and grounding. He has been working on (1) End-to-End Neural Modeling for Entities, Sentences, Documents, and Multi-party Multi-turn Dialogues, (2) Text Summarization for Emails, News, and Scientific Articles, (3) Cross-lingual Information Retrieval for Low-Resource Languages, (4) Context-Dependent Text-to-SQL Semantic Parsing in Human-Computer Interaction. Rui Zhang has published papers and served as Program Committee members at top-tier NLP and AI conferences including ACL, NAACL, EMNLP, AAAI, CoNLL. During his Ph.D., He has done research internships at IBM Thomas J. Watson Research Center, Grammarly Research, and Google AI. He was a graduate student at the University of Michigan and got his bachelor's degrees at both the University of Michigan and Shanghai Jiao Tong University from the UM-SJTU Joint Institute.