AI Innovation Institute

CSE 656 Seminars in Computer Vision - Wednesdays 11:30am-12:50pm, Room NCS 120

The overall purpose of this seminar is to bring together people with interests in Computer Vision theory and techniques and to examine current research issues. This course will be appropriate for people who already took a Computer Vision graduate course or already had research experience in Computer Vision. To enroll in this course, you must either: (1) be in the PhD program or (2) receive permission from the instructors.

Each seminar will consist of multiple short talks (around 10 minutes) by multiple people. Students can register for 1 credit for CSE656. Registered students must attend and present a minimum of 2 or 3 talks. Everyone else is welcome to attend. Fill in https://forms.gle/pCVXovgfMfQwGqG38 to subscribe to our mailing list for further announcement.

Abstract: The rapid growth of observational data presents unprecedented opportunities to enhance both the predictability and mechanistic understanding of Earth systems. However, fully harnessing big Earth data needs computational frameworks that bridge the gap between physics-based models and machine learning. In this talk, I will first demonstrate how AI methods can significantly improve the prediction of environmental systems. Despite their predictive accuracy, machine learning models often lack physical interpretability, limiting their ability for scientific inquiry. To address this, I will introduce the developed hybrid, differentiable modeling framework that unifies physical models with machine learning in an end-to-end trainable system. This framework autonomously learns from large observations while maintaining physical clarity. The machine learning components can be seamlessly embedded into physical backbones to assimilate multi-source data, support automatic parameterization, and represent uncertain processes. I will showcase applications of this framework in simulating and understanding the terrestrial water cycle and its interactions with ecosystems at continental and global scales. This talk will highlight how differentiable modeling not only improves the modeling ability in both data-rich and data-scarce scenarios, but also provides a systematic pathway to enhancing model structures, deciphering uncertain physical relations, and facilitating knowledge discovery in Earth system sciences.


IACS Seminar Speaker: Dapeng Feng, Stanford Univeristy

Location: IACS Seminar Room

Abstract:
Deep learning models have achieved remarkable success across a wide range of computer vision tasks, including image classification, semantic segmentation, etc. However, such success highly relies on a large amount of annotated data, which are expensive to obtain. Moreover, their performance often degrades when there exist distribution shifts between training and test data. Domain Adaptation overcomes these issues by transferring knowledge from a label-rich source domain to a related but different target domain. Despite its popularity, domain adaptation is still a challenging task, especially when the data distribution shifts are severe, while the target domain has no or few labeled data.

In this thesis, I develop four efficient domain adaptation approaches to improve model performance on the target domain. Firstly, inspired by the large-scale pretraining of Vision Transformers, I explore Transformer-based domain adaptation for stronger feature representation and design a safe training mechanism to avoid model collapse in the situation of a large domain gap. Secondly, I observe that source models have low confidences on the target data. To address this, I focus on the penultimate activations of target data and propose an adversarial training strategy to enhance model prediction confidences. Thirdly, I study using weak supervision from prior knowledge about target domain label distribution. A novel Knowledge-guided Unsupervised Domain Adaptation paradigm is devised, and a plug-in module is designed to rectify pseudo labels. Lastly, I step into the task of Active Domain Adaptation, where the labels of a small portion of target data can be inquired. I propose a novel active selection criterion based on the local context and devise a progressive augmentation module to better utilize queried target data. The robustness of domain adaptation approaches, in addition to accuracy, is critical yet under-explored. To conclude the thesis, I empirically study set prediction in domain adaptation using the tool of conformal prediction and conformal training.


Location: New Computer Science Bldg., Room 120
Zoom Link: https://stonybrook.zoom.us/j/92736258273?pwd=ipDdh1CTG6dRYmqa3ltUvooei8OfaT.1Meeting ID: 927 3625 8273
Passcode: 466399

George Em Karniadakis received his SM and PhD from Massachusetts Institute of Technology. He was appointed lecturer in the Department of Mechanical Engineering at MIT in 1987 and subsequently he joined the Center for Turbulence Research at Stanford/Nasa Ames. He joined Princeton University as assistant professor in the Department of Mechanical and Aerospace Engineering and as associate faculty in the program of applied and computational mathematics. He was a visiting professor at Caltech in 1993 in the Aeronautics Department and joined Brown University as associate professor of applied mathematics in the Center for Fluid Mechanics in 1994. After becoming a full professor in 1996, he continues to be a visiting professor and senior lecturer of Ocean/Mechanical Engineering at MIT. He is an AAAS fellow (2018), fellow of the Society for Industrial and Applied Mathematics (2010), fellow of the American Physical Society (2004), fellow of the American Society of Mechanical Engineers (2003) and associate fellow of the American Institute of Aeronautics and Astronautics (2006). He received the Alexander von Humboldt award in 2017, the Ralf E Kleinman award (2015), the J. Tinsley Oden Medal (2013), and the CFD award (2007) from the US Association in Computational Mechanics. His h-index is 103, and he has been cited over 52,000 times.


Abstract:
Karniadakis will present a new approach to develop a data-driven, learning-based framework for predicting outcomes of physical and biological systems, governed by PDEs, and for discovering hidden physics from noisy data. He will introduce a deep learning approach based on neural networks (NNs) and generative adversarial networks (GANs). He will also introduce new NNs that learn functionals and nonlinear operators from functions and corresponding responses for system identification. Unlike other approaches that rely on big data, here we learn from small data by exploiting the information provided by the physical conservation laws, which are used to obtain informative priors or regularize the neural networks. He will demonstrate the power of PINNs for several inverse problems in fluid mechanics, solid mechanics and biomedicine including wake flows, shock tube problems, material characterization, brain aneurysms, etc., where traditional methods fail due to lack of boundary and initial conditions or material properties. He will also present a new NN, DeepM&Mnet, which uses DeepOnets as building blocks for multiphysics problems, and he will demonstrate its unique capability in a 7-field hypersonics application.  

To register and for more information, click here.  

AI can help you write, you hear. AI can save you time, leverage your skills, enhance your productivity. . . . But you also hear: AI output is not reliable, not adequate for advanced tasks/learning, not ethical to use -- you could get in deep trouble for using AI tools without adequate mastery and caution. Which way is it?
Come join this hands-on workshop where you will explore AI tools and their affordances. Engage in writing tasks to learn how to use AI tools effectively and responsibly.
Sign up for a seat now: https://docs.google.com/forms/d/e/1FAIpQLSd0iDTKkTYnkxFd4LkgqbtP97zQSS4FI_MiPVm7p6IY5SGwSg/viewform

Do Natural Language Understanding Systems Learn to Understand or to
Find Shortcuts? (Naoya Inoue, http://naoya-i.github.io/)

ABSTRACT: Recent studies have suggested that natural language understanding (NLU) systems learn to exploit superficial, task-unrelated cues (a.k.a. annotation artifacts) in current datasets. This prevents the community from reliably measuring the progress of NLU systems. In this talk, I will discuss two latest studies from our research team: (i) analysis of annotation artifacts in commonsense causal reasoning and (ii) creation of benchmark for evaluating NLU systems' internal reasoning.
---------------------------------------------------------------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------------------------------------------------------------
Learning graph-structured sparse models (Baojian Zhou, https://baojianzhou.github.io/) 

ABSTRACT: Learning graph-structured sparse models has recently received significant attention thanks to their broad applicability to many important real-world problems. However, such models, of more effective and stronger interpretability compared with their counterparts, are difficult to learn due to optimization challenges. In this talk, we will discuss how to learn graph-structured sparse models under stochastic and online learning settings. Some interesting related problems will also be discussed.

The Center for Challenging Systems of Power is hosting a Reading Group and Workshop

Participants will be discussing 'AI and the Practice of Democracy' by Mark Coeckelbergh

Location: Humanities 1008

Abstract:
Deep neural network (DNN) is a powerful tool for solving image processing and computer vision tasks such as image and video reconstructions, object recognition, and scene understanding, etc. However, DNN have been used for only the digital domain in the imaging pipeline, such as the feature extractor and classifier models after an image is captured and digitized. In this research, we propose a new framework called deep sensing. The proposed framework also models the analog layer to the neural network model and jointly optimizes the parameters in optics and sensor designs of a camera, as well as reconstruction and classification models by the same training strategy.

Bio:
Hajime Nagahara is a professor at D3 Center, Osaka University, since 2024. He received Ph.D. degree in system engineering from Osaka University in 2001. He was a research associate of the Japan Society for the Promotion of Science from 2001 to 2003. He was an assistant professor at the Graduate School of Engineering Science, Osaka University, Japan from 2003 to 2010. He was an associate professor in Faculty of Information Science and Electrical Engineering at Kyushu University from 2010 to 2017. He was a professor at Institute for Datability Science, Osaka University, from 2017 to 2024. He was a visiting associate professor at CREA University of Picardie Jules Verns, France, in 2005. He was a visiting researcher at Columbia University in 2007-2008 and 2016-2017. Computational photography and computer vision are his research areas. He received IPSJ Nagao Special Researcher Award in 2012, ICCP2016 Best Paper Runners-up, and SSII Takagi Award in 2016. He is a program chair for ICCP2019, General Chair for upcoming ACCV2026 Osaka, Associate Editor for IEEE Transactions on Computational Imaging in 2019-2022, Associate Editor for IEEE Transactions of Pattern Recognisiton and Machine Intelligence, and Director of Information Processing Society of Japan in 2022-2024.

Zoom: https://stonybrook.zoom.us/j/4742917886?pwd=YiW3AL5lvPlEaWrd221YfLzYhlgsRK.1

AI + Music Seminar - Details to come.