AI Innovation Institute

Abstract: Foundation models are rapidly becoming capable assistants for knowledge work, but their deployment in real settings is limited by three gaps: they do not transfer reliably across environments, their internal reasoning is opaque, and their behavior is hard to control precisely. In this talk, I argue that these limitations are not only about model size -- they are fundamentally about whether learning captures and leverages the underlying structure of the data-generating process. I use causal thinking as a practical lens to model what is invariant, what changes, and what can be intervened on, and I further show how this leads to learning principles that improve trustworthiness.
I will first present methods for learning unifying mechanisms from heterogeneous data, across domains and modalities, to enable reliable transfer and controllable generation. Next, I will show how structured concepts can be recovered even from seemingly unstructured data, by analyzing and improving self-supervised objectives (such as masking and diffusion) through hierarchical latent-variable models. These concept structures can then be used to interpret generative models and support targeted, multi-level edits. Finally, I connect these two threads to generalization beyond the training distribution. I will discuss natural conditions for extrapolation and a compositional generation framework that improves prompt following for novel concept combinations. I will conclude with a brief outlook on self-improving world models and AI-assisted scientific discovery.


Bio: Lingjing Kong is a Ph.D. candidate in the Computer Science Department at Carnegie Mellon University. His research focuses on Causal AI for transferable, interpretable, and controllable systems, with an emphasis on understanding and exploiting the structure of real-world data to make foundation models actionable and more reliable. He develops identification principles and scalable algorithms for learning unified models from heterogeneous data, uncovering hierarchical concept structures in unstructured data (e.g., images and text), and generalizing beyond training support through compositionality and extrapolation. His work has appeared in top ML venues including ICML, NeurIPS, CVPR, ICLR, and EMNLP and has been prototyped and applied in industry.

Location: NCS 120

Abstract: In this talk, we take a step back and argue that many varied and seemingly unrelated attacks on the web are actually symptoms of one deeper problem that has existed since the web's inception. Whether it is attacks due to expired domain names, cloaking done by malicious websites, malvertising, or even our growing distrust of the news can be largely attributed back to the issue of stateless linking. Stateless linking refers to the absence of any integrity guarantees between the time that a link for a remote resource was created, to a future time when this link is resolved by web clients. We draw on 10+ years of research to demonstrate how stateless linking and the resulting lack of content integrity is the true culprit for many of our past, current, and likely future web problems. Successfully tackling this one really challenging problem, has the potential of solving many of our web woes.

Bio: Nick Nikiforakis is affiliated with the National Security Institute. He received his PhD in Computer Science from KU Leuven in Belgium. He received his MSc, in Parallel and Distributed Systems and BSc in Computer Science from the University of Crete, Greece. His research focuses on web security and privacy, software security, and intrusion detection.

Location: NCS 120

The University at Albany will host a national gathering of professionals and academics that will focus on the transformative potential of AI while addressing the ethical, technical and institutional challenges posed by AI in education.

The Symposium will feature dynamic keynotes, hands-on workshops and engaging conversations with other participants and subject matter experts.

Topics

• Teaching, Learning and Workforce Development
• Research, Creative Arts, and Practice
• Ethics, Governance and Academic Administration

For event information and registration, visit Events@Albany.

Abstract:
Deep learning models have achieved remarkable success across a wide range of computer vision tasks, including image classification, semantic segmentation, etc. However, such success highly relies on a large amount of annotated data, which are expensive to obtain. Moreover, their performance often degrades when there exist distribution shifts between training and test data. Domain Adaptation overcomes these issues by transferring knowledge from a label-rich source domain to a related but different target domain. Despite its popularity, domain adaptation is still a challenging task, especially when the data distribution shifts are severe, while the target domain has no or few labeled data.

In this thesis, I develop four efficient domain adaptation approaches to improve model performance on the target domain. Firstly, inspired by the large-scale pretraining of Vision Transformers, I explore Transformer-based domain adaptation for stronger feature representation and design a safe training mechanism to avoid model collapse in the situation of a large domain gap. Secondly, I observe that source models have low confidences on the target data. To address this, I focus on the penultimate activations of target data and propose an adversarial training strategy to enhance model prediction confidences. Thirdly, I study using weak supervision from prior knowledge about target domain label distribution. A novel Knowledge-guided Unsupervised Domain Adaptation paradigm is devised, and a plug-in module is designed to rectify pseudo labels. Lastly, I step into the task of Active Domain Adaptation, where the labels of a small portion of target data can be inquired. I propose a novel active selection criterion based on the local context and devise a progressive augmentation module to better utilize queried target data. The robustness of domain adaptation approaches, in addition to accuracy, is critical yet under-explored. To conclude the thesis, I empirically study set prediction in domain adaptation using the tool of conformal prediction and conformal training.


Location: New Computer Science Bldg., Room 120
Zoom Link: https://stonybrook.zoom.us/j/92736258273?pwd=ipDdh1CTG6dRYmqa3ltUvooei8OfaT.1Meeting ID: 927 3625 8273
Passcode: 466399

Abstract:

Capturing the spatio-temporal (4D) dynamics of humans has been a long standing research problem in computer vision and graphics. Synthesizing photorealistic human avatars has broad applications, ranging from immersive telepresence in AR/VR and the movie industry, to enriching the education and healthcare systems. Earlier approaches relied on hand-engineered models that use a small amount of data from one or more subjects. With the advent of neural networks, training on large datasets enhanced the output visual quality. Currently, the combination of neural networks with graphics techniques has achieved natural-looking human animation. However, most approaches are identity-specific, trained only on a single identity, and use only one modality.

In this thesis, we address the problem of learning neural representations of humans in a holistic way. Given that the video data in the real world include multiple modalities (audio and video) and multiple identities, we develop multi-modal and multi-identity representations. First, we propose to reconstruct the 4D face geometry of humans by leveraging both audio and video information. In this way, the network produces accurate lip shapes and is robust to cases when either modality is insufficient. Next, we introduce a NeRF-based representation for audio-driven human face animation that achieves high-quality lip synchronization for cinematic content. Since humans communicate with their full body, combining body pose, hand gestures, and facial expressions, we extend our network to capture the full-body human motion for multiple identities simultaneously. In order to better disentangle identity and non-identity specific information, we subsequently study non-linear interactions between latent factors of variation, and propose a specific multiplicative module. In this way, we learn a multi-identity NeRF that robustly animates human faces under novel expressions and achieves a significant decrease in the total training time. Similarly, we propose a multi-identity gaussian splatting representation for human bodies, by constructing a high-order tensor. Assuming a low-rank structure, we learn a tensor decomposition that leads to a significant decrease in the total number of learnable parameters, as well as to a robust animation under novel poses. In the future, we propose to jointly synthesize audio and visual outputs from just text input. Given the recent rise of large language models, coupling text with natural-looking avatars can enhance the overall interaction between a human and an AI system.

Speaker: Aggelina Chatziagapi

Where: NCS, Room 220

Zoom link: https://stonybrook.zoom.us/j/98775312249?pwd=uORNAnSdcssrPZdqOsqaMAF5aLcRD9.1
ID: 98775312249
Passcode: 505777

Abstract: Retrieval-augmented generation (RAG) systems empower large language models (LLMs) to access external knowledge during inference. Recent advances have enabled LLMs to act as search agents via reinforcement learning (RL), improving information acquisition through multi-turn interactions with retrieval engines. However, existing approaches either optimize retrieval using search-only metrics (e.g., NDCG) that ignore downstream utility or fine-tune the entire LLM to jointly reason and retrieve--entangling retrieval with generation and limiting the real search utility and compatibility with frozen or proprietary models. In this work, we propose s3, a lightweight, model-agnostic framework that decouples the searcher from the generator and trains the searcher using a Gain Beyond RAG reward: the improvement in generation accuracy over naïve RAG. s3 requires only 2.4k training samples to outperform baselines trained on over 70 × more data, consistently delivering stronger downstream performance across six general QA and five medical QA benchmarks.

Speaker: Peter Zeng

Location: CS2311

An interactive session to discover how to create ALT text tags from images and create high-impact visuals, from identification to communicating ideas with images.

Discover how to use AI to create ALT text from images as well as identify objects in your environment, and build relatable visuals for high-impact presentations. Images communicate ideas as a way to understand concepts. AI-generated images have helped allow anyone to create these.

In this session, you will

1. Creating image ALT Tags
2. Transform ideas into images that are visually appealing
3. Identify objects from visuals

Register here.