An interactive session to discover how to create ALT text tags from images and create high-impact visuals, from identification to communicating ideas with images.

Discover how to use AI to create ALT text from images as well as identify objects in your environment, and build relatable visuals for high-impact presentations. Images communicate ideas as a way to understand concepts. AI-generated images have helped allow anyone to create these.

In this session, you will

  1. Creating image ALT Tags
  2. Transform ideas into images that are visually appealing
  3. Identify objects from visuals

Register here.
Abstract: Retrieval-augmented generation (RAG) systems empower large language models (LLMs) to access external knowledge during inference. Recent advances have enabled LLMs to act as search agents via reinforcement learning (RL), improving information acquisition through multi-turn interactions with retrieval engines. However, existing approaches either optimize retrieval using search-only metrics (e.g., NDCG) that ignore downstream utility or fine-tune the entire LLM to jointly reason and retrieve--entangling retrieval with generation and limiting the real search utility and compatibility with frozen or proprietary models. In this work, we propose s3, a lightweight, model-agnostic framework that decouples the searcher from the generator and trains the searcher using a Gain Beyond RAG reward: the improvement in generation accuracy over naïve RAG. s3 requires only 2.4k training samples to outperform baselines trained on over 70 × more data, consistently delivering stronger downstream performance across six general QA and five medical QA benchmarks.

Speaker: Peter Zeng

Location: CS2311
Abstract: Molecular learning has become an emerging field of AI, driving breakthroughs in drug discovery, protein design, and materials design. For high-stakes scientific tasks, however, predictive accuracy alone is not sufficient: models must also be interpretable and trustworthy. Our work aims to study molecular learning under a unified explainability perspective across two major model families: Graph Neural Networks (GNNs) and Large Language Models (LLMs).

GNNs are natural choices for molecular graphs and achieve strong performance on many molecular tasks. To enhance explainability, many GNN explanation methods have been proposed and work well for 2D GNNs. However, 3D GNNs introduce two key challenges: producing chemically meaningful substructures and reducing fidelity loss caused by dense geometric graphs. To address these challenges, I present two methods. 3DGraphX decomposes dense 3D graphs into chemically meaningful 3D motifs, enabling compact explanations that align with chemical intuition. EDMA introduces an energy-based discrete mask approximation approach to reduce the discrepancy between the soft mask optimized during training and the hard mask used for explanation, improving explanation fidelity.

LLMs present different characteristics and challenges compared with GNNs. LLMs can provide a certain level of explanation through step-by-step reasoning, and their natural-language outputs are easy for humans to understand and interpret. However, because LLMs are trained for general-purpose tasks, their performance on scientific tasks often lags behind specialized GNNs. To improve performance, existing methods guide LLMs by providing suggestions through brief feedback, retrieval-augmented generation (RAG), or planner agents. However, these approaches face several limitations, such as vague guidance, introduced bias problems, and high computational cost. To fill the gap, I propose RL-Guider, a lightweight reinforcement-learning agent that converts evaluation feedback into input-specific guidance for molecular optimization. RL-Guider improves over time by accumulating historical experience and transfers efficiently across different LLMs while preserving interpretability.

Together, these efforts aim to provide explanations that are scientifically meaningful and faithful, while also preserving or improving performance on molecular tasks to better meet real scientific needs.

Speaker: Xufeng Liu

Location: New Computer Science-1-Room 115

Abstract:

It is known that models like large language models (LLMs) can often suggest colloquial plans given verbal descriptions of tasks, yet they are unable to reliably provide executable and verifiable plans given formally specified environments. In this talk, I will discuss a strand of efforts to have LLMs generate accurate and explainable plans in textual simulations. Instead of directly generating the plan or actions, LLMs are prompted to generate Planning Domain Definition Language (PDDL) that specifies the environment (domain file) and the task (problem file), which can then be deterministically solved with an off-the-shelf planner. In a 3-phase study, my collaborators and I first observed that it is possible but very challenging for LLMs to generate long-form code such as PDDL domain and problem files given textual specifications. Next, we devise methodologies for LLMs to iteratively generate and refine problem files while exploring a partially-observed, simulated, textual environment. Finally, we show that domain files are even more difficult to generate correctly, even on well-established planning tasks such as BlocksWorld. Finally, I will discuss ongoing efforts to improve said ability of structured generation and promising frontiers to explore.

Bio:
Li Harry Zhang is an assistant professor at Drexel University, focusing on Natural Language Processing (NLP) and artificial intelligence (AI). He obtained his PhD degree from the University of Pennsylvania advised by Prof. Chris Callison-Burch. Prior, he obtained his Bachelor's degree at the University of Michigan mentored by Prof. Rada Mihalcea and Prof. Dragomir Radev. His current research uses large language models (LLMs) to reason and plan via symbolic and structured representations. He has published more than 20 peer-reviewed papers in NLP and AI conferences, such as ACL, EMNLP, and AACL, that have been cited more than 1,000 times. He also consistently serves as Area Chair, Session Chair, and reviewer in those venues. Being a musician, producer, and content creator having over 50,000 subscribers, he is also passionate in the research of AI music and creativity.

The AI Symposium at SUNY Upstate Medical University will bring together researchers, clinicians, data scientists, and industry partners to explore the transformative role of artificial intelligence in personalized healthcare. The event will highlight innovations at the intersection of machine learning, clinical informatics, computational biology, and medical imaging aimed at improving diagnosis, treatment, and patient outcomes.
The program will feature keynote talks from national leaders, panel discussions, and research presentations addressing AI-driven biomarker discovery, multimodal data integration, predictive modeling, and clinical decision support systems. Discussions will also emphasize responsible AI adoption, health equity, and translating computational advances into clinical practice.
The symposium will showcase the growing AI research ecosystem at SUNY Upstate and regional institutions through poster presentations from faculty, trainees, and collaborators. By fostering collaboration across academia, healthcare, and industry, the event aims to accelerate the development of AI technologies that enable data-driven, patient-centered care.

This symposium is free and open to the public; however, registration is required.

How to Do Spectral Learning at Scale for Science and Engineering

Abstract: Spectral decompositions such as singular value decompositions (SVDs) and eigenvalue decompositions (EVDs) are central tools across a vast swath of scientific computing and machine learning, with abundant engineering applications. Yet many modern methods for learning such decompositions in high dimensions struggle with instability, bias, and poor scalability, even when approximation power is not the limiting factor. I argue that these difficulties are not intrinsic to spectral problems, but instead arise from a shared reliance on Rayleigh-quotient-based constrained optimization, which forces explicit orthogonality handling through penalties, normalization, or whitening.
To address these challenges, I present a reformulation based on unconstrained variational objectives that implicitly encode spectral structure, eliminating the need for orthogonalization and ad-hoc regularization. This perspective leads to a conceptually simpler and scalable parametric framework for learning ordered spectral representations via nested optimization. The resulting framework is well matched to diverse settings in science and engineering. As examples, I demonstrate its effectiveness on eigenvalue problems for linear PDEs such as the Schrödinger equation, spectral (Koopman) analysis of nonlinear dynamical systems such as molecular dynamics, and structured representation learning with deep neural nets. Collectively, these examples illustrate how abandoning Rayleigh-quotient-based formulations resolves long-standing optimization pathologies across domains.

Bio: Jongha (Jon) Ryu is a postdoctoral associate at MIT EECS. He received his Ph.D. in Electrical and Computer Engineering from UC San Diego. His research develops statistical and mathematical foundations for scientific machine learning, with a focus on scalable spectral methods, efficient generative modeling, and reliable uncertainty quantification for scientific and engineering systems.

Location: NCS 120
The Artificial Intelligence Innovation Institute (AI^3), with administrative support from the Office of the Vice President for Research (OVPR), invites applications to a seed grant program for collaborative projects in artificial intelligence, along three distinct tracks: Collaborative Research in AI, Technical Support for Discipline-Centric Research, and Seed Grants for AI Education and Service.

The program will fund projects for up to a one-year period, depending on the availability of funds. AI^3 anticipates making at least six awards on this call. A one-year, no-cost extension can be requested in the final 6 months of a project with approval subject to progress towards project goals and active participation in research themes.

Competitive applications will actively incorporate modern AI technologies into the work; integrate students; document significant potential for future funding or other growth-oriented outcomes; and highlight innovations.

The 2024 application deadline will be October 15, at 11:59 PM EST. Recipients will be notified by December 20, and projects are anticipated to commence at the start of the Spring 2025 semester.