The INS (International Neuroethics Society) AI and Consciousness Affinity Group is hosting a talk titled Bringing Trustworthiness in Generative AI and Agentic AI Using Thought Knowledge Graphs featuring speaker Manas Gaur, a computer scientist at UMBC.
The talk will examine the interplay between Thought Knowledge Graphs (TKGs) and how they can form more trustworthy and reasoning-based responses in AI. They will also discuss introducing novel methods on implementing TKGs and their overall impact on creating more trustworthy AI systems.
The talk will be held online via Zoom on Monday, December 2 at 1:00pm (EST).
Register to attend.
Abstract:
People shift their visual attention to gather and prioritize information from their surroundings, helping them navigate complex environments. Understanding these attentional shifts involves decoding the features that guide where attention is directed (spatial areas of focus) and when attention shifts (timing). Decoding these processes can aid applications from interface design to medical diagnosis. However, prior models have not fully explored the underlying factors addressing these aspects. In this dissertation, we study the factors that guide visual attention across diverse image types, spanning natural images, graphic design documents, and whole slide images (WSIs) of cancer tissues, while also predicting visual attention based on these factors.
First, we propose a method to quantify object recognition uncertainty as a factor influencing spatio-temporal attention (where and when) in natural images. We found that it plays a larger role than bottom-up saliency in guiding visual attention. Second, we analyze graphic design documents such as webpages, comics, posters, mobile UIs, etc., which differ from natural images in that they are designed to convey specific messages or elicit desired viewer response. We propose a unified and interpretable deep learning model that predicts both static and dynamic visual attention behavior (addressing where and when) by integrating document layout and content saliency as factors, enhancing attention prediction performance. Finally, in the domain of digital pathology, we investigate pathologists' attention during their examination of giga-pixel WSIs of prostate cancer with an objective to aid in the development of computer-assisted pathology training and clinical decision support systems. Using a digital microscope interface, we collected the largest known dataset of pathologist attention, which allows us to study the factors that guide their spatial and temporal attention patterns (where and when) and develop predictive models. Our study explores key factors guiding their attention, including magnification, slide staining, the nature of the diagnostic task, and their expertise. Motivated by this analysis, we propose deep learning models to solve two tasks: 1) predicting pathologist attention via spatial (heatmaps) and spatio-temporal (scanpaths) models, and 2) inferring pathologist expertise level, both essential technical components towards developing an AI-assisted pathology training pipeline.
Speaker:
Souradeep Chakraborty
Location: New Computer Science Bldg., Room 220
Zoom Link: https://stonybrook.zoom.us/j/
Meeting ID: 975 528 8447
Passcode: 338037
You are cordially invited to attend the biweekly Brookhaven AI Mixer (BAM). BAM includes three short talks on AI research happening at BNL, followed by an open mixer over coffee and snacks for everyone to network and discuss all things AI. The first half hour will consist of presentations that will be available via ZOOM, and the second half hour will be for in person only networking.
Join us every other Tuesday at noon in CDSD's Training Room (building 725, 2nd floor) to learn about interesting AI methods and applications, engage with potential collaborators, prepare for pending FASST funding calls, and build a community of AI for Science at BNL.
Tuesday, November 26, 2024, 12:00 pm -- CDS, Bldg. 725, Training Room
Speakers
Hanfei Yan, NSLS-II
David Park, CDS, AI Dept
Xihaier Luo, CDS, AI Dept
Join Zoom Meeting
https://bnl.zoomgov.com/j/
Meeting ID: 160 105 2863
Passcode: 442980
Reception to follow.
Abstract:
In this talk, I will present our journey of developing diverse, adaptive, uncertainty-calibrated AI planning agents that can robustly communicate and collaborate for multi-agent reasoning (on math, commonsense, coding, etc.) as well as for interpretable, controllable multimodal generation (across text, images, videos, audio, layouts, etc.). In the first part, we will discuss improving reasoning via multi-agent discussion among diverse LLMs and structured distillation of these discussion graphs (ReConcile, MAGDi), adaptively learning to balance abstraction, decomposition, refinement, and fast+slow thinking in LLM-agent reasoning (ReGAL, ADaPT, MAgICoRe, System-1.x), as well as confidence calibration in LLMs via speaker-listener pragmatic reasoning and making LLMs better teammates via multi-agent positive-negative persuasion balancing (LACIE, PBT). In the second part, we will discuss interpretable and control-lable multimodal generation via LLM-agents based planning and programming, such as layout-controllable image generation (and evaluation) via visual programming (VPGen+VPEval), consistent multi-scene video generation via LLM-guided planning (VideoDirectorGPT), interactive and composable any-to-any multimodal generation (CoDi, CoDi-2), as well as feedback-driven multi-agent interaction for adaptive environment/data generation via weakness discovery (EnvGen, DataEnvGym).
Bio:
Dr. Mohit Bansal is the John R. & Louise S. Parker Distinguished Professor and the Director of the MURGe-Lab (UNC-NLP Group) in the Computer Science department at UNC Chapel Hill. He received his PhD from UC Berkeley in 2013 and his BTech from IIT Kanpur in 2008. His research expertise is in natural language processing and multimodal machine learning, with a particular focus on multimodal generative models, grounded and embodied semantics, faithful language generation, and interpretable, efficient, and generalizable deep learning.
Abstract:
Artificial Intelligence for Science (AI4Sci) has become a transformative approach in
modeling and understanding complex physical systems, encompassing different scales such as
atomistic systems and continuum systems. In atomistic systems, AI has shown potential in
accelerating simulations, optimizing molecular dynamics, and predicting material and
molecular properties through data-driven approaches, enhancing computational efficiency
while preserving accuracy. For continuum systems, AI provides powerful tools for solving
partial differential equations (PDEs) and learning physical patterns from data, capturing
intricate dynamics that govern physical and engineering processes. This work explores AI
methods--particularly equivariance for neural networks and neural operators--bridging
atomistic and continuum representations. We analyze the implications of incorporating
symmetries to improve model robustness and learning efficiency, providing a cohesive AI- driven framework for advancing scientific discovery. The findings aim to underscore the role
of AI in enhancing accuracy, applicability, scalability, interopretability, and generalization
across scales, from molecular simulations to physical modeling, opening pathways for next- generation applications in computational science.
Biography:
Wenhan Gao is a third-year Ph.D. student in Applied Mathematics at Stony Brook University, where he works under the supervision of Professor Yi Liu. Wenhan's research focuses on
equivariant neural networks, graph neural networks, and AI for partial differential equations. Wenhan's work seeks to leverage the power of symmetries to aid AI models, particularly in
fields such as computer vision (image and video generation), physical simulation (modeling
climate change), and computational chemistry (drug discovery). He has published papers on
the aforementioned topics in leading venues like NeurIPS, Transactions on Machine Learning
Research (TMLR), and Journal of Computational Physics (JCP). He also has several preprints
under review in leading venues like ICLR and CVPR. In addition to his research, Wenhan has
served as a reviewer for top-tier conferences, including ICLR, NeurIPS, ICML, and KDD, and
as a lecturer for undergraduate and graduate courses at Stony Brook University. Wenhan was
awarded the NeurIPS Travel Award and Excellence in Teaching for Fall 2023.
Abstract:
In recent years, the landscape of artificial intelligence (AI) has been reshaped by the rapid emergence of Foundation Models (FMs). These versatile models have garnered widespread attention for their remarkable ability to transcend the boundaries of traditional, bespoke AI solutions and to generalize to a large set of downstream tasks. In this presentation we will describe the development of geospatial FMs with earth observation and weather data and discuss initial results of such models. We will also show how such foundation models can be a new and exciting tool for assisting with and accelerating scientific discovery.
Speaker:
Hendrik Hamann
Distinguished Researcher
IBM T.J. Watson Research Center
To truly understand human language, we must look at words in the context of the human generating the language. Factors such as demographics, personality, modes of communication, and emotional states have shown to play a crucial role in NLP models pre-LLMs era. Steps of mathematically defining the inclusion of human context in language modeling and more will be discussed with Nikita Soni, a PhD student at Stony Brook University co-advised by H. Andrew Schwartz and Niranjan Balasubramanian. She is the lead organizer of the workshop on human-centered large language modeling.
Please register for the STEM Speaker Series Zoom event here
Please RSVP for the STEM Speaker Series in-person event here
Abstract:
Deep learning models have achieved remarkable success across a wide range of computer vision tasks, including image classification, semantic segmentation, etc. However, such success highly relies on a large amount of annotated data, which are expensive to obtain. Moreover, their performance often degrades when there exist distribution shifts between training and test data. Domain Adaptation overcomes these issues by transferring knowledge from a label-rich source domain to a related but different target domain. Despite its popularity, domain adaptation is still a challenging task, especially when the data distribution shifts are severe, while the target domain has no or few labeled data.
In this thesis, I develop four efficient domain adaptation approaches to improve model performance on the target domain. Firstly, inspired by the large-scale pretraining of Vision Transformers, I explore Transformer-based domain adaptation for stronger feature representation and design a safe training mechanism to avoid model collapse in the situation of a large domain gap. Secondly, I observe that source models have low confidences on the target data. To address this, I focus on the penultimate activations of target data and propose an adversarial training strategy to enhance model prediction confidences. Thirdly, I study using weak supervision from prior knowledge about target domain label distribution. A novel Knowledge-guided Unsupervised Domain Adaptation paradigm is devised, and a plug-in module is designed to rectify pseudo labels. Lastly, I step into the task of Active Domain Adaptation, where the labels of a small portion of target data can be inquired. I propose a novel active selection criterion based on the local context and devise a progressive augmentation module to better utilize queried target data. The robustness of domain adaptation approaches, in addition to accuracy, is critical yet under-explored. To conclude the thesis, I empirically study set prediction in domain adaptation using the tool of conformal prediction and conformal training.
Location: New Computer Science Bldg., Room 120
Zoom Link: https://stonybrook.zoom.
Passcode: 466399
Abstract: Self-supervised representation learning (SRL) has emerged as a pivotal advancement in machine learning, offering high-quality data representations without the need for labeled datasets. While SRL has demonstrated enhanced adversarial robustness compared to supervised learning, its resilience against other attack types, particularly backdoor attacks, remains an open question. Recent studies have revealed potential vulnerabilities in SRL, underscoring the necessity for a comprehensive security analysis. However, existing research often extrapolates attacks from supervised learning paradigms, neglecting the unique challenges and opportunities inherent to self-supervised mechanisms.
This thesis proposal aims to address three critical objectives in the realm of self-supervised learning: (1) exploring novel attack vectors, (2) implementing and evaluating practical attacks, and (3) developing robust countermeasures. We focus on two key SRL paradigms: Contrastive Learning and Diffusion Models. For Contrastive Learning, we synthesize existing security vulnerabilities and introduce innovative attack vectors, such as CTRL, to uncover distinctive risks. We conduct a comparative analysis of contrastive and supervised learning approaches in their defense against these threats, exploring potential safeguards and highlighting the limitations of current protective measures in self-supervised contexts. Regarding Diffusion Models, we demonstrate inherent vulnerabilities in their application to adversarial purification.
Our research aims to illuminate the unique challenges posed by emerging attack vectors in self-supervised learning, fostering technical advancements to address underlying security risks in real-world applications. By contributing to the development of more resilient and secure self-supervised representation learning systems, we seek to enhance their reliability and trustworthiness in practical scenarios. This comprehensive examination of SRL's security landscape will provide valuable insights for the broader machine-learning community and pave the way for more robust AI systems.
Join here.
You are cordially invited to attend the biweekly Brookhaven AI Mixer (BAM). BAM includes three short talks on AI research happening at BNL, followed by an open mixer over coffee and snacks for everyone to network and discuss all things AI. The first half hour will consist of presentations that will be available via ZOOM, and the second half hour will be for in person only networking.
Join us every other Tuesday at noon in CDSD's Training Room (building 725, 2nd floor) to learn about interesting AI methods and applications, engage with potential collaborators, prepare for pending FASST funding calls, and build a community of AI for Science at BNL.
Tuesday, November 12, 2024, 12:00 pm -- CDS, Bldg. 725, Training Room
Speakers
Carlos Soto, CDS
Yi Huang, CDS
Kevin Yager, CFN