Talk by Zhenhua Liu to be followed by AI Institute updates


Abstract: Decision making with uncertainty has been studied in multiple communities extensively. Recently, online optimization has gained popularity partially because of its promising performance guarantees by incorporating predictions. In this talk, I will provide an overview of our work on algorithm designs for online optimization and its applications. Then, I will talk about our recent work in ACM Sigmetrics 2019 on choosing predictions and control algorithms simultaneously and dynamically. Finally, I will discuss some ongoing efforts and collaboration opportunities.

Bio: Zhenhua Liu is currently an assistant professor in the Department of Applied Mathematics and Statistics at Stony Brook University. He is also affiliated with the Department of Computer Science, the AI Institute and the Smart Energy Technology Cluster. He received his PhD degree in Computer Science from California Institute of Technology. His current research interests include cloud computing, online optimization and learning, smart grid, market design and distributed control. His research combines rigorous analysis and system design, and goes from theory, to prototype, and eventually to industry to make real impacts.

Abstract: Large Language Models (LLMs) have revolutionized how people interact with knowledge, offering unprecedented opportunities to accelerate the pace of scientific discovery. In this talk, I will discuss my research on the synergy between LLMs and scientific knowledge--specifically how these models extract, induce, and verify knowledge to automate the research lifecycle. First, I will cover our work on improving knowledge extraction from vast scientific literature, focusing on enabling models to comprehend long documents in a cost-efficient and comprehensive manner. I will describe a novel paradigm for representing document-level structured information as question-answer pairs and how we address the challenges of long-context understanding by leveraging global context through retrieval-augmented modeling. Next, I present our pioneering work on using LLMs for new scientific hypothesis generation. We introduce a framework employing reinforcement learning with fine-grained reward modeling and adaptive controllers.
This approach balances novelty, feasibility, and effectiveness to generate inspiring and actionable research hypotheses. Finally, I will discuss work on the first LLM Scientist for machine learning research. I will demonstrate how LLMs can move beyond hypothesis generation to participate in the execution and validation of scientific hypotheses, ensuring that the discovered knowledge is not only innovative but also grounded and verified.

Bio: Xinya Du is a tenure-track assistant professor at UT Dallas Computer Science Department. He earned a Ph.D. degree from Cornell University and was a Postdoctoral Research Associate at the University of Illinois (UIUC). He has also worked at Microsoft Research, Google Research, and Allen Institute AI. His research is on large language models, deep learning, and their applications in science.His work has been published in leading NLP and ML conferences (ACL, ICLR, NeurIPS). His research has received multiple recognitions, including a Best Paper Award at AAAI AI for Research and a Best Poster Award at ICML AI for Science workshop. His work was included in the list of Most Influential ACL Papers and has been covered by major media like New Scientist. He was named a Spotlight Rising Star in Data Science by the University of Chicago and is the recipient of several prestigious awards, including the Amazon Research Award, Cisco Research Award, Open Philanthropy Award, and the NSF CAREER Award.

Location: NCS 120

Talk Title: Knowledge-enhanced LLMs and Human-AI Collaboration Frameworks for Creativity Support


Abstract:

Large language models (LLMs) constitute a paradigm shift in Natural Language Processing and Artificial Intelligence. To build AI systems that are human-centered, I propose we need knowledge-aware models and human-AI collaboration frameworks to help them solve tasks ultimately aligning these models better with human values. In this talk, I will discuss my research agenda for human-centered AI with a case study on creativity that focuses on how to augment LMs with external knowledge, build effective human-AI collaboration frameworks as well as theoretically grounded robust evaluation protocols for measuring capabilities of NLG systems. I will begin by describing knowledge-enhanced methods for creative text generation such as metaphors. Next, I will describe how content creators can collaborate and benefit from the creative capabilities of text-to-image-based AI models. Finally, I will focus on the design and development of theoretically grounded evaluation protocols to benchmark the creative capabilities of Large Language Models in both producing as well as assessing creative text. I will end this talk by highlighting the current limitations of existing models and future directions toward building better models that will enable efficient and trustworthy human-AI collaboration systems.


Bio:

Tuhin Chakrabarty is a final-year Ph.D. candidate in the Natural Language Processing group within the Computer Science department at Columbia University. His research is supported by the Columbia Center of Artificial Intelligence & Technology (CAIT) & an Amazon Science Ph.D. Fellowship. He was also a Computational Journalism fellow at NYTimes R&D and an intern at the Allen Institute of Artificial Intelligence, Salesforce Research, and Deepmind. His research interests are broadly in Natural Language Processing, Computer Vision, and Human-Computer Interaction with a special focus on Human-Centered Methods for Understanding, Generation, and Evaluation of Creativity. His work has been recognized at top natural language processing and human-computer interaction conferences and journals such as ACL, NAACL, EMNLP, TACL, and CHI. He has been involved in organizing several workshops and tutorials at NLP conferences such Figurative Language Processing workshop at EMNLP 2022, NAACL 2024, and the tutorial on Creative Text Generation at EMNLP 2023. His work on AI and creativity has been mentioned in mainstream news media such as The Hollywood Reporter and more recently The Washington Post.

Join Zoom Meeting https://stonybrook.zoom.us/j/97103601583?pwd=TnpGMXdpeEd1N0hZcXppS1BLNFJhZz09 (ID: 97103601583, passcode: 004031) Join by phone (US) +1 646-931-3860 (passcode: 004031) Joining instructions: https://www.google.com/url?q=https://applications.zoom.us/addon/invitation/detail?meetingUuid%3DILacj94mRvSXgTYt0Cqs1w%253D%253D%26signature%3D9f2f1e7e603bbcb9034724d084eea8846c19a38b7436180170dfc3f1d718b425%26v%3D1&sa=D&source=calendar&usg=AOvVaw3MsNgLSPMRl8L5i6BosYrB Meeting host: H.Andrew.Schwartz@stonybrook.edu

Join Zoom Meeting:
https://stonybrook.zoom.us/j/97103601583?pwd=TnpGMXdpeEd1N0hZcXppS1BLNFJhZz09

You are cordially invited to attend the biweekly Brookhaven AI Mixer (BAM). BAM includes one short talk on AI research happening at BNL, followed by an open mixer over coffee and snacks for everyone to network and discuss all things AI. The first half hour will consist of presentations that will be available via ZOOM, and the second half hour will be for in person only networking.

Join us every other Tuesday at noon in CDSD's Training Room (building 725, 2nd floor) to learn about interesting AI methods and applications, engage with potential collaborators, prepare for pending FASST funding calls, and build a community of AI for Science at BNL.

Abstract: The increasing complexity and volume of data from electron microscopy necessitates advanced computational tools for timely and accurate analysis. In this talk, I will present several machine learning (ML) models developed to interpret diverse datasets from transmission electron microscopy (TEM). First, I demonstrate segmentation models for labelling regions of interest from in situ TEM images, such as atomic column positions or reaction sites that allow atomic-level quantitative analysis of data. Second, I introduce a self-supervised CNN model for denoising of low-dose HRTEM images, enabling clearer visualization of atomic features without sacrificing temporal resolution. Finally, a transformer-based model trained to predict copper oxidation states directly from their electron energy loss spectroscopy spectra will be introduced. Together, these projects showcase the power of tailored ML solutions to extract quantitative insights from complex microscopy data.

Biography: Brian Lee is a research associate working for the Electron Microscopy group and Theory and Computation group at the Center for Functional Nanomaterials. Previously, he has received PhD in Mechanical Engineering from Duke University and worked as a postdoc at Purdue University. His research focuses on applying machine learning and simulation techniques for materials science.

Location: CDS, Bldg. 725, Training Room

Join ZoomGov Meeting: https://bnl.zoomgov.com/j/1604383624?pwd=ffQ5cUPNxTI7nzClKQO6cnsNbhF9Vf.1

Meeting ID: 160 438 3624
Passcode: 558449

You are cordially invited to attend the biweekly Brookhaven AI Mixer (BAM). BAM includes one short talk on AI research happening at BNL, followed by an open mixer over coffee and snacks for everyone to network and discuss all things AI. The first half hour will consist of presentations that will be available via ZOOM, and the second half hour will be for in person only networking.

Join us every other Tuesday at noon in CDSD's Training Room (building 725, 2nd floor) to learn about interesting AI methods and applications, engage with potential collaborators, prepare for pending FASST funding calls, and build a community of AI for Science at BNL.

Embodied Intelligence at Scientific User Facilities

Abstract: This presentation explores the active work integrating artificial intelligence and robotics at the National Synchrotron Light Source II, and a perspective for the future. Through various case studies, we highlight the optimization of operations, improved experimental outcomes, and the orchestration of distributed multimodal experiments. This ongoing development includes collaborators from across the light and neutron sources in the DOE complex. We will elaborate on the open-source Bluesky project, and its capabilities to support adaptive and autonomous experiments. Additionally, we will discuss how Bluesky can be integrated with open-source robotic control software to unlock new flexible automation for autonomous scientific research, which scales to new experiments and continues to leverage human ingenuity.

Biography: Dr. Phillip M. Maffettone is an Associate Computational Scientist in the Data Science and Systems Integration Division at NSLS-II. His research focuses on accelerating scientific discovery at user facilities through the integration of robotics, artificial intelligence (AI), and advanced experiment orchestration systems. He leads the N3XTware project, constructing the software architecture for the next 12 beamlines to be built at NSLS-II. Prior to this he built the brain on the world's first mobile robotic scientist at the University of Liverpool, and later spearheaded the machine learning platform for a biotechnology start-up, BigHat Biosciences. He holds a DPhil in Inorganic Chemistry from the University of Oxford and a B.S. in Chemical Engineering from the University at Buffalo.

Location: CDS, Bldg. 725, Training Room

Link: https://bnl.zoomgov.com/j/16049713 31?pwd=nc5CV3cOFrdYxordFieP W07tIDmwYb.1

Meeting ID: 160 497 1331
Passcode: 289875

IACS Research Theme: Human Centered Computing Seminar

Abstract: The AI art platform Artbreeder hosts daily remix parties where users build on each other's work, creating transparent evolutionary chains of images from a single seed. This study analyzes 130,882 images from 368 remix parties to identify the drivers of novelty, complexity, and competitive success. The results reveal an interesting tension: while more novel parent images produce more novel and complex children and attract more likes, users paradoxically prefer to remix images that are less novel and complex. At the group level, larger remix parties produce more novelty at the cost of lower complexity. Additionally, images tend to converge towards common thematic attractors (e.g., steampunk scenes, alien architecture, furries) over the course of remix parties. These results provide quantitative insights into collective creativity--the production of novelty by groups of people--a typically opaque aspect of human cultural evolution.

Speaker: Dr. Mason Youngblood

Location: Institute for Advanced Computational Science, Seminar Room

To truly understand human language, we must look at words in the context of the human generating the language. Factors such as demographics, personality, modes of communication, and emotional states have shown to play a crucial role in NLP models pre-LLMs era. Steps of mathematically defining the inclusion of human context in language modeling and more will be discussed with Nikita Soni, a PhD student at Stony Brook University co-advised by H. Andrew Schwartz and Niranjan Balasubramanian. She is the lead organizer of the workshop on human-centered large language modeling.

Please register for the STEM Speaker Series Zoom event here

Please RSVP for the STEM Speaker Series in-person event here
The overall purpose of this seminar is to bring together people with interests in Computer Vision theory and techniques and to examine current research issues. This course will be appropriate for people who already took a Computer Vision graduate course or already had research experience in Computer Vision. To enroll in this course, you must either: (1) be in the PhD program or (2) receive permission from the instructors.

Each seminar will consist of multiple short talks (around 10 minutes) by multiple people. Students can register for 1 credit for CSE 656. Registered students must attend and present a minimum of 2 or 3 talks. Everyone else is welcome to attend. Fill in https://forms.gle/pCVXovgfMfQwGqG38 to subscribe to our mailing list for further announcement.
Topic: AI Seminar: Owen Rambow
Time: Mar 17, 2021 10:00 AM Eastern Time (US and Canada)
Join Zoom Meeting

https://stonybrook.zoom.us/j/93614644178?pwd=MzJtVDJYYmU5T1dtMzJiUFMxb0x4dz09
Meeting ID: 936 1464 4178.    Passcode: 965936






Natural Language Understanding and Semantic Parsing

(Partly joint work with former colleagues at Elemental Cognition)

Semantic parsing refers to the task of determining the propositional content of language: who did what to whom.  It is part of the larger task of natural language understanding (NLU).  I will start out by discussing what full NLU means, and argue that we are still far away, as a field, from solving full NLU, or even from knowing how to evaluate it.

In the second part of the talk, I will situate semantic parsing in the context of several other NLU subtasks.  Typically, the target representation of semantic parsing uses an ontology (such as PropBank or FrameNet).  Semantic parsing includes the subtasks of word sense disambiguation, argument detection, and argument role labeling.  I will discuss choices among possible target ontologies.  I will justify why we created a new ontology, Hector, based on FrameNet and the lexical resource NOAD, and explain some of its characteristics.

In the third part of the talk, I will present experiments we performed using transformer models.  We obtain best results using a two-phase model, in which we first choose the frame, and then, given the frame, choose the arguments.  We encode the problem for both tasks using indices in the sentence.  While we develop the parser for our new ontology Hector, this approach also beats the state of the art for FrameNet and PropBank parsing.Biography:  I am a professor in the Department of Linguistics at Stony Brook University with a joint appointment in IACS.

Until recently, I was a research scientist at Elemental Cognition. Elemental Cognition is working on deep natural language understanding.

I got my PhD with Aravind Joshi at the University of Pennsylvania in 1994. I have worked at CoGenTex, and at AT&T Labs -- Research, and for many years I was a research scientist at Columbia University in the Center for Computational Learning Systems.