Time: Mar 17, 2021 10:00 AM Eastern Time (US and Canada)
Join Zoom Meeting
https://stonybrook.zoom.us/j/
Meeting ID: 936 1464 4178. Passcode: 965936
Natural Language Understanding and Semantic Parsing
(Partly joint work with former colleagues at Elemental Cognition)
Semantic parsing refers to the task of determining the propositional content of language: who did what to whom. It is part of the larger task of natural language understanding (NLU). I will start out by discussing what full NLU means, and argue that we are still far away, as a field, from solving full NLU, or even from knowing how to evaluate it.
In the second part of the talk, I will situate semantic parsing in the context of several other NLU subtasks. Typically, the target representation of semantic parsing uses an ontology (such as PropBank or FrameNet). Semantic parsing includes the subtasks of word sense disambiguation, argument detection, and argument role labeling. I will discuss choices among possible target ontologies. I will justify why we created a new ontology, Hector, based on FrameNet and the lexical resource NOAD, and explain some of its characteristics.
In the third part of the talk, I will present experiments we performed using transformer models. We obtain best results using a two-phase model, in which we first choose the frame, and then, given the frame, choose the arguments. We encode the problem for both tasks using indices in the sentence. While we develop the parser for our new ontology Hector, this approach also beats the state of the art for FrameNet and PropBank parsing.Biography: I am a professor in the Department of Linguistics at Stony Brook University with a joint appointment in IACS.
Until recently, I was a research scientist at Elemental Cognition. Elemental Cognition is working on deep natural language understanding.
I got my PhD with Aravind Joshi at the University of Pennsylvania in 1994. I have worked at CoGenTex, and at AT&T Labs -- Research, and for many years I was a research scientist at Columbia University in the Center for Computational Learning Systems.
This is Stony Brook's quantum moment. Join us for a spotlight on the core achievements and research excellence of faculty across the Colleges of Arts and Sciences (CAS), and Engineering and Applied Sciences (CEAS) - and their collaborative advancements in quantum science and technology. Learn about the real world impact of their enduring work, their leadership in translating foundational science into entrepreneurial opportunities, and their impetus for making connections to next generation innovation.
Presented by: Catherine Chen, Ph.D., Research Development Associate
Welcome remarks: President Andrea Goldsmith
Panel moderators: Dean David Wrobel, CAS, and Dean Andrew Singer, CEAS
Presentations and panel featuring our faculty:
Jennifer Cano, CAS, Physics and Astronomy
P. Scott Carney, CEAS, Mechanical Engineering
Hyeongrak Chuck Choi, CEAS, Electrical and Computer Engineering
Eden Figueroa, CAS, Physics and Astronomy
Humanshu Gupta, CEAS, Computer Science
Angela Kelly, CAS, Physics and Astronomy
Location: Theatre at the Charles B. Wang Center, Stony Brook University
Reserve your tickets by March 26!
Join us for a mix of 30-minute virtual sessions, an in-person kickoff on Monday, and a student-focused event on Wednesday celebrating data and data-informed decision-making.
Wrap up the week at the Love Data Week Open House on Friday, 2/13 with light refreshments, data-themed swag, photos with Wolfie, and time to connect with presenters.
Learn more and register on https://www.stonybrook.edu/commcms/oee/recognition/Love%20Data%20Week%202026%20Save%20the%20Date%20Placeholder.php
Abstract: Computational pathology has revolutionized cancer diagnosis and research through the analysis of digitized whole slide images (WSIs). However, the giga-pixel size of these images presents profound technical challenges, creating two intertwined bottlenecks: computational inefficiency and label inefficiency. The immense data scale makes standard end-to-end (E2E) training of deep neural networks infeasible due to prohibitive GPU memory requirements, while the reliance on expert pathologists for annotations makes obtaining high-quality labeled data a tedious and expensive process. This proposal confronts these dual challenges by developing a series of novel model architectures, training paradigms, and self-supervised learning methods designed to create a more efficient and effective framework for WSI analysis.
To improve computational efficiency, this proposal first introduces a locally supervised learning paradigm that enables E2E training on entire WSIs by partitioning a network into gradient-isolated modules, circumventing the memory bottleneck of backpropagation. Second, it presents Prompt-MIL, a parameter-efficient fine-tuning framework that reduces the number of trainable parameters, memory consumption, and training time by fine-tuning only few prompts to guide large pre-trained models. Third, this work advances the efficient architecture on WSIs by developing novel State-Space Models (SSMs). It proposes 2DMamba, the first intrinsic Mamba architecture that preserves the crucial 2D spatial structure of images, overcoming the spatial discrepancy inherent in 1D models. Fourth, to address the inefficiency of multi-directional scans in Mamba models, including 2DMamba, it presents Locally Bi-directional Mamba (LBMamba), which introduces a novel, hardware-aware local backward scan that integrates bi-directional scan into a single forward pass, significantly improving throughput performance trade-off. Lastly, it proposes an extension to the LBMamba, warp-level Bi-directional Mamba (WLBMamba) that extends the thread-level bidirectional scan to warp-level bidirectional scan that further improves the throughput performance trade-off.
To improve label efficiency, this proposal proposes a Precise Location-based Matching strategy for self-supervised dense contrastive learning. By allowing a local patch in one augmented view to match multiple overlapping patches in another, creates a more accurate correspondence, leading to superior feature representations for dense prediction tasks like segmentation and detection.
In summary, this proposal presents a holistic investigation into the efficiency bottlenecks in computational pathology. Through these combined contributions in model architecture, training paradigms, and self-supervised learning, this work establishes a more scalable, efficient, and powerful computational framework for analyzing giga-pixel pathology images.
Speaker: Jingwei Zhang
Location: Old Computer Science Room 2114
Zoom: https://stonybrook.zoom.
Meeting ID: 951 8790 3649 | Passcode: 488916
Time:
Sep 7, Tue, 11:00am EDT
Place:
NCS 220 or on Zoom (info below)
Title: Data-Driven Document Unwarping
Abstract:
Capturing document images is a common way to digitize and record physical documents due to the ubiquitousness of mobile cameras. To make text recognition easier, it is often desirable to digitally flatten a document image when the physical document sheet is folded or curved. However, unwarping a document from a single image in natural scenes is very challenging due to the complexity of document sheet deformation, document texture, and environmental conditions. Previous model-driven approaches struggle with inefficiency and limited generalizability. In this thesis, I investigate several data-driven approaches to tackle the document unwarping problem.
Data acquisition is the central challenge in data-driven methods. I first design an efficient data synthesis pipeline based on 2D image warping and train DocUNet, the pioneering data-driven document unwarping model, on the synthetic data. A benchmark dataset is also created to facilitate comprehensive evaluation and comparison. To improve the unwarping performance by training on more realistic data, I introduce the Doc3D dataset and DewarpNet. Supervised by 3D shape ground truth in Doc3D, DewarpNet is significantly better than DocUNet. DocUNet and DewarpNet depend on the synthetic data for the ground truth deformation annotation. To exploit the real-world images, I propose PaperEdge, a weakly supervised model trained with in-the-wild document images with easy-to-obtain boundary information. PaperEdge surpasses DewarpNet by utilizing both the synthetic data and weakly annotated real data in the Document In the Wild (DIW) dataset. Finally, I propose directly predicting the $uv$ parameterized 3D mesh of the document with 3D constraints and using the accessible 3D presentations like depth maps as training targets. Predicting the 3D mesh of the document solves the unwarping task and also benefits VR/AR applications.
Join Zoom Meeting
https://stonybrook.zoom.us/j/
Meeting ID: 964 4059 2912
Passcode: 793149
One tap mobile
+16468769923,,96440592912# US (New York)
+13017158592,,96440592912# US (Washington DC)
Dial by your location
+1 646 876 9923 US (New York)
+1 301 715 8592 US (Washington DC)
+1 312 626 6799 US (Chicago)
+1 253 215 8782 US (Tacoma)
+1 346 248 7799 US (Houston)
+1 408 638 0968 US (San Jose)
+1 669 900 6833 US (San Jose)
Meeting ID: 964 4059 2912
Find your local number: https://stonybrook.
Over the past decade, researchers in neuroscience, psychology and artificial intelligence have come together to build advanced computer models that mimic how our brain processes what we see. These models are designed to closely copy the brain's visual system, all the way to a key area called the inferior temporal cortex, which plays an important role in recognizing objects.
Because these computer models can be fully observed, scientists can use them to make detailed predictions about how the brain works -- something older, more theoretical models could not do.
Dr. James DiCarlo's work explores whether these computer digital twin models of the brain could help guide safe, non- invasive ways to infl uence brain activity. In his talk, he explains how such a model could be used to design specific patterns of light. When this carefully designed light is added to what the eye naturally sees, it can precisely influence activity in groups of neurons in the inferior temporal cortex.
Since neural activity in this visual brain area may be connected to emotional states like anxiety, this research could eventually open the door to non-invasive approaches that may benefit mental well-being in the future.
Speaker: James J. DiCarlo, MD, PhD, Peter de Florez Professor, MIT Brain and Cognitive Sciences, and Director, MIT Siegel Family Quest for Intelligence
Location: Staller Center Main Stage
The event will be livestreamed at stonybrook.edu/live
Tuesday, January 28, 2:00 PM to 3:00 PM
In-person: New Computer Science, Seminar Room 120
Zoom link: https://stonybrook.zoom.us/j/91265872116?pwd=gaIUbmJavuafURujQjEa6AdVMs4d54.1
Meeting ID: 912 6587 2116
Passcode: 189681