Time:
Sep 7, Tue, 11:00am EDT

Place:
NCS 220 or on Zoom (info below)

Title: Data-Driven Document Unwarping


Abstract:
Capturing document images is a common way to digitize and record physical documents due to the ubiquitousness of mobile cameras. To make text recognition easier, it is often desirable to digitally flatten a document image when the physical document sheet is folded or curved. However, unwarping a document from a single image in natural scenes is very challenging due to the complexity of document sheet deformation, document texture, and environmental conditions. Previous model-driven approaches struggle with inefficiency and limited generalizability. In this thesis, I investigate several data-driven approaches to tackle the document unwarping problem.

Data acquisition is the central challenge in data-driven methods. I first design an efficient data synthesis pipeline based on 2D image warping and train DocUNet, the pioneering data-driven document unwarping model, on the synthetic data. A benchmark dataset is also created to facilitate comprehensive evaluation and comparison. To improve the unwarping performance by training on more realistic data, I introduce the Doc3D dataset and DewarpNet. Supervised by 3D shape ground truth in Doc3D, DewarpNet is significantly better than DocUNet. DocUNet and DewarpNet depend on the synthetic data for the ground truth deformation annotation. To exploit the real-world images, I propose PaperEdge, a weakly supervised model trained with in-the-wild document images with easy-to-obtain boundary information. PaperEdge surpasses DewarpNet by utilizing both the synthetic data and weakly annotated real data in the Document In the Wild (DIW) dataset. Finally, I propose directly predicting the $uv$ parameterized 3D mesh of the document with 3D constraints and using the accessible 3D presentations like depth maps as training targets. Predicting the 3D mesh of the document solves the unwarping task and also benefits VR/AR applications.

Join Zoom Meeting
https://stonybrook.zoom.us/j/96440592912?pwd=ZU5waTdyUzRFNW5SRHM5ME84TWdFQT09

Meeting ID: 964 4059 2912
Passcode: 793149
One tap mobile
+16468769923,,96440592912# US (New York)
+13017158592,,96440592912# US (Washington DC)

Dial by your location
        +1 646 876 9923 US (New York)
        +1 301 715 8592 US (Washington DC)
        +1 312 626 6799 US (Chicago)
        +1 253 215 8782 US (Tacoma)
        +1 346 248 7799 US (Houston)
        +1 408 638 0968 US (San Jose)
        +1 669 900 6833 US (San Jose)
Meeting ID: 964 4059 2912
Find your local number: https://stonybrook.zoom.us/u/adxTt9ZbuJ

Abstract: Theory-internal work on opacity in phonology has been focused on the challenges these interactions present for one theory (rules, constraints) versus another. But there has also been interest in studying the formal, invariant properties of opaque and other process interactions (Chandlee et al. 2018; Bakovic and Blumenfeld 2024), though these works crucially differ in their underlying assumptions. In this talk I will recontextualize Chandlee et al. (2018)'s result that opaque maps are ISL in light of Bakovic and Blumenfeld (2024)'s recent formal typology of process interactions, and this recontextualization will provide an answer to an open question about the k-value of an interaction map. I will then discuss the implications of this collective formal understanding of opacity for a recent model of lexicon and phonological grammar learning (i.e., Hua and Jardine 2021, Chandlee and Jardine to appear).


Speaker: Prof. Jane Chandlee, Associate Professor in the Department of Linguistics at Haverford College

Location: IACS Seminar room.

The Antonija Prelec Memorial Committee in collaboration with Stony Brook University Libraries are very excited to bring you the 2019 Prelec Memorial Lecture! This year, we are pleased to announce our speaker is Patricia Flatley Brennan, RN, PhD, Director of the National Library of Medicine.

No registration required. Find more information here.

Title:Deep Contextual Modeling for Natural Language Understanding, Generation, and Grounding Zoom instructions: Join Zoom Meeting https://stonybrook.zoom.us/j/645050299?pwd=TVJVRkc3dlhxdDF5d00xWGlDQkovZz09 Meeting ID: 645 050 299 Password: 810247 One tap mobile +16468769923,,645050299#,,#,810247# US (New York) +13126266799,,645050299#,,#,810247# US (Chicago) Dial by your location +1 646 876 9923 US (New York) +1 312 626 6799 US (Chicago) +1 301 715 8592 US +1 346 248 7799 US (Houston) +1 408 638 0968 US (San Jose) +1 669 900 6833 US (San Jose) +1 253 215 8782 US Meeting ID: 645 050 299 Password: 810247 Find your local number: https://stonybrook.zoom.us/u/aemTiJMXu6 Abstract: Natural language is a fundamental form of information and communication. In both human-human and human-computer communication, people reason about the context of text and world state to understand language and produce language response. In this talk, I present several deep neural network based systems that first understand the meaning of language grounded in various contexts where the language is used, and then generate effective language responses in different forms for information access and human-computer communication. First, I will introduce Speaker Interaction RNNs for addressee and response selection in multi-party conversations based on explicit representations for different discourse participants. Then, I will present a text summarization approach for generating email subject lines by optimizing quality scores in a reinforcement learning framework. Finally, I will show an editing-based multi-turn SQL query generation system towards intelligent natural language interfaces to databases. Bio:Rui Zhang is a final year Ph.D. student at Yale University advised by Professor Dragomir Radev. His research interest lies in various natural language processing problems in understanding, generation, and grounding. He has been working on (1) End-to-End Neural Modeling for Entities, Sentences, Documents, and Multi-party Multi-turn Dialogues, (2) Text Summarization for Emails, News, and Scientific Articles, (3) Cross-lingual Information Retrieval for Low-Resource Languages, (4) Context-Dependent Text-to-SQL Semantic Parsing in Human-Computer Interaction. Rui Zhang has published papers and served as Program Committee members at top-tier NLP and AI conferences including ACL, NAACL, EMNLP, AAAI, CoNLL. During his Ph.D., He has done research internships at IBM Thomas J. Watson Research Center, Grammarly Research, and Google AI. He was a graduate student at the University of Michigan and got his bachelor's degrees at both the University of Michigan and Shanghai Jiao Tong University from the UM-SJTU Joint Institute.

Abstract: Pretraining vision encoders with self-supervision (SSL) leads to stronger representations that excel across diverse downstream tasks. One of the key factors enabling self-supervision is extracting multiple views of the same scene to formulate either: 1) View-invariant pretraining (DINO, SimCLR, iBOT), where the objective is predicting the same representation for different views of the scene; or 2) Cross-view pretraining (cross-view Masked Autoencoders), where the objective is predicting missing parts of one view using other views. For extracting multiple views, view-invariant methods rely on a combination of handcrafted augmentations (random cropping, color jittering, gaussian blur, etc.) of the same image, whereas cross-view pretraining methods rely on image cropping or video frames. In this work, we present methods to effectively incorporate synthetic views from diffusion models into SSL training.
For view-invariant pretraining, we introduce Gen-SIS, a method that leverages the ability of diffusion models to generate interpolated images through interpolation in conditioning space. We introduce a disentanglement pretext task: disentangling two source images from an interpolated synthetic image. This disentanglement task, in addition to vanilla single-source generative augmentation for view extraction, improves visual pretraining of various view-invariant methods (DINO, SimCLR, iBOT).
For cross-view pretraining, we introduce CDG-MAE, a novel cross-view masked autoencoder (MAE) based method that uses diverse synthetic views generated from static images via an image-conditioned diffusion model to learn dense correspondences. We present a quantitative method to evaluate the local and global consistency of the generated views to choose the right diffusion model for cross-view pretraining. These generated views exhibit substantial changes in pose and perspective, providing a rich training signal that overcomes the limitations of video (expensive) and crop-based (less variation) methods. CDG-MAE substantially narrows the gap to video-based MAE methods on video label propagation tasks while maintaining the data advantages of image-only MAEs.

Speaker: Varun Belagali

Location: NCS 120
Zoom: https://stonybrook.zoom.us/j/93647452432?pwd=hZaX7LXCAD8KPHWYE1Afw2sDI3owpv.1

Educational objectives:

1. Describe how bias enters AI systems across the development pipeline, using aesthetic facialevaluation as a case study, and identify mitigation strategies applicable at each stage.
2. Apply the PROBAST+AI risk-of-bias framework to critically appraise the methodologicalquality of a published prediction model using regression or machine learning methods.
3. Compare the discriminative performance of multiple modeling architectures against anexisting clinical risk calculator on a large benchmarking dataset.
4. Recognize the complementary roles of clinical expertise and informatics methodology indefining clinically meaningful AI project objectives, predictors, and evaluation criteria.

Speaker: Dr. Anisha R. Kumar, MD Clinical Assistant Professor, Surgery and Biomedical Informatics, Stony Brook University School of Medicine


Location: MART Building, Room 7M-0602 (7th Floor)

Remote Access: https://stonybrook.zoom.us/j/95617197636?pwd=KytzZ2pVRG9SZGpKZUtpNXJISjNjZz09Meeting ID: 95617197636 | Passcode: 924293

Abstract: Trustworthy AI deployment in high-stakes domains requires systems that are fair, private, robust, and controllable as they scale. Yet these demands are often pursued through ad-hoc approaches, lacking a systematic understanding of the inherent trade-offs between competing objectives. We add fairness regularizers and hope bias decreases. We train on massive datasets and hope the model learns the underlying logic of how concepts combine, rather than memorizing statistical shortcuts. We encrypt data and hope the resulting computational overhead remains manageable. But hope isnot a science.
In this talk, I argue that what trustworthy AI lacks is not better heuristics but a deeper science of what these properties fundamentally cost and what is achievable. Before we can fix a system, we must map the terrain: what trade-offs are unavoidable, what regions of performance areunreachable, and how far current methods fall from what is actually achievable. My research builds this map across fairness, privacy, robustness, and controllability, following a common methodology: diagnose where models fail, characterize the fundamental limits any method must obey, and design systems that approach those limits. I will present this framework, its extension to scientific applications where we replace statistical constraints with physical laws to ensure AI systems remain grounded in reality, and a vision for scaling these principles to the rapidly expanding ecosystem of composed and interacting AI systems.


Bio: Dr. Vishnu Boddeti is an Associate Professor in the Department of Computer Science and Engineering at Michigan State University, where he leads the Human Analysis Lab (HAL). His research develops mathematical frameworks for trustworthy AI, spanning fairness, privacy, robustness, and physics-informed learning, with an emphasis on characterizing fundamental limits and building systems that achieve them. His work has been supported by NSF, NIST, DARPA, ONR, Ford, and others, and recognized with a Meta Research Award (2021). His research has been featured on the cover of Nature, recognized as an Editor's Highlight in Nature Communications, and received multiple best paper awards, including the 2024 IEEE-CCF Cloud Computing Best Paper Award and the TMLR Outstanding Certification Finalist (2023). He serves as Senior Area Editor for IEEE Transactions on Information Forensics and Security and completed his PhD in ECE from Carnegie Mellon University in 2012.

Location: NCS 120