AI Innovation Institute

Matthew Salzano (Stony Brook), AI and DEIA: Getting at the Roots

Link to the talk (no pre-registration required this time): https://stonybrook.zoom.us/j/96209347479?pwd=Cs8fEfFdbXrGTC5cQgyHRb8Msh5vp8.1Meeting ID: 962 0934 7479 Passcode: 272489

Abstract: Conversations about AI and DEIA (Diversity, Equity, Inclusion, and Access) often unwittingly assume that social problems can and should have technical fixes. Left unaddressed, scholars, advocates, and technologists inevitably miss important consequences in our proposed solutions, and focus on surface-level problems rather than addressing the root causes of inequity. Drawing from scholarship in communication, rhetoric, and critical digital studies, this talk explains how we are often trimming branches when we need to pull out roots -- and introduces new terms and questions that can help reorient our conversations about AI and DEIA.

Speaker Bio: Matthew Salzano, Ph.D., is a communication scholar researching new media technologies, user practices, and cultural trends that threaten to limit possibilities for diverse engagement in public argument, debate, and protest. His scholarship has appeared in journals like The Quarterly Journal of Speech, Critical Studies in Media Communication, and Women's Studies in Communication, and his research on DEIA, AI, and advocacy communications has been funded by the Waterhouse Family Institute at Villanova University. He is currently an Inclusion, Diversity, Equity, and Access fellow in Ethical AI at Stony Brook University's School of Communication and Journalism and Alan Alda Center for Communicating Science.

Public Interest Technology Research and Education by Ellen Zegura, Georgia Tech 2019 * 2020 Shutterstock Distinguished Lecture Series

Time: Jan 26, 2021 03:00 PM Eastern Time (US and Canada)

All are welcome!

Zoom Meeting:
https://stonybrook.zoom.us/j/93818552212?pwd=ajZkT2x4a2tiaDJUL1h3VFhLZEgwQT09

Meeting ID: 938 1855 2212
Passcode: 802722

Title: Data-Driven Document Unwarping

Abstract: Capturing document images is a common way to digitize and record physical documents due to the ubiquitousness of mobile cameras. To make text recognition easier, it is often desirable to digitally flatten a document image when the physical document sheet is folded or curved. However, unwarping a document from a single image in natural scenes is very challenging due to the complexity of document sheet deformation, document texture, and environmental conditions. Previous model-driven approaches struggle with inefficiency and limited generalizability. In this thesis, I investigate several data-driven approaches to tackle the document unwarping problem.

Data acquisition is the central challenge in data-driven methods. I first design an efficient data synthesis pipeline based on 2D image warping and train DocUNet, the pioneering data-driven document unwarping model, on the synthetic data. A benchmark dataset is also created to facilitate comprehensive evaluation and comparison. To improve the unwarping performance by training on more realistic data, I introduce the Doc3D dataset and DewarpNet. Supervised by 3D shape ground truth in Doc3D, DewarpNet is significantly better than DocUNet. DocUNet and DewarpNet depend on the synthetic data for the ground truth deformation annotation. To exploit the real-world images, I propose PaperEdge, a weakly supervised model trained with in-the-wild document images with easy-to-obtain boundary information. PaperEdge surpasses DewarpNet by utilizing both the synthetic data and weakly annotated real data in the Document In the Wild (DIW) dataset. Finally, I propose to incorporate the 3D physical constraints in training DewarpNet and PaperEdge. The constraints regulate the possible deformations on document papers. I also propose to augment the Doc3D and DIW dataset by introducing an online document segmentation model and better hardware.

Join us at the Center for Excellence in Learning and Teaching (CELT) for an interactive Zoom workshop on Generative AI designed for faculty and staff interested in enhancing teaching and assessment practices, increasing student engagement, and navigating the rapidly evolving landscape of AI tools. Participants will be introduced to common AI tools, explore potential instructional uses, and discuss key considerations such as academic integrity, transparency, and equity.

Register now: https://stonybrook.zoom.us/meeting/register/6js1eP64T1ys8tyU57EJ7Q#/registration

Place:  https://stonybrook.zoom.us/j/99167126152?pwd=TFpEYzM0aFhiOFJxSFJEb1JSS3YyQT09  

Time: 3 PM EST - Dec, 16th, 2020 

Abstract: 

Shadows provide useful cues to analyze visual scenes but also hamper many computer vision algorithms such as image segmentation, object detection, or tracking. For those reasons, shadow detection and shadow removal have been well-studied in computer vision.

Early work on shadow detection and removal focused on physical illumination models of shadows. These methods can express, identify, and remove shadows in a physically plausible manner. However, these models are often hard to optimize and are slow during inference due to their reliance on hand-designed image features. Recently, deep-learning approaches have achieved breakthroughs in performance for both shadow detection and removal. They learn to extract useful features through training while being extremely efficient during inference. However, these models are data-dependent, opaque, and ignore the physical aspects of shadows. Thus they often lack generalization and produce inconsistent results.

We propose incorporating physical illumination constraints of shadows into deep-learning models. These constraints force the networks to more closely follow the physics of shadows, enabling them to systematically and realistically modify shadows in images. For shadow detection, we present a novel Generative Adversarial Network (GAN) based model where the generator learns to generate images with realistic attenuated shadows that can be used to train a shadow detector. For shadow removal, we propose a method that uses deep-networks to estimate the unknown parameters of a shadow image formation model that removes shadows. The system outputs high-quality shadow-free images with little or no image artifacts and achieves state-of-the-art performance in shadow removal when trained on a fully-supervised setting. Moreover, the system is easy to train and constrain since the shadow removal mapping is strictly defined by the simplified illumination model with interpretable parameters. Thus, it can be trained even with a much weaker form of supervision signal. In particular, we show that we can use two sets of patches, shadow and shadow-free, to train our shadow decomposition framework via an adversarial system. These patches are cropped from the shadow images themselves.
Therefore, this is the first deep-learning method for shadow removal that can be trained without any shadow-free images, providing an alternative solution to the paired data dependency issue. The advantage of this training scheme is even more pronounced when tested on a novel domain such as video shadow removal where the method can be fine-tuned on a testing video with only the shadow masks generated by a pre-trained shadow detector and further improves shadow removal results.

Abstract: The current approach to materials design, driven by strategic experimentation and supported by physics-based simulation across relevant scales, has been the standard for decades. While the theoretical component in this workflow provides valuable understanding of material behavior, it often fails to deliver actionable guidance for implementation. Advances in artificial intelligence and machine learning (AI/ML), together with high-performance computing (HPC), now offer a viable pathway to close this gap and accelerate both discovery and process optimization. This presentation will outline practical approaches for integrating AI/ML with HPC-enabled, high-throughput computation to explore high-dimensional search spaces. Examples will include the development of engineering alloys for extreme environments, the use of neural networks to rapidly improve computational thermodynamic models, and vapor processing optimization for the manufacturing of ultra-high-temperature ceramics. I will highlight how scientific insight and domain expertise remain essential for translating surrogate model predictions into impactful outcomes. Finally, I will conclude with current challenges and future opportunities for AI/HPC-driven materials research.

Speaker: Dongwon Shin
This seminar will be held in person and online

Join Zoom Meeting: https://stonybrook.zoom.us/j/93730374357?pwd=YDLJ7ELqOQnTZEQhlN8Pa4TuhaiFK8.1