Visual Analytics and Machine Learning for Biomedical Imaging Diagnosis

 

Arie Kaufman

 

We present an integrated approach using visual analytics and machine learning (ML) to diagnose abnormalities in 3D radiological imaging and biological microscopes. The primary example will involve 3D virtual pancreatography (VP), a novel visualization-ML procedure and application for non-invasive diagnosis and classification of pancreatic lesions, the precursors of pancreatic cancer. Currently, non-invasive screening of patients is performed through visual inspection of 2D axis-aligned CT images, though the relevant features are often not clearly visible nor automatically detected. VP is an end-to-end visual diagnosis system that includes an ML-based automatic segmentation of the pancreatic gland and the lesions, a semi-automatic approach to extract the primary pancreatic duct, an ML-based automatic classification of lesions into four prominent types, and specialized 3D and 2D exploratory visualizations of the pancreas, lesions and surrounding anatomy. We combine volume rendering with pancreas- and lesion-centric visualizations and measurements for effective diagnosis. We designed VP through close collaboration and feedback from expert radiologists, and evaluated it on multiple real-world CT datasets with various pancreatic lesions and case studies examined by the expert radiologists. Other applications include virtual colonoscopy, COVID-19, pathology, brain neurites, etc.


Biography: Arie Kaufman is Distinguished Professor and formerChair of the Department of Computer Science at Stony Brook University, where he is also Director of the Center for Visual Computing (CVC), and Chief Scientist at the Center of Excellence in Wireless and Information Technology (CEWIT). 

He received his PhD in Computer Science at Ben-Gurion University of the Negev in 1977.   He is known for his work in visualization, graphics, virtual reality, user interfaces, multimedia, and their applications, especially in bio-medicine. He is especially well known for his work on the 3-dimensional virtual colonoscopy, a revolutionary low-risk technique for colon cancer screening, and for pioneering the use of Graphics Processing Units (GPUs) and GPU-clusters. In 2012, he presided over the development and opening of the Reality Deck, the largest virtual reality display in the world, at Stony Brook University.

Kaufman was the founding Editor in Chief of IEEE Transactions on Visualization and Computer Graphics (TVCG), co-founded the IEEE Visualization Conference and Volume Graphics series, and is currently the director of IEEE Computer Society Technical Committee on Visualization and Graphics. He is an IEEE Fellow, ACM Fellow, winner of many awards, including the IEEE Visualization Career Award, and member of the European Academy of Sciences.



Steven Skiena is inviting you to a scheduled Zoom meeting.

Topic: AI Seminar: Arie Kaufman
Time: Apr 21, 2021 10:00 AM Eastern Time (US and Canada)

Join Zoom Meeting
https://stonybrook.zoom.us/j/96017498640?pwd=SE0rdHB6ZVlCM2ZpY2RnRUxyVnR3Zz09

You are cordially invited to attend the biweekly Brookhaven AI Mixer (BAM). BAM includes one short talk on AI research happening at BNL, followed by an open mixer over coffee and snacks for everyone to network and discuss all things AI. The first half hour will consist of presentations that will be available via ZOOM, and the second half hour will be for in person only networking.

Join us every other Tuesday at noon in CDSD's Training Room (building 725, 2nd floor) to learn about interesting AI methods and applications, engage with potential collaborators, prepare for pending FASST funding calls, and build a community of AI for Science at BNL.

Learning Generalizable Program and Architecture Representations for Performance Modeling

Abstract: Performance modeling is an essential tool in many areas of computer science and engineering. However, existing performance modeling approaches have limitations, such as high computational cost, narrow flexibility, or restricted accuracy/generality. To address these limitations, this talk introduces PerfVec, a novel deep learning-based performance modeling framework that learns high-dimensional and independent/orthogonal program and microarchitecture representations. Once learned, a program representation can be used to predict its performance on any microarchitecture, and likewise, a microarchitecture representation can be applied in the performance prediction of any program. Additionally, PerfVec yields a foundation model that captures the performance essence of instructions, which can be directly used by developers in numerous performance modeling-related tasks without incurring its training cost. The evaluation demonstrates that PerfVec is more general and efficient than previous approaches. This talk will also introduce how PerfVec's design principles can benefit broader research areas.

Biography: Lingda Li is a computer scientist at Brookhaven National Laboratory. He is generally interested in computer architecture and programming model research, with focus on simulation/modeling, memory systems, and machine learning. Before joining BNL, he worked at the Department of Computer Science of Rutgers University as a postdoc to carry out GPGPU research. He obtained a PhD in computer architecture from the Microprocessor Research and Development Center at Peking University.

Location: CDS, Bldg. 725, Training Room

Join ZoomGov Meeting: https://bnl.zoomgov.com/j/1605837856?pwd=kYqJs4bVBt4E0cMCWR6GXH3wxzOoiw.1

Meeting ID: 160 583 7856
Passcode: 161580

Do Natural Language Understanding Systems Learn to Understand or to
Find Shortcuts? (Naoya Inoue, http://naoya-i.github.io/)

ABSTRACT: Recent studies have suggested that natural language understanding (NLU) systems learn to exploit superficial, task-unrelated cues (a.k.a. annotation artifacts) in current datasets. This prevents the community from reliably measuring the progress of NLU systems. In this talk, I will discuss two latest studies from our research team: (i) analysis of annotation artifacts in commonsense causal reasoning and (ii) creation of benchmark for evaluating NLU systems' internal reasoning.
---------------------------------------------------------------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------------------------------------------------------------
Learning graph-structured sparse models (Baojian Zhou, https://baojianzhou.github.io/) 

ABSTRACT: Learning graph-structured sparse models has recently received significant attention thanks to their broad applicability to many important real-world problems. However, such models, of more effective and stronger interpretability compared with their counterparts, are difficult to learn due to optimization challenges. In this talk, we will discuss how to learn graph-structured sparse models under stochastic and online learning settings. Some interesting related problems will also be discussed.
Abstract: Recent advances in Spatial Transcriptomics (ST) pair histology images with spatially resolved gene expression profiles, enabling predictions of gene expression across different tissue locations based on image patches. This opens up new possibilities for enhancing whole slide image (WSI) prediction tasks with localized gene expression. However, existing methods do not fully leverage the interactions between different tissue locations, which are crucial for accurate joint prediction. To address this, we introduce MERGE (Multi-faceted hiErarchical gRaph for Gene Expressions), which combines a multi-faceted hierarchical graph construction strategy with graph neural networks (GNN) to improve gene expression predictions from WSIs. By clustering tissue image patches based on both spatial and morphological features, and incorporating intra- and inter-cluster edges, our approach fosters interactions between distant tissue locations during GNN learning. As an additional contribution, we evaluate different data smoothing techniques that are necessary to mitigate artifacts in ST data, often caused by technical imperfections. We advocate for adopting gene-aware smoothing methods that are more biologically justified. Experimental results on gene expression prediction show that our GNN method outperforms state-of-the-art techniques on multiple metrics such as mean squared error (MSE), mean absolute error (MAE), and pearson correlation coefficient (PCC). Qualitative analysis establishes the effectiveness of MERGE in capturing cancer marker genes, thus consolidating its utility in diagnostics. As an extension of this work, we use MERGE in a setting with an uncertainty calibration branch to perform robust gene expression smoothing. We show that using patch-wise uncertainty from an uncertainty calibration model and the gene expression predictions from MERGE to enrich the ground truth gene expression matrix, results in better alignment with pathologist annotations, thus establishing that the smoothing is biologically informed.

Speaker: Aniruddha Ganguly

Location: Virtual Zoom Meeting


https://stonybrook.zoom.us/j/5474847973?pwd=Sng0Q2h1c1d3cm9sbFBmYUczMHZNdz09
Meeting ID: 547 484 7973
Passcode: 206739

The Provost's Spotlight Talks feature eminent visitors to the university as well as Stony Brook faculty members who have recently been recognized for outstanding contributions in their field.

Transmedia artist Stephanie Dinkins, Kusama endowed chair in art in the College of Arts and Sciences at Stony Brook University, brings her expertise in AI to the next Spotlight Talk with The Stories We Encode: AI, Love and the Future of Algorithmic Care on Tuesday, October 22, at 3:30 pm in the Charles B. Wang Center Theatre.

Working at the intersection of emerging technologies and social collaboration, Dinkins was named a 2023 TIME 100 Most Influential People in AI. She was recognized for her work with Not the Only One, an ongoing project in which she trained an AI on three generations of Black women to give it cultural roots, a deep history, and a perspective that existing systems do not offer.

The event is free and open to the public, and the discussion will be followed by a reception in the Wang Theatre lobby, hosted by the College of Arts and Sciences for new and promoted faculty.


About the Talk

AI's impact on society necessitates addressing longstanding human rights issues and prejudices. To ensure AI benefits humanity, we must confront institutional biases, rethink our relationship with other beings and emerging technologies, and reconcile ideals with actual power structures. This involves recognizing systemic inequalities, redefining human identity, and equitably distributing resources. AI, if developed and used ethically, offers an opportunity to reimagine a more equitable world for all inhabitants.

Join Stony Brook University's Center for Excellence in Learning and Teaching (CELT) for a bootcamp on how to use AI to enhance your teaching and courses. This event will demonstrate how ChatGPT, Microsoft Copilot, and other generative AI platforms can support you in crafting learning objectives, writing exam questions, composing rubrics, and designing course content such as lesson plans, in-class activities, instructional videos, and more.

Register here.