All are welcome to attend BMI grand rounds talk by Dr. Le Lu on 04/14. 

Le Lu, Ph.D 
Executive Director, PAII Inc 
Johns Hopkins University
IEEE Fellow, MICCAI Board Member


Time: Wednesday, April 14, 2021 3:00 pm - 4:00 pm 

Zoom Meeting 
https://stonybrook.zoom.us/j/95617197636?pwd=KytzZ2pVRG9SZGpKZUtpNXJISjNjZz09 
Meeting ID: 956 1719 7636 Passcode: 924293

Title: 
In Search of Effective and Reproducible Clinical Imaging Biomarkers for Population Health and Oncology Applications of Screening, Diagnosis and Prognosis

Bio: 
Le Lu received a PhD in 2007 from Johns Hopkins University. During his first six years at Siemens, he made significant contributions to the company's CT colonography and Lung CAD product lines. From 2013 to 2017, Dr. Lu served as a staff scientist in the Radiology and Imaging Sciences department of the National Institutes of Health Clinical Center. He then went on to found Nvidia's medical image analysis group and he held the position of senior research manager until June 2018. Since then, he has been the Executive Director at PAII Inc., Bethesda Research lab, Maryland, USA which has become one of the leading industrial research labs in medical imaging. He was the main technical leader for two of the most-impactful public radiology image dataset releases (NIH ChestXray14, NIH DeepLesion 2018). He won NIH Clinical Center Director Award in 2017, NIH Mentor of the year award in 2015, and won numerous best paper awards in MICCAI and RSNA from 2016 to 2020 (over 10000 citations). In 2021, He was elected into IEEE Fellow class cited for his contribution to machine learning for cancer detection and diagnosis, and MICCAI society board member (MICCAI-Industry Workgroup Chair). He is currently an Associate Editor for IEEE Trans. Pattern Analysis and Machine Intelligence and IEEE Signal Processing Letters. He has served as an Area Chair for recent MICCAI, AAAI, CVPR, WACV, ICIP and ICHI conferences for 14 times.

Abstract: 
This talk will first give an overall on the work of employing deep learning to permit novel clinical workflows in two population health tasks, namely using conventional ultrasound for liver steatosis screening and quantitative reporting; osteoporosis screening via conventional X-ray imaging and AI readers. These two tasks were generally considered as infeasible tasks for human readers, but as proved by our scientific and clinical studies and peer-reviewed publications, they are suitable for AI readers. AI can be a supplementary and useful tool to assist physicians for cheaper and more convenient/precision patient management. Next, the main part of this talk describes a roadmap on three key problems in pancreatic cancer imaging solution: early screening, precision differential diagnosis, and deep prognosis on patient survival prediction. (1) Based on a new self- learning framework, we train the pancreatic ductal adenocarcinoma (PDAC) segmentation model using a larger quantity of patients (≈1,000, four institutions), with a mix of annotated/unannotated venous or multi-phase CT images. Pseudo annotations are generated by combining two teacher models with different PDAC segmentation specialties on unannotated images, and can be further refined by a teaching assistant model that identifies associated vessels around the pancreas. Our approach makes it technically feasible for robust large-scale PDAC screening from multi-institutional multi-phase partially-annotated CT scans. (2) We propose a holistic segmentation-mesh classification network (SMCN) to provide patient-level diagnosis, by fully utilizing the geometry and location information. SMCN learns the pancreas and mass segmentation task and builds an anatomical correspondence-aware organ mesh model by progressively deforming a pancreas prototype on the raw segmentation mask. Our results are comparable to a multimodality clinical test that combines clinical, imaging, and molecular testing for clinical management of patients with cysts. (3) Accurate preoperative prognosis of resectable PDACs for personalized treatment is highly desired in clinical practice. We present a novel deep neural network for the survival prediction of resectable PDAC patients, 3D Contrast-Enhanced Convolutional Long Short-Term Memory network (CE- ConvLSTM), to derive the tumor attenuation signatures from CE-CT imaging studies. Our framework can significantly improve the prediction performances upon existing state-of-the-art survival analysis methods. This deep tumor signature has evidently added values (as a predictive biomarker) to be combined with the existing clinical staging system.

More information can be found at:
https://bmi.stonybrookmedicine.edu/sites/default/files/Lu_le_04_14.pdf
Hieu Le presents Incorporating Physical Illumination Constraints into Deep Learning Shadow Detection and Removal (PhD Proposal)

Shadows provide useful cues to analyze the scene but also hamper many computer vision algorithms such as image segmentation, object detection or tracking. For those reasons, shadow detection and shadow removal have been well studied topics in computer vision. Early approaches for shadow detection and removal focus on physical illumination models of shadows. These methods can express, identify, and remove shadows in a physically plausible manner. However, these models are often hard to optimize and slow in inference due to reliance on hand-designed image features. On the other hand, recent deep-learning approaches have achieved breakthroughs in performances for both shadow detection and removal. They learn to extract useful features automatically through training while being extremely efficient in computation. However, these models are data-dependent, opaque and ignore the physical aspects of shadows.

We propose to incorporate physical illumination constraints into deep-learning frameworks. Thus the mapping learned by the deep-network closely follows the physics of shadows, enabling the network to systematically and realistically modify shadows in images. For shadow detection, we present a novel GAN framework in which the generator can generate realistic images with attenuated shadows that can be used to train a shadow detector. For shadow removal, we propose a method that uses deep-networks to estimate the unknown parameters for a shadow image formation model that removes shadows. The system outputs shadow-free images in high-quality with no image artifacts and achieves state-of-the-art shadow removal performance. Lastly, we propose a system trained without the need for any shadow-free images in which physical constraints play pivotal roles that enable training the networks.

For Zoom information, please email events@cs.stonybrook.edu.

Abstract: Modern large language models (LLMs) increasingly operate not on single inputs, but over sets of information--collections of generated statements, retrieved documents, or intermediate reasoning steps. However, existing frameworks largely treat these elements independently, overlooking the fact that correctness and reliability often emerge at the set level, where interactions such as contradictions, redundancy, or joint relevance become critical. In this talk, I argue that reasoning over sets is a fundamental challenge for building reliable and controllable LLM systems, and present a unified perspective centered on set-level verification, retrieval, and editing.

Bio: Jay-Yoon Lee is an associate professor in the Graduate School of Data Science at
Seoul National University. The goal of his research is injecting knowledge/constraints into neural models, primarily for natural language processing (NLP) tasks. He is broadly interested in structured prediction, multi-task learning, logical reasoning, and better representation learning for the aforementioned topics. Currently, he is exploring these topics at the Graduate School of Data Science (GSDS), Seoul National University (SNU), with SKI-ML lab members. Prior to joining SNU he was a postdoc associate at IESL with Professor Andrew McCallum and during his Ph.D. with his advisor Jaime Carbonell.

Location: NCS 120
Educational objectives:
  1. Describe the data modalities available in All of Us (EHR, genomics, wearables) and explainhow OMOP CDM standardization enables reproducible cohort definitions across CDRversions.
  2. Evaluate the role of AI-assisted coding tools in accelerating the research-to-manuscriptpipeline while maintaining investigator oversight and scientific rigor.
  3. Apply a secure two-environment workflow to distinguish between exportable code artifactsand environment-bound data outputs when working within controlled-access researchplatforms.


Speaker: Hants Williams, Ph.D., R.N., Applied Health Informatics, Stony Brook University

Location: MART Building, Room: 7M-0602 (7th Floor)

Zoom: https://stonybrook.zoom.us/j/95617197636?pwd=KytzZ2pVRG9SZGpKZUtpNXJISjNjZz09
Meeting ID: 95617197636
Passcode: 924293
Join your friends at DoIT for a workshop on Zoom's AI Companion

AI Companion is a meeting summary tool that can capture and summarize what is said in a Zoom meeting transcript, eliminating the need for a notetaker. In addition, meeting participants can use AI Companion to ask questions to a chat bot to get clarity on information within a meeting, and they can also use it to create stylistic virtual backgrounds.

Register here for the online session
Speaker Petar Djuric Refreshments will be provided Deep Gaussian processes: Theory and applications Petar M. Djurić Department of Electrical and Computer Engineering Stony Brook University Abstract: Gaussian processes are an infinite-dimensional generalization of multivariate normal distributions. They provide a principled approach to learning with kernel machines and they have found wide applications in many fields. More recently, with the advance of deep learning, the concept of deep Gaussian processes has emerged. Deep Gaussian processes can be viewed as multilayer hierarchical organizations of Gaussian processes that are equivalent to infinitely wide multiple layer neural networks. Deep Gaussian processes have improved capacity for prediction and classification over standard Gaussian processes, while models based on them continue to allow for full Bayesian treatment and for applications when the amount of available data is limited. The theory of recent progress in deep Gaussian processes will be presented and some applications will be provided. Biosketch: Petar M. Djurić received the B.S. and M.S. degrees in electrical engineering from the University of Belgrade, Belgrade, Yugoslavia, respectively, and the Ph.D. degree in electrical engineering from the University of Rhode Island, Kingston, RI, USA. He is a SUNY Distinguished Professor and currently, he is a Chair of the Department of Electrical and Computer Engineering, Stony Brook University, Stony Brook, NY, USA. Djurić was a recipient of the IEEE Signal Processing Magazine Best Paper Award in 2007 and the EURASIP Technical Achievement Award in 2012. From 2008 to 2009, he was a Distinguished Lecturer of the IEEE Signal Processing Society. He was the Editor-in-Chief of the IEEE Transactions on Signal and Information Processing over Networks (2015-2018). Djurić is a Fellow of IEEE and EURASIP

Talk Title: Knowledge-enhanced LLMs and Human-AI Collaboration Frameworks for Creativity Support


Abstract:

Large language models (LLMs) constitute a paradigm shift in Natural Language Processing and Artificial Intelligence. To build AI systems that are human-centered, I propose we need knowledge-aware models and human-AI collaboration frameworks to help them solve tasks ultimately aligning these models better with human values. In this talk, I will discuss my research agenda for human-centered AI with a case study on creativity that focuses on how to augment LMs with external knowledge, build effective human-AI collaboration frameworks as well as theoretically grounded robust evaluation protocols for measuring capabilities of NLG systems. I will begin by describing knowledge-enhanced methods for creative text generation such as metaphors. Next, I will describe how content creators can collaborate and benefit from the creative capabilities of text-to-image-based AI models. Finally, I will focus on the design and development of theoretically grounded evaluation protocols to benchmark the creative capabilities of Large Language Models in both producing as well as assessing creative text. I will end this talk by highlighting the current limitations of existing models and future directions toward building better models that will enable efficient and trustworthy human-AI collaboration systems.


Bio:

Tuhin Chakrabarty is a final-year Ph.D. candidate in the Natural Language Processing group within the Computer Science department at Columbia University. His research is supported by the Columbia Center of Artificial Intelligence & Technology (CAIT) & an Amazon Science Ph.D. Fellowship. He was also a Computational Journalism fellow at NYTimes R&D and an intern at the Allen Institute of Artificial Intelligence, Salesforce Research, and Deepmind. His research interests are broadly in Natural Language Processing, Computer Vision, and Human-Computer Interaction with a special focus on Human-Centered Methods for Understanding, Generation, and Evaluation of Creativity. His work has been recognized at top natural language processing and human-computer interaction conferences and journals such as ACL, NAACL, EMNLP, TACL, and CHI. He has been involved in organizing several workshops and tutorials at NLP conferences such Figurative Language Processing workshop at EMNLP 2022, NAACL 2024, and the tutorial on Creative Text Generation at EMNLP 2023. His work on AI and creativity has been mentioned in mainstream news media such as The Hollywood Reporter and more recently The Washington Post.

Join Zoom Meeting https://stonybrook.zoom.us/j/97103601583?pwd=TnpGMXdpeEd1N0hZcXppS1BLNFJhZz09 (ID: 97103601583, passcode: 004031) Join by phone (US) +1 646-931-3860 (passcode: 004031) Joining instructions: https://www.google.com/url?q=https://applications.zoom.us/addon/invitation/detail?meetingUuid%3DILacj94mRvSXgTYt0Cqs1w%253D%253D%26signature%3D9f2f1e7e603bbcb9034724d084eea8846c19a38b7436180170dfc3f1d718b425%26v%3D1&sa=D&source=calendar&usg=AOvVaw3MsNgLSPMRl8L5i6BosYrB Meeting host: H.Andrew.Schwartz@stonybrook.edu

Join Zoom Meeting:
https://stonybrook.zoom.us/j/97103601583?pwd=TnpGMXdpeEd1N0hZcXppS1BLNFJhZz09
Making sense of Twitter @ Bloomberg presented by Daniel Preotiuc-Pietro

ABSTRACT: The Bloomberg Terminal has provided ways for investors and journalists to sift through and understand the immense volume of tweets and discover financially-relevant content ever since the SEC approved the use of Twitter for company disclosures back in 2013.

In the first part of the talk, I will showcase how tweets impact financial markets and how Bloomberg is using Natural Language Processing methods to identify financially relevant tweets that move the markets. Our processing pipeline feeds directly to clients, journalists in the newsroom and powers several news analytic products offered by the company including trending companies and consumer sentiment for publicly traded equities.

However, understanding user pragmatic intent in individual tweets would allow us to gain deeper insights and enable new applications. I will present several recent research studies focused on understanding intent including identifying complaints and the roles with which vulgarity is used in social media and how these can help improve applications such as sentiment analysis and hate speech detection.

BIO: Daniel Preotiuc-Pietro is a Senior Research Engineer and Team Lead at Bloomberg LP, where he works on analyzing and building models for real-world large scale social media and news mining and information extraction. His research interests are focused on understanding the social and temporal aspects of text, especially from social media, with applications in domains such as Social Psychology, Law, Political Science and Journalism. Several of his research studies were featured in popular press including the Washington Post, BBC, New Scientist, Scientific American or FiveThirtyEight. He is a co-organizer of the Natural Legal Language Processing workshop series. Prior to joining Bloomberg LP, Daniel was a postdoctoral researcher at the University of Pennsylvania with the interdisciplinary World Well Being Project and obtained his PhD in Natural Language Processing and Machine Learning at the University of Sheffield, UK.
Abstract: Facial emotion understanding aims to recognize, represent, and interpret human affect from facial behavior, and it is important for affective computing, human-computer interaction, digital humans, and mental health assessment. Existing work has represented facial emotion through discrete emotion categories, continuous affective dimensions such as valence and arousal, facial landmarks or geometry, and more recently semantic or language-based emotion descriptors. With the development of deep learning, supervised facial expression recognition has achieved strong performance on benchmark datasets, while self-supervised learning, multimodal large language models, and controllable facial generation have introduced new ways to learn emotion-related facial representations from images, videos, text, audio, and speech. However, many current models still rely heavily on manually annotated emotion labels, third-party perception judgments, or multimodal contextual cues, making it difficult to determine how much emotional information is captured directly from facial behavior itself, especially in naturalistic and clinically meaningful settings. Because these limitations make it challenging to evaluate whether facial representations capture emotionally meaningful behavior in real-world interactions, we propose to study facial emotion understanding through the relationship between facial behavior and language-derived emotional expression in psychiatric interview videos. Using a large dataset of mental health interviews, we extract multiple types of facial representations, including Action Unit features from FaceReader and OpenFace, non-AU facial behavior features from OpenFace, and 3D facial representations from EMOCA and SMIRK. We train segment-aligned transformer regressors to predict language-derived emotional targets, including valence, arousal, and RoBERTa-derived semantic-affective features from transcript segments. The results show that all facial representations achieve meaningful predictive performance across MSE and Pearson correlation metrics in both within-participant and between-participant evaluation settings. This indicates that facial behavior encodes information related to linguistic emotion at multiple levels: moment-to-moment emotional variation within individuals and broader affective differences across individuals. These findings suggest that structured facial representations can support vision-based emotion understanding in naturalistic mental health interviews and motivate future work on self-supervised, personalized, and controllable facial emotion models.

Speaker: Shao-Yu Chang

Zoom: https://stonybrook.zoom.us/j/3679036240?omn=98419305450
West Campus - SAC- Student Activities Center - Ballrooms A & B 100 Nicolls Road Stony Brook NY 11794 Job Fair.jpg The Career Center invites Alumni Employers and Job Seekers to the IT/Computer Science Job and Internship Fair this spring. Job Seekers: A job fair is an opportunity for you to present yourself professionally in person to a potential employer, while showcasing your communication skills. Get more information Alumni Employers: Held in both the fall and spring semesters, this event is ideal for employers looking to fill internship, co-op, part-time and full-time opportunities in the field of information technology (i.e. Software Engineering, Network Administration, Web Development, etc.). Register here to recruit top SBU talent.