Abstract: Pretraining vision encoders with self-supervision (SSL) leads to stronger representations that excel across diverse downstream tasks. One of the key factors enabling self-supervision is extracting multiple views of the same scene to formulate either: 1) View-invariant pretraining (DINO, SimCLR, iBOT), where the objective is predicting the same representation for different views of the scene; or 2) Cross-view pretraining (cross-view Masked Autoencoders), where the objective is predicting missing parts of one view using other views. For extracting multiple views, view-invariant methods rely on a combination of handcrafted augmentations (random cropping, color jittering, gaussian blur, etc.) of the same image, whereas cross-view pretraining methods rely on image cropping or video frames. In this work, we present methods to effectively incorporate synthetic views from diffusion models into SSL training.
For view-invariant pretraining, we introduce Gen-SIS, a method that leverages the ability of diffusion models to generate interpolated images through interpolation in conditioning space. We introduce a disentanglement pretext task: disentangling two source images from an interpolated synthetic image. This disentanglement task, in addition to vanilla single-source generative augmentation for view extraction, improves visual pretraining of various view-invariant methods (DINO, SimCLR, iBOT).
For cross-view pretraining, we introduce CDG-MAE, a novel cross-view masked autoencoder (MAE) based method that uses diverse synthetic views generated from static images via an image-conditioned diffusion model to learn dense correspondences. We present a quantitative method to evaluate the local and global consistency of the generated views to choose the right diffusion model for cross-view pretraining. These generated views exhibit substantial changes in pose and perspective, providing a rich training signal that overcomes the limitations of video (expensive) and crop-based (less variation) methods. CDG-MAE substantially narrows the gap to video-based MAE methods on video label propagation tasks while maintaining the data advantages of image-only MAEs.

Speaker: Varun Belagali

Location: NCS 120
Zoom: https://stonybrook.zoom.us/j/93647452432?pwd=hZaX7LXCAD8KPHWYE1Afw2sDI3owpv.1
Hidden Biases. Ethical Issues in NLP, and What to Do about Them presented by Dirk Hovy of Bocconi University

ABSTRACT: Through language, we fundamentally express who we are as humans. This property makes text a fantastic resource for research into the complexity of the human mind, from social sciences to humanities. However, it is exactly that property that also creates some ethical problems. Texts reflect the authors' biases, which get magnified by statistical models. This has unintended consequences for our analysis: If our data is not reflective of the population as a whole, if we do not pay attention to the biases contained, we can easily draw the wrong conclusions, and create disadvantages for our users.

In this talk, I will discuss several types of biases that affect NLP models, their sources, and potential counter measures: (1) Bias stemming from data, i.e., selection bias (if our texts do not adequately reflect the population we want to study), label bias (if the labels we use are skewed) and semantic bias (the latent stereotypes encoded in embeddings); (2) Biases deriving from the models themselves, i.e., their tendency to amplify any imbalances that are present in the data; (3) Design bias, i.e., the biases arising from our (the researchers) decisions which topics to analyze, which data sets to use, and what to do with them. For each bias, I will provide examples and discuss the possible ramifications for a wide range of applications, and various ways to address and counteract these biases, ranging from simple labeling considerations to new types of models.

BIO: Dirk Hovey is an associate professor of Computer Science in the department of marketing at Bocconi University. He received his PhD from the University of Southern California in Los Angeles, where he worked as a research assistant at the Information Sciences Institute. 

He works in Natural Language Processing (NLP), a subfield of artificial intelligence. His research focuses on computational social science. His interests include integrating sociolinguistic knowledge into NLP models, using large-scale statistics to model the interaction between people's socio-demographic profile and their language use, and ethics for data science and algorithmic fairness.

Visual Analytics and Machine Learning for Biomedical Imaging Diagnosis

 

Arie Kaufman

 

We present an integrated approach using visual analytics and machine learning (ML) to diagnose abnormalities in 3D radiological imaging and biological microscopes. The primary example will involve 3D virtual pancreatography (VP), a novel visualization-ML procedure and application for non-invasive diagnosis and classification of pancreatic lesions, the precursors of pancreatic cancer. Currently, non-invasive screening of patients is performed through visual inspection of 2D axis-aligned CT images, though the relevant features are often not clearly visible nor automatically detected. VP is an end-to-end visual diagnosis system that includes an ML-based automatic segmentation of the pancreatic gland and the lesions, a semi-automatic approach to extract the primary pancreatic duct, an ML-based automatic classification of lesions into four prominent types, and specialized 3D and 2D exploratory visualizations of the pancreas, lesions and surrounding anatomy. We combine volume rendering with pancreas- and lesion-centric visualizations and measurements for effective diagnosis. We designed VP through close collaboration and feedback from expert radiologists, and evaluated it on multiple real-world CT datasets with various pancreatic lesions and case studies examined by the expert radiologists. Other applications include virtual colonoscopy, COVID-19, pathology, brain neurites, etc.


Biography: Arie Kaufman is Distinguished Professor and formerChair of the Department of Computer Science at Stony Brook University, where he is also Director of the Center for Visual Computing (CVC), and Chief Scientist at the Center of Excellence in Wireless and Information Technology (CEWIT). 

He received his PhD in Computer Science at Ben-Gurion University of the Negev in 1977.   He is known for his work in visualization, graphics, virtual reality, user interfaces, multimedia, and their applications, especially in bio-medicine. He is especially well known for his work on the 3-dimensional virtual colonoscopy, a revolutionary low-risk technique for colon cancer screening, and for pioneering the use of Graphics Processing Units (GPUs) and GPU-clusters. In 2012, he presided over the development and opening of the Reality Deck, the largest virtual reality display in the world, at Stony Brook University.

Kaufman was the founding Editor in Chief of IEEE Transactions on Visualization and Computer Graphics (TVCG), co-founded the IEEE Visualization Conference and Volume Graphics series, and is currently the director of IEEE Computer Society Technical Committee on Visualization and Graphics. He is an IEEE Fellow, ACM Fellow, winner of many awards, including the IEEE Visualization Career Award, and member of the European Academy of Sciences.



Steven Skiena is inviting you to a scheduled Zoom meeting.

Topic: AI Seminar: Arie Kaufman
Time: Apr 21, 2021 10:00 AM Eastern Time (US and Canada)

Join Zoom Meeting
https://stonybrook.zoom.us/j/96017498640?pwd=SE0rdHB6ZVlCM2ZpY2RnRUxyVnR3Zz09

Submit an abstract celebrating research, new discoveries and achievements in medicine and science!

We encourage faculty, nurse practitioners, post-doctoral fellows, fellows, residents, medical students, graduate students and undergraduate students to submit an abstract. Original research, case reports and case series are welcome.

Abstract submission deadline: FEBRUARY 7, 2025

For more details, visit here.





https://stonybrook.zoom.us/j/91775729097pwd=Qlc5Nks0NmlyKzJwMjR0S0hrdVZ3QT09

Meeting ID: 917 7572 9097
Passcode: 555459


Abstract: As the saying goes, there are many ways to skin a cat.
While we don't want to go around skinning cats, the world of
optimization is rich with different problems, problem formulations,
and methods and approaches, each with different guarantees and
computational benefits. In this talk we will take a tour down the
problem of structured sparsity in sensing to see how one simple
problem can inspire a wide range of analysis and tools. First, I will
present the optimality conditions for a generalized structured sparse
problem, which can be geometrically visualized as alignment of vectors
and matrices. Then I will introduce three approximation methods for
the problem of phase retrieval, which are a twist on stochastic
gradient and coordinate descent methods. These methods leverage
fundamental numerical linear algebra concepts to give fast approximate
solutions to large-scale problems, which then after postprocessing can
produce more reliable sensing results.

Bio: Yifan Sun received her PhD in Electrical Engineering from the
University of California Los Angeles in 2015, with research focusing
on convex optimization and semidefinite programming. She was then
Technicolor Research and Innovation, focusing on machine learning and
data science applications. More recently, she completed two postdocs,
at the University of British Columbia in Vancouver, Canada and
L'Institut National de Recherche en Informatique et Automatique
(INRIA) in Paris, France.
Optimization and Machine Learning - presented by Yifan Sun

Abstract: Optimization is a growing topic of interest in the machine learning community. It starts out as an option to check in Tensorflow (SGD? Adam? Adagrad?), but as we get more into the how and why of these options, we uncover many fundamental principles relating to operations research, control theory, and dynamical systems, dating back as far as the Cold World era. 

In this talk I will give a broad overview of some of the important optimization themes in machine learning. I will try to give connections between tools we are used to seeing in popular packages 
and fundamental optimization concepts like duality, convexity, contractive operators, etc. While we cannot hope to completely cover this diverse research area, I hope to provide a glimpse of this exciting research area that is permeating more and more into the machine learning world. 

Bio: Yifan Sun received her PhD in Electrical Engineering from the University of California Los Angeles in 2015, with research focusing on convex optimization and semidefinite programming. She was then Technicolor Research and Innovation, focusing on machine learning and 
data science applications. More recently, she completed two postdocs focusing on optimization, at the University of British Columbia in Vancouver, Canada and INRIA, in Paris, France.
CSE 600 Seminar Series | Fall 2025

Abstract: Imagine machines that can see the invisible: drones locating wildfire survivors, cameras predicting building failures, and smartphones detecting skin tumors. These applications lie beyond today's vision systems, which focus only on human-visible information. In this talk, I argue that a wealth of scene information is hidden in light properties invisible to the human eye, such as the travel time of photons and polarization of light waves. I will present how co- designing camera hardware, graphics models, and learning algorithms unlocks these invisible properties to create superhuman vision systems. I will present three superhuman vision capabilities: seeing around blind corners, turning objects into cameras, and extracting internal stress fields. By analyzing faint light reflections on diffuse walls and shiny objects, we create virtual cameras that reveal scenes hidden from the line of sight - enabling autonomous systems to navigate safely. Using the polarization of light, we recover mechanical stress fields hidden inside objects - opening new possibilities for non-destructive material characterization. These capabilities point toward a future where machines can see the invisible: around us, beneath our bodies, and beyond our scientific understanding.

Bio:
Akshat Dave is an Assistant Professor in the Department of Computer Science at Stony Brook
University, USA. His research lies at the intersection of applied optics, computer vision, and
machine learning. His work has been recognized by Rice University's Best Thesis Award, Optica Best Paper Prize, SIGGRAPH Asia Doctoral Consortium, and fellowships by Qualcomm, Texas Instruments, and INK Global Foundation. Prior to Stony Brook, he was a Postdoctoral Associate at MIT Media Lab. He holds a Ph.D. from Rice University and a Masters and a Bachelors from Indian Institute of Technology Madras.
The Empirical Methods in Natural Language Processing (EMNLP) conference is a premier international academic conference in the field of artificial intelligence and natural language processing (NLP). Organized annually by the Association for Computational Linguistics (ACL) special interest group on linguistic data (SIGDAT), it focuses on research that uses empirical methods to solve language processing problems.

For more information, and registration, visit the official website.