Topological Constraints for Structure-aware Dense Prediction
Event Description
Abstract: Humans perceive the world around them by recognizing global patterns and structures such as object parts, branches, their spatial arrangement, and so on. Most deep learning models, however, take a fundamentally local approach. They process images pixel-by-pixel rather than focusing on structures as a whole. While these models indeed perform well on many tasks, the local (pixel-level) versus global (structure-level) disconnect makes them harder to interpret and control.
Topology, in a general sense, is a mathematical language for describing structure. It delineates how different parts of an image relate to one another, capturing both individual structures and their overall layout. Preserving topology enforces structural correctness and, by extension, semantic validity.
In this thesis, we investigate how topological constraints can be used to bridge the gap between local and global understanding. We use topology to inform the design of deep learning models that are explicitly structure-aware. Our thesis focuses on dense prediction tasks, which include image segmentation, uncertainty estimation, and generative modeling. First, we introduce a topological interaction module for semantic segmentation that encodes containment and exclusion constraints directly into the learning process. This preserves anatomical hierarchies and improves multi-class consistency. Next, since segmentation models can never be truly perfect, we address the need for reliable uncertainty estimation to identify error-prone regions. Unlike conventional pixel-wise uncertainty maps, which tend to be noisy and difficult to interpret, we propose reasoning at the level of structural units--branches and connections--which are more visually discernible and actionable. Finally, we leverage topology for generative modeling. We propose a topology-guided diffusion framework that can be controlled using structural attributes like object count and connectivity.
Together, these contributions establish a unified approach to topology-informed, structure-preserving dense prediction models. By integrating topological reasoning with deep networks, this thesis advances models that are not only accurate, but also structurally consistent, interpretable, and controllable. The results from this thesis have been published in ECCV, NeurIPS, and ICLR.
Speaker: Saumya Gupta
Location: New Computer Science (NCS) 120
Zoom: https://stonybrook.zoom.us/j/
Topology, in a general sense, is a mathematical language for describing structure. It delineates how different parts of an image relate to one another, capturing both individual structures and their overall layout. Preserving topology enforces structural correctness and, by extension, semantic validity.
In this thesis, we investigate how topological constraints can be used to bridge the gap between local and global understanding. We use topology to inform the design of deep learning models that are explicitly structure-aware. Our thesis focuses on dense prediction tasks, which include image segmentation, uncertainty estimation, and generative modeling. First, we introduce a topological interaction module for semantic segmentation that encodes containment and exclusion constraints directly into the learning process. This preserves anatomical hierarchies and improves multi-class consistency. Next, since segmentation models can never be truly perfect, we address the need for reliable uncertainty estimation to identify error-prone regions. Unlike conventional pixel-wise uncertainty maps, which tend to be noisy and difficult to interpret, we propose reasoning at the level of structural units--branches and connections--which are more visually discernible and actionable. Finally, we leverage topology for generative modeling. We propose a topology-guided diffusion framework that can be controlled using structural attributes like object count and connectivity.
Together, these contributions establish a unified approach to topology-informed, structure-preserving dense prediction models. By integrating topological reasoning with deep networks, this thesis advances models that are not only accurate, but also structurally consistent, interpretable, and controllable. The results from this thesis have been published in ECCV, NeurIPS, and ICLR.
Speaker: Saumya Gupta
Location: New Computer Science (NCS) 120
Zoom: https://stonybrook.zoom.us/j/
