Discrete Inverse Rendering: Biological Image Analysis with Integer Programming

Abstract

Biological image and video analysis is full of discrete decisions: whether an object is present, which multi-hypothesis detections are real, whether two detections are tracking the same object, or whether a cell divides or not. Standard pipelines resolve these locally and in sequence, e.g through non-max suppression, per-frame segmentation, distance-based linking, or dedicated lineage rules. Image evidence and temporal evidence are thus rarely weighed against each other in a single objective. We recast these discrete steps as discrete inverse rendering : candidate renderings are generated, and then a integer programming solver selects the subset that best reconstructs the observed video subject to problem-specific constraints. We demonstrate that the same solver and the same reconstruction principle can handle three otherwise separate motifs: suppression of overlapping detections in dense C. elegans tracking, extraction of a single connected curve in sperm tracking, and event-structured tracking in single cell tracking.