Thin Structure IoU (TS-IoU)

• Thin Structure IoU is a metric designed to assess fine and thin object boundaries by aggregating contour overlaps across multiple scales.
• It leverages a multiscale approach inspired by fractal geometry, evaluating both coarse and fine details to penalize smoothed or missing boundary features.
• Experimental validations on synthetic and real datasets demonstrate TS-IoU's sensitivity to structural degradations, offering robust assessment for segmentation tasks.

Thin Structure IoU (TS-IoU), formally introduced as Multiscale IoU (MIoU), is an evaluation metric designed to address the insensitivity of conventional region-based scores—such as Intersection over Union (IoU) and F₁-score—to the fine and thin structural properties of detected object boundaries. This metric evaluates the similarity between a predicted and ground-truth contour by measuring boundary overlap simultaneously at a collection of spatial resolutions, thereby quantifying both coarse and fine-scale agreement, and providing enhanced sensitivity to thin, jagged, or highly detailed object shapes (Ahmadzadeh et al., 2021).

1. Formal Definition and Computational Procedure

Let $o \subset \mathbb{Z}^2$ denote the ground-truth contour and $\tilde{o}$ the predicted contour. Conventional boundary-based IoU is defined as:

$$\mathrm{IoU}(o, \tilde{o}) = \frac{| o \cap \tilde{o} |}{| o \cup \tilde{o} |}$$

To generalize IoU across scales, MIoU proceeds as follows:

• Define a set of cell sizes (scales) $\Delta = \{ \delta_1, \delta_2, ..., \delta_K \}$, e.g., $\Delta = \{2^0, 2^1, ..., 2^9\}$.
• For each $\delta \in \Delta$:
  • Partition the image into a grid of $\delta \times \delta$ cells.
  • Downsample both $o$ and $\tilde{o}$ into binary occupancy grids $s(o, \delta)$ and $s(\tilde{o}, \delta)$, by marking a cell as "on" if it overlaps the contour.
  • Set $n(o, \delta) = \| s(o, \delta) \|_1$ (total “on” cells in the ground truth at scale $\delta$).
  • Compute the scale-specific intersection ratio:

  $$r(o, \tilde{o}; \delta) = \frac{n\big(s(o, \delta) \cap s(\tilde{o}, \delta)\big)}{n(o, \delta)}$$

• Average over scales:

$$\mathrm{MIoU}(o, \tilde{o}) = \frac{1}{K} \sum_{i=1}^K r(o, \tilde{o}; \delta_i)$$

In the case $K=1$ and $\delta_1 = 1$, MIoU reduces to the standard boundary-IoU.

2. Motivational Foundations: Fractal Dimension Connection

The rationale for MIoU's multiscale approach draws directly from concepts in fractal geometry, specifically the box-counting estimation of fractal dimension (FD):

$$D(o) = \lim_{\delta \to 0} \frac{\log n(o, \delta)}{\log(1/\delta)}$$

Fractal dimension quantifies the complexity or jaggedness of boundaries. While MIoU does not explicitly compute $D(o)$, the process of assessing overlap across a range of decreasing cell sizes is motivated by the technique used to estimate FD. Contours with high-frequency features will maintain high overlap scores across fine scales if predicted well, distinguishing MIoU from scale-invariant metrics.

3. Multi-resolution Evaluation and Implementation Aspects

To realize MIoU in practice:

• Choose a scale set $\Delta$ to cover fine (pixel-level) to coarse resolutions.
• For each $\delta\in\Delta$, downsample both contours via block reduction (e.g., binary max-pooling).
• Compute $r(o, \tilde{o}; \delta)$ as above.
• Aggregate via arithmetic mean over all scales.

Pseudocode as presented recommends:

• Operating strictly on contour (not filled region) images.
• Downsampling via maximum pooling in each block to preserve fine boundary occupancy.
• Flexibility in choice of scale sequence $\Delta$, which may but need not be dyadic.

Implementation is facilitated by Python/Numpy with skimage.measure.block_reduce(..., np.max) or equivalent torch routines.

4. Sensitivity to Thin and Fine Structures

Standard pixelwise area-based metrics (IoU, F₁-score) are largely insensitive to discrepancies in narrow or high-frequency boundary elements, since such features contribute negligibly to total pixel area. MIoU, by requiring localized overlap at the finest scales, penalizes missing or smoothed-out fine structures in $\tilde{o}$ relative to $o$. If a narrow protrusion exists in the ground truth but is omitted or over-smoothed in the prediction, $r(o, \tilde{o}; \delta)$ declines sharply at low $\delta$. Conversely, MIoU remains robust to coarse spatial alignment or minor boundary-conforming perturbations, as confirmed empirically.

5. Experimental Validation and Comparative Behavior

Two main experimental evaluations are reported:

• Synthetic jagged-contour test: Applying translation, scaling, and smoothing to a base shape, MIoU demonstrates monotonic sensitivity to removal of fine structure. In contrast, IoU, Precision, Recall, and F₁-score plateau when overall area overlap is held constant, missing structural degradations.
• COCO-derived real-object masks: Testing affine deformations and deliberate smoothing on 2,500 masks (cars, bicycles, boats, dogs, persons), MIoU and IoU agree in median for minor boundary-preserving perturbations, but MIoU exhibits lower variance. For smoothed-out (structure-erased) masks, MIoU medians are consistently lower, highlighting the added penalty for loss of fine detail (Ahmadzadeh et al., 2021).

A summary of experiment findings is shown below:

Experiment Type	IoU Behavior	MIoU Behavior
Synthetic smoothing/translation	Plateaus as overlap stays	Drops with loss of fine detail
COCO, slight affine transform	Variance persists	Lower variance (robust to perturbation)
COCO, aggressive smoothing	Medians high (∼ area overlap)	Medians low (sensitive to smoothing)

6. Practical Usage and Implications

MIoU is particularly well-suited for applications where accurate detection of fine-scale object boundaries is essential, such as salient object detection or evaluation of segmentation outcomes involving thin, jagged, or high-frequency structural features. By aggregating boundary overlap over multiple scales, MIoU automatically down-weights trivial pixelwise matches and up-weights fine-scale accuracy. Operating over contour images, the metric can be computed efficiently and avoids the overestimation of quality associated with smoothed or structure-obliterating predictions. A plausible implication is that adoption of MIoU forces models to capture not just rough object location but also detailed boundary conformity, facilitating comparison of methods that explicitly target thin structure preservation (Ahmadzadeh et al., 2021).