Saliency Driven Region-Edge-based Top Down Level Set Evolution Reveals the Asynchronous Focus in Image Segmentation
1. Abstract
Level set method (LSM) is popular in image segmentation due to its intrinsic features for handling complex shapes and topological changes. Existing LSM-based segmentation models can be generally grouped into region- and edge-based models. The region-LSM-based approaches often have problems to deal with images whose objects have similar color intensity to that of the background when the region descriptor is insufficient. The edge-LSM-based models usually suffer from boundary leakage problem when the images’ edges are weak. To overcome these problems, we present a novel hierarchical level set evolution protocol (SDREL), where we propose to use both saliency map and color intensity as region external energy to motivate an initial evolution of level set function (LSF) first, followed by the LSF further smoothed by an internal energy (regulation term) to realize a more precise boundary positioning. Our results show that the newly introduced saliency map term enhances the ability to extract objects from complex background and the asynchronous evolution of a single LSF results in a better segmentation. The new hierarchical SDREL model has been validated extensively and the results indicate that it has the merits of flexible initialization, robust evolution, and fast convergence.
2. Introduction
The level set method (LSM) has been widely used in image segmentation due to its intrinsic features when handling complex shapes and topological changes. Existing level set-based segmentation models can be generally divided into two groups: region-based and edge-based models. The region-based level set models can be further divided into global fitting model and local fitting model. The global fitting model relies on the fitting of global color and intensity information, and the local fitting model depends on the fitting of local intensity information. In most region-based level set approaches, the region descriptor is only constituted by color intensity, which is insufficient to describe those images whose objects have similar color intensity to that of the background. The edge-based models often suffer from boundary leakage problem when the images’ edges are weak. To overcome these problems, we propose a novel level set evolution protocol, which uses saliency map and color intensity as external energy, apart from internal energy, to motivate the evolution of level set function (LSF). As a new term of energy function of LSF, saliency map enhances the ability to extract objects from background, and the combined edge information ensures the precise boundary poisoning. A top-down two-stage LSF evolution approach is designed in this study: in the first stage, evolution is motivated by the external energy which consists of color intensity, saliency and gradient map, resulting in a rough segmentation of the image; in the second stage, the LSF is further smoothed by the internal energy (regulation term) so that boundary positioning becomes more precise and singularities are eliminated. The proposed method is called SDREL, which has been validated extensively on both synthetic and real world gray scale and color images. By comparing segmentation results and speed of evolution with that of existing level set methods, SDREL shows the merits of flexible initialization, robust evolution, fast convergence and better segmentation results.
3. SDREL model
The energy function of SDREL model is:
To make sufficient use of external energy and internal energy in different stage of evolution, we implement the evolution of LSF as a two-stage process, where external energy is mainly depended in the first stage and internal energy plays a key role in the last stage.
Here is an example showing the effect of two-stage segmentation:
The left image is the original image with initial checkerboard contour, and the right image shows the energy graph of LSF.
The result of first stage is in the left and the final result is in the right.
6. Reference and contact
Reference: Xu-Hao Zhi and Hong-Bin Shen, Saliency Driven Region-Edge-based Top Down Level Set Evolution Reveals the Asynchronous Focus in Image Segmentation, Pattern Recognition, 2018, 80: 241-255.
Contact: zhixuhao@163.com (X.H. Zhi), hbshen@sjtu.edu.cn (H.B. Shen)
