文献速递：肿瘤分割---- 弱监督肝肿瘤分割，使用Couinaud区段标注

文献速递介绍

肝癌是世界上导致癌症死亡的主要原因之一，也是第二大常见的癌症死因。本稿件于2021年10月28日收到，2021年11月24日修订，2021年12月1日接受。发表日期为2021年12月6日；当前版本日期为2022年5月2日。此项工作得到了07180216号健康和医学研究基金项目的支持。（通讯作者：袁邦成。）吕飞和袁邦成与香港浸会大学计算机科学系合作（电子邮件：feilyu@comp.hkbu.edu.hk; pcyuen@comp.hkbu.edu.hk）。马俊浩与中山大学计算机科学与工程学院合作，位于中国广州510006（电子邮件：majh8@mail.sysu.edu.cn）。叶卓锋和黄丽红与香港中文大学医学及治疗学系和消化疾病研究所的医学数据分析中心合作（电子邮件：terryfungyip@gmail.com; wonglaihung@cuhk.edu.hk）。数字对象标识符10.1109/TMI.2021.3132905

从计算机断层扫描（CT）体积图像中分割肝肿瘤对于许多临床应用（如治疗计划和治疗后评估）至关重要。最近，深度神经网络在自动肝肿瘤分割方面取得了令人印象深刻的进展。然而，这些领先的方法依赖于精确的像素级注释。获取此类注释非常困难，因为它既耗时又对专业知识有严格要求。因此，开发能在高质量标记数据不可用时表现良好的深度学习方法是可取的。

Title

题目

Weakly Supervised Liver Tumor Segmentation Using Couinaud Segment Annotation

弱监督肝肿瘤分割，使用Couinaud区段标注

Abstract

摘要

Automaticliver tumor segmentationis of great importance for assisting doctors in liver cancer diagnosis and treatment planning.Recently,deep learning approaches trained with pixel-level annotations have contributed many breakthroughs in image segmentation. However, acquiring such accurate dense annotations is time-consuming and labor-intensive,which limits the performance of deep neural networks for medical image segmentation. We note that Couinaud segment is widely used by radiologists when recording liver cancer-related findings in the reports, since it is well-suited for describing the localization of tumors. In this paper, we propose a novel approach to train convolutional networks for liver tumor segmentation using Couinaud segment annotations. Couinaud segment annotations are image-level labels with values ranging from 1 to 8, indicating a specific region of the liver. Our proposed model, namely CouinaudNet, can estimate pseudo tumor masks from the Couinaud segment annotations as pixel-wise supervision for training a fully supervised tumor segmentation model, and it is composed of two components: 1) an inpainting network with Couinaud segment masks which can effectively remove tumors for pathological images by filling the tumor regions with plausible healthy-looking intensities; 2) a difference spotting network for segmenting the tumors, which is trained with healthy-pathological pairs generated by an effective tumor synthesis strategy. The proposed method is extensively evaluated on two liver tumor segmentation datasets. The experimental results demonstrate that our method can achieve competitive performance compared to the fully supervised counterpart and the state-of-the-art methods while requiring significantly less annotation effort.

自动化肝肿瘤分割对于协助医生诊断肝癌和规划治疗方案至关重要。最近，通过像素级注释训练的深度学习方法在图像分割方面取得了许多突破。然而，获取这种精确的密集注释是耗时且劳力密集的，这限制了深度神经网络在医学图像分割中的性能。我们注意到，放射科医生在记录肝癌相关发现的报告中广泛使用Couinaud区段，因为它非常适合描述肿瘤的定位。在本文中，我们提出了一种使用Couinaud区段注释训练卷积网络进行肝肿瘤分割的新方法。Couinaud区段注释是图像级标签，值范围从1到8，指示肝脏的特定区域。我们提出的模型，即CouinaudNet，可以从Couinaud区段注释中估算出伪肿瘤掩模，作为像素级监督，用于训练全监督肿瘤分割模型，它由两部分组成：1）一个带有Couinaud区段掩模的修补网络，可以有效地通过用看似健康的强度填充肿瘤区域来移除病理图像中的肿瘤；2）一个用于分割肿瘤的差异定位网络，通过有效的肿瘤合成策略生成的健康-病理对进行训练。所提出的方法在两个肝肿瘤分割数据集上进行了广泛评估。实验结果表明，与全监督对应方法和最先进方法相比，我们的方法在需要显著减少注释工作量的情况下，可以实现具有竞争力的性能。

Methods

方法

Most leading deep learning based semantic segmentation methods demand pixel-level supervision, and they are not directly applicable if only Couinaud segment annotations are given. Our CouinaudNet is proposed to address this problem, which can estimate pseudo tumor masks from Couinaud segment annotations as pixel-level supervision. A simple schematic of our proposed method for utilizing Couinaud segment annotation is shown in Fig. 2. Training images can be split into two parts: healthy training images without tumors (label:0), pathological training images with tumors (label: 1∼8). Couinaud scheme can be obtained by an automatic Couinaud Constructor (CC). To remove irrelevant information, only the regions inside the liver are kept after pre-processing (Pre). First, only healthy training images areused to train the proposed CouinaudNet which includes an Inpainting network (I) and a Difference Spotting network (DS). Vessel map detected by the Frangi filter (F) is used as an extra input to assist DS. Next, the trained CouinaudNet is used to estimate pseudo tumor masks for those pathological training images. Finally, we can train a fully supervised Tumor Segmentation model (TS) with all training images and their tumor masks, i.e., healthy images with healthy masks (black masks with no tumors), pathological images with estimated pseudo tumor masks. In the testing phase, only the trained TS is used for inference.

大多数领先的基于深度学习的语义分割方法都需要像素级监督，并且如果只给出Couinaud区段注释，则无法直接应用。我们提出的CouinaudNet旨在解决这个问题，它可以从Couinaud区段注释中估算出伪肿瘤掩模，作为像素级监督。图2展示了我们提出的方法利用Couinaud区段注释的简单示意图。训练图像可以分为两部分：没有肿瘤的健康训练图像（标签：0），有肿瘤的病理训练图像（标签：1∼8）。Couinaud方案可以通过自动Couinaud构造器（CC）获得。为了去除无关信息，在预处理（Pre）后只保留肝脏内的区域。首先，只使用健康训练图像来训练提出的CouinaudNet，其中包括修补网络（I）和差异定位网络（DS）。Frangi滤波器（F）检测到的血管图作为额外输入以协助DS。接下来，训练好的CouinaudNet用于估算病理训练图像的伪肿瘤掩模。最后，我们可以用所有训练图像及其肿瘤掩模来训练全监督肿瘤分割模型（TS），即健康图像配合健康掩模（没有肿瘤的黑色掩模），病理图像配合估算的伪肿瘤掩模。在测试阶段，只使用训练好的TS进行推断。

Conclusions

结论

In this paper, we present a weakly-supervised method for liver tumor segmentation based on Couinaud segment annotations. A novel CouinaudNet is introduced to estimate pseudo tumor masks from the Couinaud segment annotations, which can be used as pixel-wise supervision to train deep neural networks for liver tumor segmentation. Our method achieves competitive performances compared to fully supervised methods while requiring significantly less annotation effort. Moreover, Couinaud segment information can be easily extracted from free-text reports with text mining techniques, and our method provides a promising direction for exploring the potential of huge amounts of free-text reports which are stored in hospital archiving and communication systems.

在本文中，我们提出了一种基于Couinaud区段注释的肝肿瘤弱监督分割方法。我们引入了一种新颖的CouinaudNet来估算来自Couinaud区段注释的伪肿瘤掩模，这可以作为像素级监督来训练深度神经网络进行肝肿瘤分割。我们的方法与全监督方法相比取得了具有竞争力的性能，同时显著减少了注释工作量。此外，Couinaud区段信息可以通过文本挖掘技术轻松从自由文本报告中提取，我们的方法为探索存储在医院归档和通信系统中的大量自由文本报告的潜力提供了有希望的方向。

Figure

图

Fig. 1. (a) An example of input CT image, its fully supervised pixel-wise tumor mask annotation, and weakly supervised Couinaud segment annotation used in our method, which is a number ranging from 1 to 8 and refers to a specific region of the liver. We annotate healthy images without tumors using label 0, and annotate those pathological images with tumors using Couinaud segment annotations (1 ∼ 8) indicating the tumor location. (b) The Couinaud segment mask can be derived from the Couinaud segment annotation based on Couinaud scheme. Couinaud scheme is the widely used system which divides the liver into eight functionally independent segments based on its vascular structure. Automatic methods have been successfully applied to construct the Couinaud scheme from CT volumes without manual efforts.

图1.（a）输入CT图像的示例，其全监督像素级肿瘤掩模注释，以及我们方法中使用的弱监督Couinaud区段注释，该注释是一个从1到8的数字，指的是肝脏的特定区域。我们使用标签0来注释没有肿瘤的健康图像，并使用Couinaud区段注释（1 ∼ 8）来注释那些有肿瘤的病理图像，表明肿瘤位置。（b）基于Couinaud方案，可以从Couinaud区段注释中推导出Couinaud区段掩模。Couinaud方案是一种广泛使用的系统，根据肝脏的血管结构将肝脏划分为八个功能独立的区段。自动化方法已成功应用于从CT体积图像中构建Couinaud方案，无需人工努力。

Fig. 2. Schematic of our approach. CouinaudNet is first trained only with healthy training images, then the trained CouinaudNet is used to generate pseudo tumor mask mt for the pathological training image xt. A healthy training image xh has no tumors, and its tumor mask is denoted as a black mask mh. Provided all training images and their tumor masks, we can train a fully supervised tumor segmentation model.

图2.我们方法的示意图。CouinaudNet首先只用健康训练图像进行训练，然后使用训练好的CouinaudNet为病理训练图像xt生成伪肿瘤掩模mt。健康训练图像xh没有肿瘤，其肿瘤掩模表示为黑色掩模mh。提供所有训练图像及其肿瘤掩模后，我们可以训练一个全监督肿瘤分割模型。

Fig. 3. Training the proposed CouinaudNet. Couinaud scheme cs can be obtained by an automatic Couinaud constructor (CC). In Stage 1, we randomly select one Couinaud segment as the mask mc for training the inpainting network (I). In Stage 2, a synthetic tumor image xs, its inpainted image xˆs and its vessel map xv by Frangi filter (F) are sent to a difference spotting network (DS) for segmenting the tumors.

图3.训练提出的CouinaudNet。Couinaud方案cs可以通过自动Couinaud构造器（CC）获得。在第一阶段，我们随机选择一个Couinaud区段作为训练修补网络（I）的掩模mc。在第二阶段，合成肿瘤图像xs、其修补后图像xˆs以及Frangi滤波器（F）生成的血管图xv被送入差异定位网络（DS）进行肿瘤分割。

Fig. 4. Example intensity distributions of liver and tumor.

图4.肝脏和肿瘤的强度分布示例。

Fig. 5. (a) and (b) Distributions of μd and σt. © is an illustration of the Beta distributions with different shape parameters.

图5.（a）和（b）显示了μd和σt的分布。（c）是具有不同形状参数的Beta分布的示意图。

Fig. 6. Performance of tumor segmentation based on the inpainting models using different masks.

图6。基于使用不同掩模的修补模型的肿瘤分割性能。

Fig. 7. Parameter analysis of the threshold τ for binarization.

图7。二值化阈值τ的参数分析。

Fig. 8. Diverse appearances of synthetic tumors based on different mask and intensity parameters, where nSP, sp, αe, σe are used for mask generation, μd and σt are used for intensity generation.

图8。基于不同掩模和强度参数产生的合成肿瘤的多样化外观，其中nSP、sp、αe、σe用于掩模生成，μd和σt用于强度生成。

Fig. 9. Parameter analysis of the shape parameters in Beta distribution for generating tumor intensities (Equation 4), and the dashed lines represent the performance in Stage 1.

图9。用于生成肿瘤强度（方程4）的Beta分布形状参数的参数分析，虚线代表第一阶段的性能。

Fig. 10. Generated pseudo tumor masks used as supervision. (a) A training image with its ground truth tumor mask. (b)–(d) The tumor-free versions generated by VAE, AAE, f-AnoGAN* respectively, and their tumor masks after binarization which are restricted within the Couinaud segment masks. (e) and (f) show pseudo-healthy images from the inpainting network, and the tumor masks from Stage 1 and Stage 2 respectively.

图10。用作监督的生成的伪肿瘤掩模。（a）训练图像及其真实肿瘤掩模。（b）-（d）分别为VAE、AAE、f-AnoGAN*生成的无肿瘤版本及其二值化后的肿瘤掩模，这些掩模被限制在Couinaud区段掩模内。（e）和（f）分别展示了修补网络的伪健康图像，以及第一阶段和第二阶段的肿瘤掩模。

Fig. 11. Examples of tumor segmentation results by models trained with Couinaud segment annotations using our proposed method on the MSD08 test dataset. The green ones denote the ground truths, while the red ones denote the segmentation results using our method.

图11。使用我们提出的方法，通过Couinaud区段注释训练的模型在MSD08测试数据集上的肿瘤分割结果示例。绿色部分表示真实标签，红色部分表示使用我们方法的分割结果。