Paper Notes

[Kinpzz]

[Kinpzz]

ChestX-ray8

ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks（标杆） on Weakly-Supervised（弱监督） Classification and Localization of Common Thorax（胸部） Diseases

Keyword

• Weakly-Supervised（弱监督）
• Multi-labels Classification（多标签分类）
• Localization（定位）
• Hospital-scale Chest X-ray Database（医院规模X光胸片数据库）
• Commn Thorax Diseases（常见胸部疾病）
• NLP（自然语言处理）

弱监督：没有基于像素标记的训练图像，只有基于图像类别标签的图像， image-level class labels only

Background

Mordern hospitals' PACS(Picture Archiving and Communication Systems) has a tremendous number of X-ray imaging studies accompanied by radiological reports(ie. loosely labeled). Open question: How this type of hospital-size Knowledge database --used for-> large-scale high precision computer-aided diagnosis(CAD) systems.

State-of-the-art object dection and segmentation

Dataset

• MS COCO: 80 categories, 200k images, 1.2M instances(350k people)
• PASCAL VOC: 20 categories, 11530 images, containing 27450 annotated objects with bounding-boxes(BBox)

Object Detection

• MS COCO: 0.413 mAP
• PASCAL VOC: 0.884 mAP

Main limitation of recent notable work

All proposed methods are eavaluated on some small-to-middle scale problems of (at most) several hunders patients. The performance of deep learning techniques remians unclear when it scales up to tens of thousands of patient sudies. 目前研究的不足：样本量偏小，数据稀缺

Related Work

There have been recent efforts on creating openly available annotated medical image database.

• OpenI (An open access biomedical search engine): 3955 radiology reports, 7470 associated chest x-rays --> caption generation

Motivation

1. Generic, opened iamge-level anatomy and pathology labels cannot be obtained through crowd-sourcing, such as AMT(Amazon Mechanical Turk) 无医学背景的标注者无法标注医学图像标签，所以使用NLP结合image和reports 提取标签。
2. The spatial dimensions of an chest X-ray are usually 2000x3000 pixels. 但是局部病理图片区域大小差异大，且相对于原图片很小。对此，本文提出了一种弱监督多标签分类和定位的框架来解决这个困难。
3. 医学图像诊断不适合直接使用ImageNet pre-trained DCNN model来fine-tune，因此需要建立弱监督医学图像数据库并学习recognition和localization。

Main Work

• 构建了一个新的弱监督多标签医学图像数据集数据集ChestX-ray8，比以往做研究的样本量要大。
• 证明了在弱监督多标签图像的情况下，还可以判断和空间定位常见的胸部疾病。
• 未来目标：构建全自动的“reading chest X-rays” systems PS: 介绍可围绕chestX-ray8和判断定位方法两个方面来介绍

Construting Database

ChestX-ray8

• A new chest X-ray database proposed by this paper.
• 108,948 frontal-view X-ray images of 32,717 unique patients (collected from 1992 to 2015).
• 24636 images contain one or more pathologies. 84312 images are normal cases.
• Mined from authors' institute's PACS system.
• Dimension: 1024 x 1024 bitmap
• 200 instances for each pathology (totally 983 images) labeled with a B-Box as GT
• 8 thoracic pathologies
  • Atelectasis（肺不张，肺萎缩）
  • Cardiomegaly（心肥大，心脏扩大症）
  • Effusion（积液）
  • Infiltration（浸润）
  • Mass（肿块）
  • Nodule（结节）
  • Pneumonia（肺炎）
  • Pneumothorax（气胸）
• Each image can have multi-labels, including 8 thoracic pathologies and "Normal" labels. (Nomal指的是不含任何病理，不局限上述8个病理）
• labels from the associated radiological reports using NLP.
• There are connections between different pathologies

Labeling Disease Names by Text Mining（标签提取）

Tools

• DNorm: a machine learning method for disease recognition and normalization. It maps every mention of keywords in a report to a unique concept ID in the Systematized Nomenclature of Medicine Clinical Terms (or SNOMED-CT), which is a standardized vocalbulary of clinical terminology for the electronic exchange of clinical health infromation.（临床术语转换）
• MetaMap: a tool to detect bioconcepts from the biomedical text corpus. Different from DNorm, it is an ontology-based approach for the detection of Unified Medical Language System(UMLS) Metathesaurus.（也是用户临床术语检测和转换） ProcessMerge the results of DNorm and MetaMap. Transefer : reports -->SNOMED-CT --> UMLS terminology

Noise（上述工具存在噪声问题）

Eliminate noisy labeling by ruling out negated pathological statements（否认形式的陈述） and uncertain mentions of findings and diseases, e.g., "suggesting obstructive lung disease". Use regular expression can not capture various syntatic constructions for multiple subjects. for example, "clear of A and B" -> A as a negation but not B.

Improvement: syntactic level, utilize the syntactic dependency information. Define rules on the dependency graph, by utilizing the dependency label and direction information between words. 相比于之前Tools的改进：

Steps

1. split and tokenize the reports into sentences using NLTK.
2. parse each sentence by Bllip parser using David McCloskys biomedical model.
3. the syntactic dependencies are obtained from "CCProcesed" dependencies output by applying Stanford dependencies converter on the parse tree.

Quality Control

Using OpenI API, retrieve a total of 3851 unique radiology reports for validation. Performance相比于MetaMap有较大的提升

Processing Chest X-ray Images

• DICOM file : (TODO) introduction
• Typical X-ray image dimensions: 3000 x 2000
• ChestX-ray8 : resized as 1024 x 1024 bitmap
• OpenI: 512 x 512

Bouding Box for Pathologies

• 200 instances for each pathology( 1600 instances total), consisting of 983 images.
• Given an image and a disease keyword, a board-certified radiologist identified only the corresponding disease instance in the image and labeled it with a B-Box.

Unified DCNN Framework

• models: ImageNet pre-trained models, e.g., AlexNet, GoogleNet, VGGNet-16, ResNet-50, by leaving out the fully-connected layers and the final classification layers.
• 改进：Insert a transition layer, a global pooling layer, a prediction layer, a loss layer

Multi-label Setup

8-dimensional label vector $$ y = [y_1,..., y_c, ..., y_C], y_c \in {0,1], C= 8$$ for each image. $$ y_c $$ indicates the presence with respect to according pathology. Normal: [0, 0, 0, 0, 0, 0, 0, 0]

Transition Layer

To transform the activations from previous layers into a uniform dimension of output, $ S \times S \times D, S \in {8, 16, 32}$. D represents the dimension of features at spatial location $$ (i, j), i,j \in {1, ..., S}, which can be avried in different model settings, e.g., D=1024 for GoogLeNet and D=2048 for ResNet.

通过卷积层运算把不同的pre-trained model转换为$S \times S \times D$的输出。

Multi-label Classification Loss Layer

• Firstly, Hinge Loss(HL), Euclidean Loss(EL), and Cross Entropy Loss(CEL) replace softmax loss.
  • It has difficulty learning positive instances(with pathologies), because there are more '0's than '1's
• Improvement: positive/negative instances balanced loss function: weight CEL(W-CEL)
  • $$ L_{W-CEL}(f(x), y) = \Beta_P\sum_{y_c=1}-\ln (f(x_c))+\Beta_N\sum_{y_c=0}-\ln (1-f(x_c))$$, where $$\Beta_P$$ is set to $$\frac{|P|+|N|}{|P|}$$, while $$\Beta_N$$ is set to $$\frac{|P|+|N|}{|N|}. |P| and |N| are the total number of '1's and '0's in a batch of image labels.

在损失函数中加入了对正负样本均衡性的考虑。

Global Pooling Layer

• Role: choose what information to be passed down
• Max pooling, average pooling, Log-Sum-Exp(LSE) pooling
  • LSE pooled value $$x_p = \frac{1}{r} \dot \log[\frac{1}{S} \dot \sum_{(i,j)\in S} exp(r \dot x_{ij})]$$, $$x_{ij}$$ is the activation value at $$(i,j),(i,j)$$ is one location in the pooling region S, and $$ S=s\times s$$ is the total number of locations in S.
  • LSE has overflow/underflow problems
• Improvment: $$ x_p = x^* + \frac{1}{r} \dot \log [\frac{1}{S} \dot \sum_{(i,j) \in S} exp(r \dot(x_{ij} - x^)]$$, where $$ x^ = max{|x_{ij|, (i,j)\in S}$$

采用了全局池化层代替全连接层和softmax层，减少参数量，防止过拟合。并且设计了一个全局最大池化层和全局LSE池化层结合的方案，max和ave之间权衡。

prediction layer

• Weight of prediction layer : size of $$ D \times C $$ (C = 8, class number)
• Transfer $$1 \times D $$ to $ 1 \times C $$

预测层将全局池化层的输出转换为$1 \times C$ 维度。并利用ROC曲线进行不同阈值效果的筛选。 ROC参考：http://blog.csdn.net/pipisorry/article/details/51788927?locationNum=1&fps=1 http://blog.csdn.net/taoyanqi8932/article/details/54409314?locationNum=5&fps=1

Heat map

• Multiply activations form transition layer with weights from prediction layer
• 8 heatmap for 8 diseases
• size: $$ S \times S, S \in {8, 16, 32} $$

Bounding Box Generation:

• Normalized heatmaps to [0,255]
• Threshold heatmap by {60, 180} individually (ad-hoc)
• B-Boxes are generated to cover the isolated regions in the resulting binary maps

Experiments

CNN

• SGD
• training(70%), validation(10%), testing(20%)
• Recognition on testing set
• generate B-Boxes on 983 images with GT
• Caffe framework
• ImageNet pre-trained models , i.e., AlexNet. GoogLeNet, VGGNet-16, ResNet-50 from Caffe model zoo.
• set $$ batch_size \times iter_size = 80$$ as a constant
• total trainning iterations are customized for different CNN models to prevent over-fitting.

Performance

ROC

• performance on four models

AUC

• Mass : AUC=0.5609, huge within-class appearance variation
• Pneumonia: AUC=0.6333, lack of total instances(less than 1%)

Different pooling strategies

• AVE, MAX, different hyper-parameter r in LSE
• best in r = 10

W-CEL

Disease Localization

• standard Intersection over Union ratio (IoU)
• the Intersection over the detected B-Box area ratio (IoBB)
• computed B-Boxes are often larger than GT due to heat map resolution
  • difine a correct localization by requiring either $$IoU > T(IoU) or IoBB > T(IoBB)$$
  • T(IoBB) 为超参数，用来筛选B-Box
• localization accuracy (Acc,)
• Average False Positive (AFP)

[Kinpzz]

SCAN

Introduction

• Title: Structure Correcting Adversarial Network for Organ Segmentation in Chest X-rays
• Author: Wei Dai, Joseph Doyle, Xiaodan Liang, Hao Zhang, Nanqing Dong, Yuan Li, Eric P. Xing Petuum Inc.
• arXiv:

本文主要是一篇关于对双肺和心脏进行语义分割的论文，作者认为器官语义分割是针对胸片（CXR）构建计算机辅助诊断系统的重要一步，器官的区域提供了丰富的结构信息，可用于诊断许多病症。而目前胸片又因辐射小、花费低，而十分普遍，给放射科工作者带来了巨大的工作量。所以本文的研究具有现实意义。同时该研究也存在着巨大的挑战，CXR为2d灰度图片，且目前公开数据集数据量很少（多只有几百张），无法直接应用在大规模数据集上训练好的网络模型。作者据此提出了SCAN框架，该模型采用了GAN（生成对抗网络）的思想，包含了一个分割网络(segmentation network)和一个判别网络(critic network)，采用零和博弈的思想，在公开数据集JSRT和Montgomery上进行单独交替训练。这两个网络都是一个复杂的神经网络，包含FCN、和VGG-based（VGG基础上进行修改）、残差块(residual block)。这是一个数据依赖性小（不依赖大规模数据）、参数量小的模型，取得了一个高准确率（人类专家水平）、高效率（<1s）、迁移性强（泛化能力强）的结果，超过该研究领域的state-of-the-art Registration-based approach。

Keyword

• Adversarial Network（对抗网络）
  • critic network（判别网络）
  • segmentaiton network（分割网络）
• Organ Segmentation（器官图像分割）
• Chest X-rays (CXR)
• Structure Correcting（通过critic network 获取 Global structure information 全局结构信息）
• FCN + GAN

Main Work

• Propose Structure Correcting Adversarial Network(SCAN) to segment lung fields and the heart in CXR images
  • a critic network: learns to discriminate between the ground truth organ annotations from the masks synthesized by the segmentation network during trainning; learns the higher order regularities and effectively transfers this global information back to the segmentation model to achieve realistic segmentation outcome（critic network帮助学习到高层的结构信息，单靠分割模型会面临训练样本量不足问题）
  • segmentation model : convolutional network
  • end-to-end
• The model produces highly accurate and natural segmentation. （高准确率）
  • 94.7% IoU for lung fields (human experts: 94.6%)
  • 86.6% IoU for heart fields (human experts: 87.8%)
  • Surpass current state-of-the-art（超越当前最高水平）
• Using only very limited trainning data availabel, the model reaches human-level performance without relying on any existing trained model or dataset.（数据依赖性小，可达人类识别水平）
  • SCAN model is more robust when applied to a new, unseen dataset, outperforming the vanilla segmentation model by 4.3%

Background

Motivation

• Chest X-ray (CXR) ofen with over 2-10x more scans than other imaging modalities such as MRI, CT scan and PET scans due to its low cost and low dose of radiation. It is asignificant workloads on radiologists and medical practitioners.（CXR花费低，辐射少，导致数量多, 工作负担重）
  • In 2015/16, in UK's public medical sector: 22.5M X-ray images (8M CXR), 4.5M CT, 3.1M MRI
  • Shortage of radilogists in the world
• Organ segmentation is a crucial step to obtain effecive computer-aided detection on CXR.（器官分割CXR计算机辅助诊断重要一步）
  • The segmentation of the lung fields and the heart provides rich structure information about shape irregularities and size measurements that can be used to directly asses certain serious clinical conditions, such as cardiomegaly（心肥大）, pneumothorax（气胸）, pleural effusion（胸腔积液）, emphysema（肺气肿）.
  • Explicit lung region masks can improve interpretability of computer-aided detection, which is important fir clinical use.

Challenge

• X-rays have low resolution and 2-D projection compared with the more modern medical imaging technologies such as CT scan and PET scans.（X光分辨率低，2d成像）
• Very limited CXR trainning data with pixel-level annotations due to expense（像素级标注的CXR训练数据很少）
• CXRs exhibit substantial variations across different patient populations, pathological conditions, imaging technology and operation（CXR样本差异性大）
• CXR images are gray-scale and drastically different from natural images（CXR图是灰度图，现有模型可迁移性差）
• to incorporate the implicit mdedical knowledge involved in contour determination.（如何将医学知识融入边缘判定）
  • medical experts look for certain consistent structures surrounding the lung fields while annotating the lung fields.（医学专家在标定边缘的时候会寻找特定结构，如aortic arch（主动脉弓），cardiodiaphragmatic angles（心隔角））
  • Therefore， a successful segmentation model must effectively leverage global structual information to resolve the local details.（可突破点：应用全局结构信息）
• high contrast between rib cage and lung fields.

Related Work

Lung Field Segmentation

Categories

1. Rule-based systems apply pre-defined set of thresholding and morphological operations that are derived from heuristics.
2. Pixel classification methods classify the pixels as inside or outside of the lung fields based on pixel intensities.（像素分类）
3. Based on deformable（可变形） models such as Active Shape Model (ASM) and Active Appearance Model.

Current state-of-the-art

Registration-based approach: to build a lung model for a test patient, finds patients in an existing database that are most similar to the test patient and perform linear deformation of their lung profiles based on key point matching.（比较法；关键点匹配）

Semantic Segmentation with Convolutional Networks

Aims to assign a pre-defined class to each pixel

Current state-of-the art

• Fully convolutional network (FCN)
• Improvement: Semantic segmentation using adversarial networks

We note that there is a growing body of recent works that apply neural networks end-to-end on CXR images [25, 34]. These models directly output clinical targets such as disease labels without well-defined intermediate outputs to aid interpretability. Furthermore, they generally require a large number of CXR images for training, which is not readily available for many clinical tasks involving CXR images.（目前一些成果的不足：结果未输出辅助性中间结果，直接输出标签，且需要大量训练数据）

Problem Definition

• CXRs in the posteroanterior (PA 由后向前) view
• Lung fields definition: lung fileds consist of all the pixels for which radiation passes through the lung but not through the following structures, the heart, the mediastinum（纵膈，介于两肺之间的不透明区域）, below the diaphragm（膈）, the aorta（主动脉）, and if visible, the superior vena cava（上腔静脉）.
• The heart boundary is generally visible on two sides, while the top and bottom borders of the heart have to be inferred due to occlusion by the mediastinum（心脏左右边界通常可见，上下边界被纵膈遮挡需要推测）

Structure Correcting Adversarial Network (SCAN)

Authors adapt FCNs to gray-scale CXR images uder the stringent constraint of very limited trainning dataset of 247 images. It departs from the usual VGG architecture and can be trained without transfer learning from existing models or dataset.（论文方法：FCN+对抗网络，仅需要少量训练数据，不依赖现有模型或数据库）

Adversarial Training for Smeantic Segmentation

GAN

Adversarial trainning was first proposed in Generative Adversarial Network (GAN)

• a generator network: learn the data distribution
• a critic network: estimates the probability that a sample comes from the tranning data instead of synthesized by the generator
• Adversarial process: The generator's objective is to maximize the probability that the critic makes a mistake, while the critic is optimized to minimize the chance of mistake.
• The critic, which itself can be a complex neural network, can learn to exploit higher order inconsistencies in the samples synthesized by the generator.

Use the critic to learn these higher order structures and guide the segmentation network to generate masks more consistent with the learned global structures.

Training Objectives

Data

• $S$: segmentation network
• $D$: critic network
• $x_i$: input image, shape $[H,W,1]$ for a single-channel gray-scale image with heigh $H$ and width $W$
• $y_i$: the associated mask labels, shape $[H,W,C]$ where $C$ is the number of classes including the background.
  • for each pixel location $(j,k)$, $y_i^{jkc}=1$ for the labeled class $c$ while the rest of the channels are zero($y_i^{jkc'}=0$ for $c' \neq c$).
• $S(x) \in \lbrace 0, 1 \rbrace ^{\lbrace H,W,C \rbrace}$: denote the class probabilities predicted by $S$ at each pixel location such that class probailities normalize to 1 at each pixel.（$S(x)$:通过S预测的每一个像素点每个类的概率）
• $D(x_i, y)$: scalar probability estimate of $y$ coming from the traning data (ground truth) $y_i$ instead of the predicted mask $S(x_i)$ （$D(x_\i,y)$: $y$来自训练数据(ground truth)$y_i$而非$S(x_i)$的概率）

Optimization problem

$$ \min_S \max_D \lbrace J(S,D):=\sum_{i=1}^N J_s(S(x_i), y_i) - \lambda [J_d(D(x_i, y_i), 1) + J_d(D(x_i, S(x_i)),0)] \rbrace$$

• 固定$S$，针对$D$（max下标）最大化$J(S,D)$
• 固定$D$，针对$S$最大化$J(S,D)$,
• $ J_s(y^\hat, y) := \frac{1}{HW} \sum_{j,k} \sum_{c=1}^C-y^{jkc} \ln y^{jkc}$: multi-class cross-entropy loss for predicted mask $y^\hat$ averaged over all pixels.
• $J_\d(t^\hat, t):= -t\ln t^\hat + (1-t) \ln(1-t^\hat)$ : binary logistic loss for the critic's predition
• $\lambda$ : tuning parameter balancing pixel-wise loss and the adversarial loss We can solve Eq.(1) by alternate between optimizing $S$ and optimizing $D$ using their respective loss functions.（训练方法：单独交替迭代训练）

上述公式可以拆分为下面两个阶段：

Trainning the Critic

Train the critic network by minimizing the following objective with respect to $D$ for a fixed $S$: $$ \sum_{i=1}[J_d(D(x_i, y_i), 1) + J_d(D(x_i, S(x_i)),0) $$ 相比于Eq(1) 优化公式，少了负号，所以变成了最小化问题。

Trainning the Segmentation Network

Given a fixed D, we train the segmentation network by minimizing hte following objective with respect to $S$: $$ \sum_{i=1}^N J_s(S(x_i),y_i) + \lambda J_d(D(x_i,S(x_i)),0)$$

• Use $J_d(D(x_i, S(x_i)),1) in place of $-J_d(D(x_i, S(x_i)),0)$, for $J_d(D(x_i, S(x_i)),0) leads to weaker gradient signals when $D$ makes accurate predictions.

参考

• GAN理解: http://blog.csdn.net/on2way/article/details/72773771?locationNum=7&fps=1
• minimax：https://en.wikipedia.org/wiki/Minimax
• 零和博弈(zero-sum game)
• minimizing the possible loss for a worst case (maximum loss) scenario

Segmentation Network

FCN

• The down-sampling path(下采样) 类似图像分类网络架构
  • convolutional layers
  • max/average pooling layer
  • VGG-based
  • residual block architecture
• The up-sampling path(上采样)
  • convolutional layers
  • deconvolutional layers(transposed convolution) 反卷积层
• Most FCNs are applied to color images with RGB channels which this model cannot use.
• 3 classes
  • the left lung
  • the right lung
  • the heart
• 247 CXR images

Critic Network

• input: 4 or 5(including input image) channels
• segmentation network
• global average pool
• fully connected layer

Experiments

Dataset and Processing

Dataset

Use two publicly available dataset with at least lung field annotations.

JSRT

• Released by Japanese Society of Radiological Technology (JSRT)
• 247 CXRs (154 have lung nodules and 93 have no lung nodule)
• Resolution: $2048 \times 2048$
• gray-scale with color depth of 12 bits.
• represents mostly normal lung and heart masks (lung nodules in most cases do not alter the counter of the lungs and heart

Montgomery

• Department of Health and Human Services, Montgomer Country, Marland, USA
• 138 CXRs (80 normal patients and 58 patients with manifested tuberculosis(TB肺结核))
• Resolution: $4020 \times 4892$ or $4892 \times 4020$
• gray-scale with color depth of 12 bits.
• Only the two lung masks annotations are available

Processing

• scale all images to $ 400 \times 400$ pixels(with sufficient visual details for vascular structures)
• $800 \times 800$ does not improve the segmentation performance
• image normalization : for given image $x$
  • $x^{jk} := \frac{x^{jk} - x^\hat}{\sqrt{var(x)}}$
  • $x^\hat$: mean of pixels in $x$
  • $var(x)$: variance of pixels in $x$
  • do not use statistics from the whole dataset（取单张图片均值和方差非整个数据集）
• post-processing: fill in any hole in the predicted mask, and remove the samll patches disjoint from the largest mask
  • PS: In practice, this is importtant for the predition output of the segmentation network (FCN alone), but dose not affect the evalutation results for FCN with adversarial trainning*（post-prcessing对FCN有效，对FCN对抗网络无提升效果）

Training Protocols

• GANs are unstable during the training process
• pre-train the segmentation network using onl the pixel-wise loss $J_s$
  • faster
  • do not train critic network
• Adam optimizer with learning rate 0.0002 to train all models for 350 epochs
• mini-batch size : 10
• with critic network : perform 5 optimization steps on the segmentation for each optimization steps on the critic network ( 5次segementation，1次critic）
• evaluation: IoU (Intersection-over-Union)
  • $P$: the set of pixels in the predicted segmentation mask for a class
  • $G$: the set of pixels in the ground truth mask for a class
  • $IoU=\frac{|P \concave G|}{|P \convex G|} = \frac{|TP|}{|TP|+|FP|+|FN|}$
  • Dice Coefficient: $ \frac{2|P \concave G|}{|P + G|} = \frac{2|TP|}{2|TP|+|FP|+|FN|}$

Experiment Design and Result

Design

• JSRT
  • development set: 209 images (randomly)
  • evaluation set: 38 images
  • tune hyperparameters (such as $\lambda$ in Eq.(1)) using a validation set within development set
• Montgomery
  • development set: 117 images(randomly)
  • evaluation set: 21 images
  • use the same hyperparameters tuned in JSRT

Performance

1. Compare FCN with SCAN on JSRT
2. Compare to existing methods on JSRT

• current state-of-the-art: registration-based

3. on Different dataset（迁移性）

• different population
• train on the full JSRT and test on the full montgomery
• 单纯使用FCN数据集迁移性不佳

4. time efficiency

Reference

• 简单理解与实验生成对抗网络GAN
• 生成式模型 & 生成对抗网络——资料梳理（专访资料 + 论文分类）
• 图像语义分割之FCN和CRF
• wikipedia: minimax
• I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D.Warde-Farley, S. Ozair, A. Courville, and Y. Bengio. Gen-erative adversarial nets. In Advances in Neural Information Processing Systems, pages 2672–2680, 2014. 3, 4
• J. Long, E. Shelhamer, and T. Darrell. Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3431–3440, 2015. 3, 4, 5

[Kinpzz]

数学理论参考

• 统计学习精要（The Element of Statistical Learning）