mmagic.evaluation.metrics.fid

Module Contents

Classes

FrechetInceptionDistance	FID metric. In this metric, we calculate the distance between real
TransFID	FID metric. In this metric, we calculate the distance between real

class mmagic.evaluation.metrics.fid.FrechetInceptionDistance(fake_nums: int, real_nums: int = - 1, inception_style='StyleGAN', inception_path: Optional[str] = None, inception_pkl: Optional[str] = None, fake_key: Optional[str] = None, real_key: Optional[str] = 'gt_img', need_cond_input: bool = False, sample_model: str = 'orig', collect_device: str = 'cpu', prefix: Optional[str] = None, sample_kwargs: dict = dict())

Bases: mmagic.evaluation.metrics.base_gen_metric.GenerativeMetric

FID metric. In this metric, we calculate the distance between real distributions and fake distributions. The distributions are modeled by the real samples and fake samples, respectively. Inception_v3 is adopted as the feature extractor, which is widely used in StyleGAN and BigGAN.

Parameters

• fake_nums (int) – Numbers of the generated image need for the metric.

• real_nums (int) – Numbers of the real images need for the metric. If -1 is passed, means all real images in the dataset will be used. Defaults to -1.

• inception_style (str) – The target inception style want to load. If the given style cannot be loaded successful, will attempt to load a valid one. Defaults to ‘StyleGAN’.

• inception_path (str, optional) – Path the the pretrain Inception network. Defaults to None.

• inception_pkl (str, optional) – Path to reference inception pickle file. If None, the statistical value of real distribution will be calculated at running time. Defaults to None.

• fake_key (Optional[str]) – Key for get fake images of the output dict. Defaults to None.

• real_key (Optional[str]) – Key for get real images from the input dict. Defaults to ‘img’.

• need_cond_input (bool) – If true, the sampler will return the conditional input randomly sampled from the original dataset. This require the dataset implement get_data_info and field gt_label must be contained in the return value of get_data_info. Noted that, for unconditional models, set need_cond_input as True may influence the result of evaluation results since the conditional inputs are sampled from the dataset distribution; otherwise will be sampled from the uniform distribution. Defaults to False.

• sample_model (str) – Sampling mode for the generative model. Support ‘orig’ and ‘ema’. Defaults to ‘orig’.

• collect_device (str, optional) – Device name used for collecting results from different ranks during distributed training. Must be ‘cpu’ or ‘gpu’. Defaults to ‘cpu’.

• prefix (str, optional) – The prefix that will be added in the metric names to disambiguate homonymous metrics of different evaluators. If prefix is not provided in the argument, self.default_prefix will be used instead. Defaults to None.

name = FID

prepare(module: torch.nn.Module, dataloader: torch.utils.data.dataloader.DataLoader) → None

Preparing inception feature for the real images.

Parameters

• module (nn.Module) – The model to evaluate.

• dataloader (DataLoader) – The dataloader for real images.

_load_inception(inception_style: str, inception_path: Optional[str]) → Tuple[torch.nn.Module, str]

Load inception and return the successful loaded style.

Parameters

• inception_style (str) – Target style of Inception network want to load.

• inception_path (Optional[str]) – The path to the inception.

Returns

The actually loaded inception network and

corresponding style.

Return type

Tuple[nn.Module, str]

forward_inception(image: torch.Tensor) → torch.Tensor

Feed image to inception network and get the output feature.

Parameters

data_samples (Sequence[dict]) – A batch of data sample dict used to extract inception feature.

Returns

Image feature extracted from inception.

Return type

Tensor

process(data_batch: dict, data_samples: Sequence[dict]) → None

Process one batch of data samples and predictions. The processed results should be stored in self.fake_results, which will be used to compute the metrics when all batches have been processed.

Parameters

• data_batch (dict) – A batch of data from the dataloader.

• data_samples (Sequence[dict]) – A batch of outputs from the model.

static _calc_fid(sample_mean: numpy.ndarray, sample_cov: numpy.ndarray, real_mean: numpy.ndarray, real_cov: numpy.ndarray, eps: float = 1e-06) → Tuple[float]

Refer to the implementation from:

https://github.com/rosinality/stylegan2-pytorch/blob/master/fid.py#L34

compute_metrics(fake_results: list) → dict

Compute the result of FID metric.

Parameters

fake_results (list) – List of image feature of fake images.

Returns

A dict of the computed FID metric and its mean and

covariance.

Return type

dict

class mmagic.evaluation.metrics.fid.TransFID(fake_nums: int, real_nums: int = - 1, inception_style='StyleGAN', inception_path: Optional[str] = None, inception_pkl: Optional[str] = None, fake_key: Optional[str] = None, real_key: Optional[str] = 'img', sample_model: str = 'ema', collect_device: str = 'cpu', prefix: Optional[str] = None)

Bases: FrechetInceptionDistance

FID metric. In this metric, we calculate the distance between real distributions and fake distributions. The distributions are modeled by the real samples and fake samples, respectively. Inception_v3 is adopted as the feature extractor, which is widely used in StyleGAN and BigGAN.

Parameters

• fake_nums (int) – Numbers of the generated image need for the metric.

• real_nums (int) – Numbers of the real images need for the metric. If -1 is passed, means all real images in the dataset will be used. Defaults to -1.

• inception_style (str) – The target inception style want to load. If the given style cannot be loaded successful, will attempt to load a valid one. Defaults to ‘StyleGAN’.

• inception_path (str, optional) – Path the the pretrain Inception network. Defaults to None.

• inception_pkl (str, optional) – Path to reference inception pickle file. If None, the statistical value of real distribution will be calculated at running time. Defaults to None.

• fake_key (Optional[str]) – Key for get fake images of the output dict. Defaults to None.

• real_key (Optional[str]) – Key for get real images from the input dict. Defaults to ‘img’.

• need_cond_input (bool) – If true, the sampler will return the conditional input randomly sampled from the original dataset. This require the dataset implement get_data_info and field gt_label must be contained in the return value of get_data_info. Noted that, for unconditional models, set need_cond_input as True may influence the result of evaluation results since the conditional inputs are sampled from the dataset distribution; otherwise will be sampled from the uniform distribution. Defaults to False.

• sample_model (str) – Sampling mode for the generative model. Support ‘orig’ and ‘ema’. Defaults to ‘orig’.

• collect_device (str, optional) – Device name used for collecting results from different ranks during distributed training. Must be ‘cpu’ or ‘gpu’. Defaults to ‘cpu’.

• prefix (str, optional) – The prefix that will be added in the metric names to disambiguate homonymous metrics of different evaluators. If prefix is not provided in the argument, self.default_prefix will be used instead. Defaults to None.

get_metric_sampler(model: torch.nn.Module, dataloader: torch.utils.data.dataloader.DataLoader, metrics: List[mmagic.evaluation.metrics.base_gen_metric.GenerativeMetric]) → torch.utils.data.dataloader.DataLoader

Get sampler for normal metrics. Directly returns the dataloader.

Parameters

• model (nn.Module) – Model to evaluate.

• dataloader (DataLoader) – Dataloader for real images.

• metrics (List['GenMetric']) – Metrics with the same sample mode.

Returns

Default sampler for normal metrics.

Return type

DataLoader