mmagic.models.editors.eg3d.eg3d

Module Contents

Classes

EG3D	Implementation of `Efficient Geometry-aware 3D Generative Adversarial

Attributes

ModelType

mmagic.models.editors.eg3d.eg3d.ModelType

class mmagic.models.editors.eg3d.eg3d.EG3D(generator: ModelType, discriminator: Optional[ModelType] = None, camera: Optional[ModelType] = None, data_preprocessor: Optional[Union[dict, mmengine.Config]] = None, generator_steps: int = 1, discriminator_steps: int = 1, noise_size: Optional[int] = None, ema_config: Optional[Dict] = None, loss_config: Optional[Dict] = None)

Bases: mmagic.models.base_models.BaseConditionalGAN

Implementation of Efficient Geometry-aware 3D Generative Adversarial Networks

<https://openaccess.thecvf.com/content/CVPR2022/papers/Chan_Efficient_Geometry-Aware_3D_Generative_Adversarial_Networks_CVPR_2022_paper.pdf>_ (EG3D). # noqa

Detailed architecture can be found in TriplaneGenerator and DualDiscriminator

Parameters

• generator (ModelType) – The config or model of the generator.

• discriminator (Optional[ModelType]) – The config or model of the discriminator. Defaults to None.

• camera (Optional[ModelType]) – The pre-defined camera to sample random camera position. If you want to generate images or videos via high-level API, you must set this argument. Defaults to None.

• data_preprocessor (Optional[Union[dict, Config]]) – The pre-process config or DataPreprocessor.

• generator_steps (int) – Number of times the generator was completely updated before the discriminator is updated. Defaults to 1.

• discriminator_steps (int) – Number of times the discriminator was completely updated before the generator is updated. Defaults to 1.

• noise_size (Optional[int]) – Size of the input noise vector. Default to 128.

• num_classes (Optional[int]) – The number classes you would like to generate. Defaults to None.

• ema_config (Optional[Dict]) – The config for generator’s exponential moving average setting. Defaults to None.

• loss_config (Optional[Dict]) – The config for training losses. Defaults to None.

label_fn(label: Optional[torch.Tensor] = None, num_batches: int = 1) → torch.Tensor

Label sampling function for EG3D model.

Parameters

label (Optional[Tensor]) – Conditional for EG3D model. If not passed, self.camera will be used to sample random camera-to-world and intrinsics matrix. Defaults to None.

Returns

Conditional input for EG3D model.

Return type

torch.Tensor

data_sample_to_label(data_sample: mmagic.utils.typing.SampleList) → Optional[torch.Tensor]

Get labels from input data_sample and pack to torch.Tensor. If no label is found in the passed data_sample, None would be returned.

Parameters

data_sample (List[DataSample]) – Input data samples.

Returns

Packed label tensor.

Return type

Optional[torch.Tensor]

pack_to_data_sample(output: Dict[str, torch.Tensor], data_sample: Optional[mmagic.structures.DataSample] = None) → mmagic.structures.DataSample

Pack output to data sample. If data_sample is not passed, a new DataSample will be instantiated. Otherwise, outputs will be added to the passed datasample.

Parameters

• output (Dict[Tensor]) – Output of the model.

• index (int) – The index to save.

• data_sample (DataSample, optional) – Data sample to save outputs. Defaults to None.

Returns

Data sample with packed outputs.

Return type

DataSample

forward(inputs: mmagic.utils.typing.ForwardInputs, data_samples: Optional[list] = None, mode: Optional[str] = None) → List[mmagic.structures.DataSample]

Sample images with the given inputs. If forward mode is ‘ema’ or ‘orig’, the image generated by corresponding generator will be returned. If forward mode is ‘ema/orig’, images generated by original generator and EMA generator will both be returned in a dict.

Parameters

• inputs (ForwardInputs) – Dict containing the necessary information (e.g. noise, num_batches, mode) to generate image.

• data_samples (Optional[list]) – Data samples collated by data_preprocessor. Defaults to None.

• mode (Optional[str]) – mode is not used in BaseConditionalGAN. Defaults to None.

Returns

Generated images or image dict.

Return type

List[DataSample]

interpolation(num_images: int, num_batches: int = 4, mode: str = 'both', sample_model: str = 'orig', show_pbar: bool = True) → List[dict]

Interpolation input and return a list of output results. We support three kinds of interpolation mode:

• ‘camera’: First generate style code with random noise and forward

  camera. Then synthesis images with interpolated camera position and fixed style code.

• ‘conditioning’: First generate style code with fixed noise and

  interpolated camera. Then synthesis images with style codes and forward camera.

• ‘both’: Generate images with interpolated camera position.

Parameters

• num_images (int) – The number of images want to generate.

• num_batches (int, optional) – The number of batches to generate at one time. Defaults to 4.

• mode (str, optional) – The interpolation mode. Supported choices are ‘both’, ‘camera’, and ‘conditioning’. Defaults to ‘both’.

• sample_model (str, optional) – The model used to generate images, support ‘orig’ and ‘ema’. Defaults to ‘orig’.

• show_pbar (bool, optional) – Whether display a progress bar during interpolation. Defaults to True.

Returns

The list of output dict of each frame.

Return type

List[dict]