mmagic.apis.inferencers

Package Contents

Classes

ColorizationInferencer	inferencer that predicts with colorization models.
ConditionalInferencer	inferencer that predicts with conditional models.
ControlnetAnimationInferencer	Base inferencer.
DiffusersPipelineInferencer	inferencer that predicts with text2image models.
EG3DInferencer	Base inferencer.
ImageSuperResolutionInferencer	inferencer that predicts with restoration models.
InpaintingInferencer	inferencer that predicts with inpainting models.
MattingInferencer	inferencer that predicts with matting models.
Text2ImageInferencer	inferencer that predicts with text2image models.
TranslationInferencer	inferencer that predicts with translation models.
UnconditionalInferencer	inferencer that predicts with unconditional models.
VideoInterpolationInferencer	inferencer that predicts with video interpolation models.
VideoRestorationInferencer	inferencer that predicts with video restoration models.

class mmagic.apis.inferencers.ColorizationInferencer(config: Union[mmagic.utils.ConfigType, str], ckpt: Optional[str], device: Optional[str] = None, extra_parameters: Optional[Dict] = None, seed: int = 2022, **kwargs)

Bases: mmagic.apis.inferencers.base_mmagic_inferencer.BaseMMagicInferencer

inferencer that predicts with colorization models.

func_kwargs

preprocess(img: mmagic.apis.inferencers.base_mmagic_inferencer.InputsType) → Dict

Process the inputs into a model-feedable format.

参数

img (InputsType) – Image to be translated by models.

返回

Results of preprocess.

返回类型

results(Dict)

forward(inputs: mmagic.apis.inferencers.base_mmagic_inferencer.InputsType) → mmagic.apis.inferencers.base_mmagic_inferencer.PredType

Forward the inputs to the model.

visualize(preds: mmagic.apis.inferencers.base_mmagic_inferencer.PredType, result_out_dir: str = None) → List[numpy.ndarray]

Visualize predictions.

参数

• preds (List[Union[str, np.ndarray]]) – Forward results by the inferencer.

• data (List[Dict]) – Not needed by this kind of inferencer.

• result_out_dir (str) – Output directory of image. Defaults to ‘’.

返回

Result of visualize

返回类型

List[np.ndarray]

class mmagic.apis.inferencers.ConditionalInferencer(config: Union[mmagic.utils.ConfigType, str], ckpt: Optional[str], device: Optional[str] = None, extra_parameters: Optional[Dict] = None, seed: int = 2022, **kwargs)

Bases: mmagic.apis.inferencers.base_mmagic_inferencer.BaseMMagicInferencer

inferencer that predicts with conditional models.

func_kwargs

extra_parameters

preprocess(label: mmagic.apis.inferencers.base_mmagic_inferencer.InputsType) → Dict

Process the inputs into a model-feedable format.

参数

label (InputsType) – Input label for condition models.

返回

Results of preprocess.

返回类型

results(Dict)

forward(inputs: mmagic.apis.inferencers.base_mmagic_inferencer.InputsType) → mmagic.apis.inferencers.base_mmagic_inferencer.PredType

Forward the inputs to the model.

visualize(preds: mmagic.apis.inferencers.base_mmagic_inferencer.PredType, result_out_dir: str = None) → List[numpy.ndarray]

Visualize predictions.

参数

• preds (List[Union[str, np.ndarray]]) – Forward results by the inferencer.

• data (List[Dict]) – Not needed by this kind of inferencer.

• result_out_dir (str) – Output directory of image. Defaults to ‘’.

返回

Result of visualize

返回类型

List[np.ndarray]

_pred2dict(data_sample: mmagic.structures.DataSample) → Dict

Extract elements necessary to represent a prediction into a dictionary. It’s better to contain only basic data elements such as strings and numbers in order to guarantee it’s json-serializable.

参数

data_sample (DataSample) – The data sample to be converted.

返回

The output dictionary.

返回类型

dict

class mmagic.apis.inferencers.ControlnetAnimationInferencer(config: Union[mmagic.utils.ConfigType, str], device: Optional[str] = None, extra_parameters: Optional[Dict] = None, dtype=torch.float32, **kwargs)

Bases: mmagic.apis.inferencers.base_mmagic_inferencer.BaseMMagicInferencer

Base inferencer.

参数

• config (str or ConfigType) – Model config or the path to it.

• ckpt (str, optional) – Path to the checkpoint.

• device (str, optional) – Device to run inference. If None, the best device will be automatically used.

• result_out_dir (str) – Output directory of images. Defaults to ‘’.

func_kwargs

func_order

extra_parameters

__call__(prompt=None, video=None, negative_prompt=None, controlnet_conditioning_scale=0.7, image_width=512, image_height=512, save_path=None, strength=0.75, num_inference_steps=20, seed=1, output_fps=None, reference_img=None, **kwargs) → Union[Dict, List[Dict]]

Call the inferencer.

参数

kwargs – Keyword arguments for the inferencer.

返回

Results of inference pipeline.

返回类型

Union[Dict, List[Dict]]

class mmagic.apis.inferencers.DiffusersPipelineInferencer(config: Union[mmagic.utils.ConfigType, str], ckpt: Optional[str], device: Optional[str] = None, extra_parameters: Optional[Dict] = None, seed: int = 2022, **kwargs)

Bases: mmagic.apis.inferencers.base_mmagic_inferencer.BaseMMagicInferencer

inferencer that predicts with text2image models.

func_kwargs

preprocess(text: mmagic.apis.inferencers.base_mmagic_inferencer.InputsType = None, negative_prompt: mmagic.apis.inferencers.base_mmagic_inferencer.InputsType = None, num_inference_steps: int = 20, height=None, width=None) → Dict

Process the inputs into a model-feedable format.

参数

• text (InputsType) – text input for text-to-image model.

• negative_prompt (InputsType) – negative prompt.

返回

Results of preprocess.

返回类型

result(Dict)

forward(inputs: mmagic.apis.inferencers.base_mmagic_inferencer.InputsType) → mmagic.apis.inferencers.base_mmagic_inferencer.PredType

Forward the inputs to the model.

visualize(preds: mmagic.apis.inferencers.base_mmagic_inferencer.PredType, result_out_dir: str = None) → List[numpy.ndarray]

Visualize predictions.

参数

• preds (List[Union[str, np.ndarray]]) – Forward results by the inferencer.

• result_out_dir (str) – Output directory of image. Defaults to ‘’.

返回

Result of visualize

返回类型

List[np.ndarray]

class mmagic.apis.inferencers.EG3DInferencer(config: Union[mmagic.utils.ConfigType, str], ckpt: Optional[str], device: Optional[str] = None, extra_parameters: Optional[Dict] = None, seed: int = 2022, **kwargs)

Bases: mmagic.apis.inferencers.base_mmagic_inferencer.BaseMMagicInferencer

Base inferencer.

参数

• config (str or ConfigType) – Model config or the path to it.

• ckpt (str, optional) – Path to the checkpoint.

• device (str, optional) – Device to run inference. If None, the best device will be automatically used.

• extra_parameters (Dict, optional) – Extra parameters for different models in inference stage.

• seed (str, optional) – Seed for inference.

func_kwargs

extra_parameters

preprocess(inputs: mmagic.apis.inferencers.base_mmagic_inferencer.InputsType = None) → mmagic.utils.ForwardInputs

Process the inputs into a model-feedable format.

参数

inputs (List[Union[str, np.ndarray]]) – The conditional inputs for the inferencer. Defaults to None.

返回

The preprocessed inputs and data samples.

返回类型

ForwardInputs

forward(inputs: mmagic.utils.ForwardInputs, interpolation: Optional[str] = 'both', num_images: int = 100) → Union[dict, List[dict]]

Forward the inputs to the model.

参数

• inputs (ForwardInputs) – Model inputs. If data sample (the second element of inputs) is not passed, will generate a sequence of images corresponding to passed interpolation mode.

• interpolation (str) – The interpolation mode. Supported choices are ‘both’, ‘conditioning’, and ‘camera’. Defaults to ‘both’.

• num_images (int) – The number of frames of interpolation. Defaults to 500.

返回

Output dict corresponds to the input

condition or the list of output dict of each frame during the interpolation process.

返回类型

Union[dict, List[dict]]

visualize(preds: Union[mmagic.apis.inferencers.base_mmagic_inferencer.PredType, List[mmagic.apis.inferencers.base_mmagic_inferencer.PredType]], vis_mode: str = 'both', save_img: bool = True, save_video: bool = True, img_suffix: str = '.png', video_suffix: str = '.mp4', result_out_dir: str = 'eg3d_output') → None

Visualize predictions.

参数

• preds (Union[PredType, List[PredType]]) – Prediction os model.

• vis_mode (str, optional) – Which output to visualize. Supported choices are ‘both’, ‘depth’, and ‘img’. Defaults to ‘all’.

• save_img (bool, optional) – Whether save images. Defaults to True.

• save_video (bool, optional) – Whether save videos. Defaults to True.

• img_suffix (str, optional) – The suffix of saved images. Defaults to ‘.png’.

• video_suffix (str, optional) – The suffix of saved videos. Defaults to ‘.mp4’.

• result_out_dir (str, optional) – The save director of image and videos. Defaults to ‘eg3d_output’.

preprocess_img(preds: List[dict]) → torch.Tensor

Preprocess images in the predictions.

参数

preds (List[dict]) – List of prediction dict of each frame.

返回

Preprocessed image tensor shape like

[num_frame * bz, 3, H, W].

返回类型

torch.Tensor

preprocess_depth(preds: List[dict]) → torch.Tensor

Preprocess depth in the predictions.

参数

preds (List[dict]) – List of prediction dict of each frame.

返回

Preprocessed depth tensor shape like

[num_frame * bz, 3, H, W].

返回类型

torch.Tensor

postprocess(preds: mmagic.apis.inferencers.base_mmagic_inferencer.PredType, imgs: Optional[List[numpy.ndarray]] = None, is_batch: bool = False, get_datasample: bool = False) → Dict[str, torch.tensor]

Postprocess predictions.

参数

• preds (List[Dict]) – Predictions of the model.

• imgs (Optional[np.ndarray]) – Visualized predictions.

• is_batch (bool) – Whether the inputs are in a batch. Defaults to False.

• get_datasample (bool) – Whether to use Datasample to store inference results. If False, dict will be used.

返回

Inference results as a dict.

返回类型

Dict[str, torch.Tensor]

class mmagic.apis.inferencers.ImageSuperResolutionInferencer(config: Union[mmagic.utils.ConfigType, str], ckpt: Optional[str], device: Optional[str] = None, extra_parameters: Optional[Dict] = None, seed: int = 2022, **kwargs)

Bases: mmagic.apis.inferencers.base_mmagic_inferencer.BaseMMagicInferencer

inferencer that predicts with restoration models.

func_kwargs

preprocess(img: mmagic.apis.inferencers.base_mmagic_inferencer.InputsType, ref: mmagic.apis.inferencers.base_mmagic_inferencer.InputsType = None) → Dict

Process the inputs into a model-feedable format.

参数

• img (InputsType) – Image to be restored by models.

• ref (InputsType) – Reference image for restoration models. Defaults to None.

返回

Results of preprocess.

返回类型

data(Dict)

forward(inputs: mmagic.apis.inferencers.base_mmagic_inferencer.InputsType) → mmagic.apis.inferencers.base_mmagic_inferencer.PredType

Forward the inputs to the model.

visualize(preds: mmagic.apis.inferencers.base_mmagic_inferencer.PredType, result_out_dir: str = None) → List[numpy.ndarray]

Visualize predictions.

参数

• preds (List[Union[str, np.ndarray]]) – Forward results by the inferencer.

• data (List[Dict]) – Not needed by this kind of inferencer.

• result_out_dir (str) – Output directory of image. Defaults to ‘’.

返回

Result of visualize

返回类型

List[np.ndarray]

class mmagic.apis.inferencers.InpaintingInferencer(config: Union[mmagic.utils.ConfigType, str], ckpt: Optional[str], device: Optional[str] = None, extra_parameters: Optional[Dict] = None, seed: int = 2022, **kwargs)

Bases: mmagic.apis.inferencers.base_mmagic_inferencer.BaseMMagicInferencer

inferencer that predicts with inpainting models.

func_kwargs

_init_pipeline(cfg) → mmengine.dataset.Compose

Initialize the test pipeline.

preprocess(img: mmagic.apis.inferencers.base_mmagic_inferencer.InputsType, mask: mmagic.apis.inferencers.base_mmagic_inferencer.InputsType) → Dict

Process the inputs into a model-feedable format.

参数

• img (InputsType) – Image to be inpainted by models.

• mask (InputsType) – Image mask for inpainting models.

返回

Results of preprocess.

返回类型

results(Dict)

forward(inputs: mmagic.apis.inferencers.base_mmagic_inferencer.InputsType) → mmagic.apis.inferencers.base_mmagic_inferencer.PredType

Forward the inputs to the model.

visualize(preds: mmagic.apis.inferencers.base_mmagic_inferencer.PredType, result_out_dir: str = None) → List[numpy.ndarray]

Visualize predictions.

参数

• preds (List[Union[str, np.ndarray]]) – Forward results by the inferencer.

• data (List[Dict]) – Mask of input image.

• result_out_dir (str) – Output directory of image. Defaults to ‘’.

返回

Result of visualize

返回类型

List[np.ndarray]

class mmagic.apis.inferencers.MattingInferencer(config: Union[mmagic.utils.ConfigType, str], ckpt: Optional[str], device: Optional[str] = None, extra_parameters: Optional[Dict] = None, seed: int = 2022, **kwargs)

Bases: mmagic.apis.inferencers.base_mmagic_inferencer.BaseMMagicInferencer

inferencer that predicts with matting models.

func_kwargs

preprocess(img: mmagic.apis.inferencers.base_mmagic_inferencer.InputsType, trimap: mmagic.apis.inferencers.base_mmagic_inferencer.InputsType) → Dict

Process the inputs into a model-feedable format.

参数

• img (InputsType) – Image to be processed by models.

• mask (InputsType) – Mask corresponding to the input image.

返回

Results of preprocess.

返回类型

results(Dict)

forward(inputs: mmagic.apis.inferencers.base_mmagic_inferencer.InputsType) → mmagic.apis.inferencers.base_mmagic_inferencer.PredType

Forward the inputs to the model.

visualize(preds: mmagic.apis.inferencers.base_mmagic_inferencer.PredType, result_out_dir: str = None) → List[numpy.ndarray]

Visualize predictions.

参数

• preds (List[Union[str, np.ndarray]]) – Forward results by the inferencer.

• data (List[Dict]) – Not needed by this kind of inferencer.

• result_out_dir (str) – Output directory of image. Defaults to ‘’.

返回

Result of visualize

返回类型

List[np.ndarray]

_pred2dict(data_sample: mmagic.structures.DataSample) → Dict

Extract elements necessary to represent a prediction into a dictionary. It’s better to contain only basic data elements such as strings and numbers in order to guarantee it’s json-serializable.

参数

data_sample (DataSample) – The data sample to be converted.

返回

The output dictionary.

返回类型

dict

class mmagic.apis.inferencers.Text2ImageInferencer(config: Union[mmagic.utils.ConfigType, str], ckpt: Optional[str], device: Optional[str] = None, extra_parameters: Optional[Dict] = None, seed: int = 2022, **kwargs)

Bases: mmagic.apis.inferencers.base_mmagic_inferencer.BaseMMagicInferencer

inferencer that predicts with text2image models.

func_kwargs

extra_parameters

preprocess(text: mmagic.apis.inferencers.base_mmagic_inferencer.InputsType, control: str = None, negative_prompt: mmagic.apis.inferencers.base_mmagic_inferencer.InputsType = None) → Dict

Process the inputs into a model-feedable format.

参数

• text (InputsType) – text input for text-to-image model.

• control (str) – control img dir for controlnet.

• negative_prompt (InputsType) – negative prompt.

返回

Results of preprocess.

返回类型

result(Dict)

forward(inputs: mmagic.apis.inferencers.base_mmagic_inferencer.InputsType) → mmagic.apis.inferencers.base_mmagic_inferencer.PredType

Forward the inputs to the model.

visualize(preds: mmagic.apis.inferencers.base_mmagic_inferencer.PredType, result_out_dir: str = None) → List[numpy.ndarray]

Visualize predictions.

参数

• preds (List[Union[str, np.ndarray]]) – Forward results by the inferencer.

• result_out_dir (str) – Output directory of image. Defaults to ‘’.

返回

Result of visualize

返回类型

List[np.ndarray]

class mmagic.apis.inferencers.TranslationInferencer(config: Union[mmagic.utils.ConfigType, str], ckpt: Optional[str], device: Optional[str] = None, extra_parameters: Optional[Dict] = None, seed: int = 2022, **kwargs)

Bases: mmagic.apis.inferencers.base_mmagic_inferencer.BaseMMagicInferencer

inferencer that predicts with translation models.

func_kwargs

preprocess(img: mmagic.apis.inferencers.base_mmagic_inferencer.InputsType) → Dict

Process the inputs into a model-feedable format.

参数

img (InputsType) – Image to be translated by models.

返回

Results of preprocess.

返回类型

results(Dict)

forward(inputs: mmagic.apis.inferencers.base_mmagic_inferencer.InputsType) → mmagic.apis.inferencers.base_mmagic_inferencer.PredType

Forward the inputs to the model.

visualize(preds: mmagic.apis.inferencers.base_mmagic_inferencer.PredType, result_out_dir: str = None) → List[numpy.ndarray]

Visualize predictions.

参数

• preds (List[Union[str, np.ndarray]]) – Forward results by the inferencer.

• data (List[Dict]) – Not needed by this kind of inferencer.

• result_out_dir (str) – Output directory of image. Defaults to ‘’.

返回

Result of visualize

返回类型

List[np.ndarray]

class mmagic.apis.inferencers.UnconditionalInferencer(config: Union[mmagic.utils.ConfigType, str], ckpt: Optional[str], device: Optional[str] = None, extra_parameters: Optional[Dict] = None, seed: int = 2022, **kwargs)

Bases: mmagic.apis.inferencers.base_mmagic_inferencer.BaseMMagicInferencer

inferencer that predicts with unconditional models.

func_kwargs

extra_parameters

preprocess() → Dict

Process the inputs into a model-feedable format.

返回

Results of preprocess.

返回类型

results(Dict)

forward(inputs: mmagic.apis.inferencers.base_mmagic_inferencer.InputsType) → mmagic.apis.inferencers.base_mmagic_inferencer.PredType

Forward the inputs to the model.

visualize(preds: mmagic.apis.inferencers.base_mmagic_inferencer.PredType, result_out_dir: str = '') → List[numpy.ndarray]

Visualize predictions.

参数

• preds (List[Union[str, np.ndarray]]) – Forward results by the inferencer.

• data (List[Dict]) – Not needed by this kind of inferencer.

• result_out_dir (str) – Output directory of image. Defaults to ‘’.

返回

Result of visualize

返回类型

List[np.ndarray]

_pred2dict(data_sample: mmagic.structures.DataSample) → Dict

Extract elements necessary to represent a prediction into a dictionary. It’s better to contain only basic data elements such as strings and numbers in order to guarantee it’s json-serializable.

参数

data_sample (DataSample) – The data sample to be converted.

返回

The output dictionary.

返回类型

dict

class mmagic.apis.inferencers.VideoInterpolationInferencer(config: Union[mmagic.utils.ConfigType, str], ckpt: Optional[str], device: Optional[str] = None, extra_parameters: Optional[Dict] = None, seed: int = 2022, **kwargs)

Bases: mmagic.apis.inferencers.base_mmagic_inferencer.BaseMMagicInferencer

inferencer that predicts with video interpolation models.

func_kwargs

extra_parameters

preprocess(video: mmagic.apis.inferencers.base_mmagic_inferencer.InputsType) → Dict

Process the inputs into a model-feedable format.

参数

video (InputsType) – Video to be interpolated by models.

返回

Video to be interpolated by models.

返回类型

video(InputsType)

forward(inputs: mmagic.apis.inferencers.base_mmagic_inferencer.InputsType, result_out_dir: mmagic.apis.inferencers.base_mmagic_inferencer.InputsType = '') → mmagic.apis.inferencers.base_mmagic_inferencer.PredType

Forward the inputs to the model.

参数

• inputs (InputsType) – Input video directory.

• result_out_dir (str) – Output directory of video. Defaults to ‘’.

返回

Result of forwarding

返回类型

PredType

visualize(preds: mmagic.apis.inferencers.base_mmagic_inferencer.PredType, result_out_dir: str = '') → List[numpy.ndarray]

Visualize is not needed in this inferencer.

postprocess(preds: mmagic.apis.inferencers.base_mmagic_inferencer.PredType, imgs: Optional[List[numpy.ndarray]] = None) → Union[mmagic.apis.inferencers.base_mmagic_inferencer.ResType, Tuple[mmagic.apis.inferencers.base_mmagic_inferencer.ResType, numpy.ndarray]]

Postprocess is not needed in this inferencer.

class mmagic.apis.inferencers.VideoRestorationInferencer(config: Union[mmagic.utils.ConfigType, str], ckpt: Optional[str], device: Optional[str] = None, extra_parameters: Optional[Dict] = None, seed: int = 2022, **kwargs)

Bases: mmagic.apis.inferencers.base_mmagic_inferencer.BaseMMagicInferencer

inferencer that predicts with video restoration models.

func_kwargs

extra_parameters

preprocess(video: mmagic.apis.inferencers.base_mmagic_inferencer.InputsType) → Dict

Process the inputs into a model-feedable format.

参数

video (InputsType) – Video to be restored by models.

返回

Results of preprocess.

返回类型

results(InputsType)

forward(inputs: mmagic.apis.inferencers.base_mmagic_inferencer.InputsType) → mmagic.apis.inferencers.base_mmagic_inferencer.PredType

Forward the inputs to the model.

参数

inputs (InputsType) – Images array of input video.

返回

Results of forwarding

返回类型

PredType

visualize(preds: mmagic.apis.inferencers.base_mmagic_inferencer.PredType, result_out_dir: str = '') → List[numpy.ndarray]

Visualize predictions.

参数

• preds (List[Union[str, np.ndarray]]) – Forward results by the inferencer.

• data (List[Dict]) – Not needed by this kind of inferencer.

• result_out_dir (str) – Output directory of image. Defaults to ‘’.

返回

Result of visualize

返回类型

List[np.ndarray]

postprocess(preds: mmagic.apis.inferencers.base_mmagic_inferencer.PredType, imgs: Optional[List[numpy.ndarray]] = None) → Union[mmagic.apis.inferencers.base_mmagic_inferencer.ResType, Tuple[mmagic.apis.inferencers.base_mmagic_inferencer.ResType, numpy.ndarray]]

Postprocess is not needed in this inferencer.