`mmagic.apis.inferencers`¶

Package Contents¶

Classes¶

`ColorizationInferencer`	inferencer that predicts with colorization models.
`ConditionalInferencer`	inferencer that predicts with conditional models.
`ControlnetAnimationInferencer`	Base inferencer.
`DiffusersPipelineInferencer`	inferencer that predicts with text2image models.
`EG3DInferencer`	Base inferencer.
`ImageSuperResolutionInferencer`	inferencer that predicts with restoration models.
`InpaintingInferencer`	inferencer that predicts with inpainting models.
`MattingInferencer`	inferencer that predicts with matting models.
`Text2ImageInferencer`	inferencer that predicts with text2image models.
`TranslationInferencer`	inferencer that predicts with translation models.
`UnconditionalInferencer`	inferencer that predicts with unconditional models.
`VideoInterpolationInferencer`	inferencer that predicts with video interpolation models.
`VideoRestorationInferencer`	inferencer that predicts with video restoration models.

class mmagic.apis.inferencers.ColorizationInferencer(config: Union[mmagic.utils.ConfigType, str], ckpt: Optional[str], device: Optional[str] = None, extra_parameters: Optional[Dict] = None, seed: int = 2022, **kwargs)[source]¶

Bases: mmagic.apis.inferencers.base_mmagic_inferencer.BaseMMagicInferencer

inferencer that predicts with colorization models.

func_kwargs¶

preprocess(img: mmagic.apis.inferencers.base_mmagic_inferencer.InputsType) → Dict[source]¶

Process the inputs into a model-feedable format.

Parameters: img (InputsType) – Image to be translated by models.
Returns: Results of preprocess.
Return type: results(Dict)

forward(inputs: mmagic.apis.inferencers.base_mmagic_inferencer.InputsType) → mmagic.apis.inferencers.base_mmagic_inferencer.PredType[source]¶: Forward the inputs to the model.

visualize(preds: mmagic.apis.inferencers.base_mmagic_inferencer.PredType, result_out_dir: str = None) → List[numpy.ndarray][source]¶

Visualize predictions.

Parameters

preds (List[Union[str, np.ndarray]]) – Forward results by the inferencer.
data (List[Dict]) – Not needed by this kind of inferencer.
result_out_dir (str) – Output directory of image. Defaults to ‘’.

Returns

Result of visualize

Return type

List[np.ndarray]

class mmagic.apis.inferencers.ConditionalInferencer(config: Union[mmagic.utils.ConfigType, str], ckpt: Optional[str], device: Optional[str] = None, extra_parameters: Optional[Dict] = None, seed: int = 2022, **kwargs)[source]¶

Bases: mmagic.apis.inferencers.base_mmagic_inferencer.BaseMMagicInferencer

inferencer that predicts with conditional models.

func_kwargs¶

extra_parameters¶

preprocess(label: mmagic.apis.inferencers.base_mmagic_inferencer.InputsType) → Dict[source]¶

Process the inputs into a model-feedable format.

Parameters: label (InputsType) – Input label for condition models.
Returns: Results of preprocess.
Return type: results(Dict)

forward(inputs: mmagic.apis.inferencers.base_mmagic_inferencer.InputsType) → mmagic.apis.inferencers.base_mmagic_inferencer.PredType[source]¶: Forward the inputs to the model.

visualize(preds: mmagic.apis.inferencers.base_mmagic_inferencer.PredType, result_out_dir: str = None) → List[numpy.ndarray][source]¶

Visualize predictions.

Parameters

preds (List[Union[str, np.ndarray]]) – Forward results by the inferencer.
data (List[Dict]) – Not needed by this kind of inferencer.
result_out_dir (str) – Output directory of image. Defaults to ‘’.

Returns

Result of visualize

Return type

List[np.ndarray]

_pred2dict(data_sample: mmagic.structures.DataSample) → Dict[source]¶

Extract elements necessary to represent a prediction into a dictionary. It’s better to contain only basic data elements such as strings and numbers in order to guarantee it’s json-serializable.

Parameters: data_sample (DataSample) – The data sample to be converted.
Returns: The output dictionary.
Return type: dict

class mmagic.apis.inferencers.ControlnetAnimationInferencer(config: Union[mmagic.utils.ConfigType, str], device: Optional[str] = None, extra_parameters: Optional[Dict] = None, dtype=torch.float32, **kwargs)[source]¶

Bases: mmagic.apis.inferencers.base_mmagic_inferencer.BaseMMagicInferencer

Base inferencer.

Parameters

config (str or ConfigType) – Model config or the path to it.
ckpt (str, optional) – Path to the checkpoint.
device (str, optional) – Device to run inference. If None, the best device will be automatically used.
result_out_dir (str) – Output directory of images. Defaults to ‘’.

func_kwargs¶

func_order¶

extra_parameters¶

__call__(prompt=None, video=None, negative_prompt=None, controlnet_conditioning_scale=0.7, image_width=512, image_height=512, save_path=None, strength=0.75, num_inference_steps=20, seed=1, output_fps=None, reference_img=None, **kwargs) → Union[Dict, List[Dict]][source]¶

Call the inferencer.

Parameters: kwargs – Keyword arguments for the inferencer.
Returns: Results of inference pipeline.
Return type: Union[Dict, List[Dict]]

class mmagic.apis.inferencers.DiffusersPipelineInferencer(config: Union[mmagic.utils.ConfigType, str], ckpt: Optional[str], device: Optional[str] = None, extra_parameters: Optional[Dict] = None, seed: int = 2022, **kwargs)[source]¶

Bases: mmagic.apis.inferencers.base_mmagic_inferencer.BaseMMagicInferencer

inferencer that predicts with text2image models.

func_kwargs¶

preprocess(text: mmagic.apis.inferencers.base_mmagic_inferencer.InputsType = None, negative_prompt: mmagic.apis.inferencers.base_mmagic_inferencer.InputsType = None, num_inference_steps: int = 20, height=None, width=None) → Dict[source]¶

Process the inputs into a model-feedable format.

Parameters

text (InputsType) – text input for text-to-image model.
negative_prompt (InputsType) – negative prompt.

Returns

Results of preprocess.

Return type

result(Dict)

forward(inputs: mmagic.apis.inferencers.base_mmagic_inferencer.InputsType) → mmagic.apis.inferencers.base_mmagic_inferencer.PredType[source]¶: Forward the inputs to the model.

visualize(preds: mmagic.apis.inferencers.base_mmagic_inferencer.PredType, result_out_dir: str = None) → List[numpy.ndarray][source]¶

Visualize predictions.

Parameters

preds (List[Union[str, np.ndarray]]) – Forward results by the inferencer.
result_out_dir (str) – Output directory of image. Defaults to ‘’.

Returns

Result of visualize

Return type

List[np.ndarray]

class mmagic.apis.inferencers.EG3DInferencer(config: Union[mmagic.utils.ConfigType, str], ckpt: Optional[str], device: Optional[str] = None, extra_parameters: Optional[Dict] = None, seed: int = 2022, **kwargs)[source]¶

Bases: mmagic.apis.inferencers.base_mmagic_inferencer.BaseMMagicInferencer

Base inferencer.

Parameters

config (str or ConfigType) – Model config or the path to it.
ckpt (str, optional) – Path to the checkpoint.
device (str, optional) – Device to run inference. If None, the best device will be automatically used.
extra_parameters (Dict, optional) – Extra parameters for different models in inference stage.
seed (str, optional) – Seed for inference.

func_kwargs¶

extra_parameters¶

preprocess(inputs: mmagic.apis.inferencers.base_mmagic_inferencer.InputsType = None) → mmagic.utils.ForwardInputs[source]¶

Process the inputs into a model-feedable format.

Parameters: inputs (List[Union[str, np.ndarray]]) – The conditional inputs for the inferencer. Defaults to None.
Returns: The preprocessed inputs and data samples.
Return type: ForwardInputs

forward(inputs: mmagic.utils.ForwardInputs, interpolation: Optional[str] = 'both', num_images: int = 100) → Union[dict, List[dict]][source]¶

Forward the inputs to the model.

Parameters

inputs (ForwardInputs) – Model inputs. If data sample (the second element of inputs) is not passed, will generate a sequence of images corresponding to passed interpolation mode.
interpolation (str) – The interpolation mode. Supported choices are ‘both’, ‘conditioning’, and ‘camera’. Defaults to ‘both’.
num_images (int) – The number of frames of interpolation. Defaults to 500.

Returns

Output dict corresponds to the input: condition or the list of output dict of each frame during the interpolation process.

Return type

Union[dict, List[dict]]

visualize(preds: Union[mmagic.apis.inferencers.base_mmagic_inferencer.PredType, List[mmagic.apis.inferencers.base_mmagic_inferencer.PredType]], vis_mode: str = 'both', save_img: bool = True, save_video: bool = True, img_suffix: str = '.png', video_suffix: str = '.mp4', result_out_dir: str = 'eg3d_output') → None[source]¶

Visualize predictions.

Parameters

preds (Union[PredType, List[PredType]]) – Prediction os model.
vis_mode (str, optional) – Which output to visualize. Supported choices are ‘both’, ‘depth’, and ‘img’. Defaults to ‘all’.
save_img (bool, optional) – Whether save images. Defaults to True.
save_video (bool, optional) – Whether save videos. Defaults to True.
img_suffix (str, optional) – The suffix of saved images. Defaults to ‘.png’.
video_suffix (str, optional) – The suffix of saved videos. Defaults to ‘.mp4’.
result_out_dir (str, optional) – The save director of image and videos. Defaults to ‘eg3d_output’.

preprocess_img(preds: List[dict]) → torch.Tensor[source]¶

Preprocess images in the predictions.

Parameters

preds (List[dict]) – List of prediction dict of each frame.

Returns

Preprocessed image tensor shape like: [num_frame * bz, 3, H, W].

Return type

torch.Tensor

preprocess_depth(preds: List[dict]) → torch.Tensor[source]¶

Preprocess depth in the predictions.

Parameters

preds (List[dict]) – List of prediction dict of each frame.

Returns

Preprocessed depth tensor shape like: [num_frame * bz, 3, H, W].

Return type

torch.Tensor

postprocess(preds: mmagic.apis.inferencers.base_mmagic_inferencer.PredType, imgs: Optional[List[numpy.ndarray]] = None, is_batch: bool = False, get_datasample: bool = False) → Dict[str, torch.tensor][source]¶

Postprocess predictions.

Parameters

preds (List[Dict]) – Predictions of the model.
imgs (Optional[np.ndarray]) – Visualized predictions.
is_batch (bool) – Whether the inputs are in a batch. Defaults to False.
get_datasample (bool) – Whether to use Datasample to store inference results. If False, dict will be used.

Returns

Inference results as a dict.

Return type

Dict[str, torch.Tensor]

class mmagic.apis.inferencers.ImageSuperResolutionInferencer(config: Union[mmagic.utils.ConfigType, str], ckpt: Optional[str], device: Optional[str] = None, extra_parameters: Optional[Dict] = None, seed: int = 2022, **kwargs)[source]¶

Bases: mmagic.apis.inferencers.base_mmagic_inferencer.BaseMMagicInferencer

inferencer that predicts with restoration models.

func_kwargs¶

preprocess(img: mmagic.apis.inferencers.base_mmagic_inferencer.InputsType, ref: mmagic.apis.inferencers.base_mmagic_inferencer.InputsType = None) → Dict[source]¶

Process the inputs into a model-feedable format.

Parameters

img (InputsType) – Image to be restored by models.
ref (InputsType) – Reference image for restoration models. Defaults to None.

Returns

Results of preprocess.

Return type

data(Dict)

forward(inputs: mmagic.apis.inferencers.base_mmagic_inferencer.InputsType) → mmagic.apis.inferencers.base_mmagic_inferencer.PredType[source]¶: Forward the inputs to the model.

visualize(preds: mmagic.apis.inferencers.base_mmagic_inferencer.PredType, result_out_dir: str = None) → List[numpy.ndarray][source]¶

Visualize predictions.

Parameters

preds (List[Union[str, np.ndarray]]) – Forward results by the inferencer.
data (List[Dict]) – Not needed by this kind of inferencer.
result_out_dir (str) – Output directory of image. Defaults to ‘’.

Returns

Result of visualize

Return type

List[np.ndarray]

class mmagic.apis.inferencers.InpaintingInferencer(config: Union[mmagic.utils.ConfigType, str], ckpt: Optional[str], device: Optional[str] = None, extra_parameters: Optional[Dict] = None, seed: int = 2022, **kwargs)[source]¶

Bases: mmagic.apis.inferencers.base_mmagic_inferencer.BaseMMagicInferencer

inferencer that predicts with inpainting models.

func_kwargs¶

_init_pipeline(cfg) → mmengine.dataset.Compose[source]¶: Initialize the test pipeline.

preprocess(img: mmagic.apis.inferencers.base_mmagic_inferencer.InputsType, mask: mmagic.apis.inferencers.base_mmagic_inferencer.InputsType) → Dict[source]¶

Process the inputs into a model-feedable format.

Parameters

img (InputsType) – Image to be inpainted by models.
mask (InputsType) – Image mask for inpainting models.

Returns

Results of preprocess.

Return type

results(Dict)

forward(inputs: mmagic.apis.inferencers.base_mmagic_inferencer.InputsType) → mmagic.apis.inferencers.base_mmagic_inferencer.PredType[source]¶: Forward the inputs to the model.

visualize(preds: mmagic.apis.inferencers.base_mmagic_inferencer.PredType, result_out_dir: str = None) → List[numpy.ndarray][source]¶

Visualize predictions.

Parameters

preds (List[Union[str, np.ndarray]]) – Forward results by the inferencer.
data (List[Dict]) – Mask of input image.
result_out_dir (str) – Output directory of image. Defaults to ‘’.

Returns

Result of visualize

Return type

List[np.ndarray]

class mmagic.apis.inferencers.MattingInferencer(config: Union[mmagic.utils.ConfigType, str], ckpt: Optional[str], device: Optional[str] = None, extra_parameters: Optional[Dict] = None, seed: int = 2022, **kwargs)[source]¶

Bases: mmagic.apis.inferencers.base_mmagic_inferencer.BaseMMagicInferencer

inferencer that predicts with matting models.

func_kwargs¶

preprocess(img: mmagic.apis.inferencers.base_mmagic_inferencer.InputsType, trimap: mmagic.apis.inferencers.base_mmagic_inferencer.InputsType) → Dict[source]¶

Process the inputs into a model-feedable format.

Parameters

img (InputsType) – Image to be processed by models.
mask (InputsType) – Mask corresponding to the input image.

Returns

Results of preprocess.

Return type

results(Dict)

forward(inputs: mmagic.apis.inferencers.base_mmagic_inferencer.InputsType) → mmagic.apis.inferencers.base_mmagic_inferencer.PredType[source]¶: Forward the inputs to the model.

visualize(preds: mmagic.apis.inferencers.base_mmagic_inferencer.PredType, result_out_dir: str = None) → List[numpy.ndarray][source]¶

Visualize predictions.

Parameters

preds (List[Union[str, np.ndarray]]) – Forward results by the inferencer.
data (List[Dict]) – Not needed by this kind of inferencer.
result_out_dir (str) – Output directory of image. Defaults to ‘’.

Returns

Result of visualize

Return type

List[np.ndarray]

_pred2dict(data_sample: mmagic.structures.DataSample) → Dict[source]¶

Extract elements necessary to represent a prediction into a dictionary. It’s better to contain only basic data elements such as strings and numbers in order to guarantee it’s json-serializable.

Parameters: data_sample (DataSample) – The data sample to be converted.
Returns: The output dictionary.
Return type: dict

class mmagic.apis.inferencers.Text2ImageInferencer(config: Union[mmagic.utils.ConfigType, str], ckpt: Optional[str], device: Optional[str] = None, extra_parameters: Optional[Dict] = None, seed: int = 2022, **kwargs)[source]¶

Bases: mmagic.apis.inferencers.base_mmagic_inferencer.BaseMMagicInferencer

inferencer that predicts with text2image models.

func_kwargs¶

extra_parameters¶

preprocess(text: mmagic.apis.inferencers.base_mmagic_inferencer.InputsType, control: str = None, negative_prompt: mmagic.apis.inferencers.base_mmagic_inferencer.InputsType = None) → Dict[source]¶

Process the inputs into a model-feedable format.

Parameters

text (InputsType) – text input for text-to-image model.
control (str) – control img dir for controlnet.
negative_prompt (InputsType) – negative prompt.

Returns

Results of preprocess.

Return type

result(Dict)

forward(inputs: mmagic.apis.inferencers.base_mmagic_inferencer.InputsType) → mmagic.apis.inferencers.base_mmagic_inferencer.PredType[source]¶: Forward the inputs to the model.

visualize(preds: mmagic.apis.inferencers.base_mmagic_inferencer.PredType, result_out_dir: str = None) → List[numpy.ndarray][source]¶

Visualize predictions.

Parameters

preds (List[Union[str, np.ndarray]]) – Forward results by the inferencer.
result_out_dir (str) – Output directory of image. Defaults to ‘’.

Returns

Result of visualize

Return type

List[np.ndarray]

class mmagic.apis.inferencers.TranslationInferencer(config: Union[mmagic.utils.ConfigType, str], ckpt: Optional[str], device: Optional[str] = None, extra_parameters: Optional[Dict] = None, seed: int = 2022, **kwargs)[source]¶

Bases: mmagic.apis.inferencers.base_mmagic_inferencer.BaseMMagicInferencer

inferencer that predicts with translation models.

func_kwargs¶

preprocess(img: mmagic.apis.inferencers.base_mmagic_inferencer.InputsType) → Dict[source]¶

Process the inputs into a model-feedable format.

Parameters: img (InputsType) – Image to be translated by models.
Returns: Results of preprocess.
Return type: results(Dict)

forward(inputs: mmagic.apis.inferencers.base_mmagic_inferencer.InputsType) → mmagic.apis.inferencers.base_mmagic_inferencer.PredType[source]¶: Forward the inputs to the model.

visualize(preds: mmagic.apis.inferencers.base_mmagic_inferencer.PredType, result_out_dir: str = None) → List[numpy.ndarray][source]¶

Visualize predictions.

Parameters

preds (List[Union[str, np.ndarray]]) – Forward results by the inferencer.
data (List[Dict]) – Not needed by this kind of inferencer.
result_out_dir (str) – Output directory of image. Defaults to ‘’.

Returns

Result of visualize

Return type

List[np.ndarray]

class mmagic.apis.inferencers.UnconditionalInferencer(config: Union[mmagic.utils.ConfigType, str], ckpt: Optional[str], device: Optional[str] = None, extra_parameters: Optional[Dict] = None, seed: int = 2022, **kwargs)[source]¶

Bases: mmagic.apis.inferencers.base_mmagic_inferencer.BaseMMagicInferencer

inferencer that predicts with unconditional models.

func_kwargs¶

extra_parameters¶

preprocess() → Dict[source]¶

Process the inputs into a model-feedable format.

Returns: Results of preprocess.
Return type: results(Dict)

forward(inputs: mmagic.apis.inferencers.base_mmagic_inferencer.InputsType) → mmagic.apis.inferencers.base_mmagic_inferencer.PredType[source]¶: Forward the inputs to the model.

visualize(preds: mmagic.apis.inferencers.base_mmagic_inferencer.PredType, result_out_dir: str = '') → List[numpy.ndarray][source]¶

Visualize predictions.

Parameters

preds (List[Union[str, np.ndarray]]) – Forward results by the inferencer.
data (List[Dict]) – Not needed by this kind of inferencer.
result_out_dir (str) – Output directory of image. Defaults to ‘’.

Returns

Result of visualize

Return type

List[np.ndarray]

_pred2dict(data_sample: mmagic.structures.DataSample) → Dict[source]¶

Extract elements necessary to represent a prediction into a dictionary. It’s better to contain only basic data elements such as strings and numbers in order to guarantee it’s json-serializable.

Parameters: data_sample (DataSample) – The data sample to be converted.
Returns: The output dictionary.
Return type: dict

class mmagic.apis.inferencers.VideoInterpolationInferencer(config: Union[mmagic.utils.ConfigType, str], ckpt: Optional[str], device: Optional[str] = None, extra_parameters: Optional[Dict] = None, seed: int = 2022, **kwargs)[source]¶

Bases: mmagic.apis.inferencers.base_mmagic_inferencer.BaseMMagicInferencer

inferencer that predicts with video interpolation models.

func_kwargs¶

extra_parameters¶

preprocess(video: mmagic.apis.inferencers.base_mmagic_inferencer.InputsType) → Dict[source]¶

Process the inputs into a model-feedable format.

Parameters: video (InputsType) – Video to be interpolated by models.
Returns: Video to be interpolated by models.
Return type: video(InputsType)

forward(inputs: mmagic.apis.inferencers.base_mmagic_inferencer.InputsType, result_out_dir: mmagic.apis.inferencers.base_mmagic_inferencer.InputsType = '') → mmagic.apis.inferencers.base_mmagic_inferencer.PredType[source]¶

Forward the inputs to the model.

Parameters

inputs (InputsType) – Input video directory.
result_out_dir (str) – Output directory of video. Defaults to ‘’.

Returns

Result of forwarding

Return type

PredType

visualize(preds: mmagic.apis.inferencers.base_mmagic_inferencer.PredType, result_out_dir: str = '') → List[numpy.ndarray][source]¶: Visualize is not needed in this inferencer.

postprocess(preds: mmagic.apis.inferencers.base_mmagic_inferencer.PredType, imgs: Optional[List[numpy.ndarray]] = None) → Union[mmagic.apis.inferencers.base_mmagic_inferencer.ResType, Tuple[mmagic.apis.inferencers.base_mmagic_inferencer.ResType, numpy.ndarray]][source]¶: Postprocess is not needed in this inferencer.

class mmagic.apis.inferencers.VideoRestorationInferencer(config: Union[mmagic.utils.ConfigType, str], ckpt: Optional[str], device: Optional[str] = None, extra_parameters: Optional[Dict] = None, seed: int = 2022, **kwargs)[source]¶

Bases: mmagic.apis.inferencers.base_mmagic_inferencer.BaseMMagicInferencer

inferencer that predicts with video restoration models.

func_kwargs¶

extra_parameters¶

preprocess(video: mmagic.apis.inferencers.base_mmagic_inferencer.InputsType) → Dict[source]¶

Process the inputs into a model-feedable format.

Parameters: video (InputsType) – Video to be restored by models.
Returns: Results of preprocess.
Return type: results(InputsType)

forward(inputs: mmagic.apis.inferencers.base_mmagic_inferencer.InputsType) → mmagic.apis.inferencers.base_mmagic_inferencer.PredType[source]¶

Forward the inputs to the model.

Parameters: inputs (InputsType) – Images array of input video.
Returns: Results of forwarding
Return type: PredType

visualize(preds: mmagic.apis.inferencers.base_mmagic_inferencer.PredType, result_out_dir: str = '') → List[numpy.ndarray][source]¶

Visualize predictions.

Parameters

preds (List[Union[str, np.ndarray]]) – Forward results by the inferencer.
data (List[Dict]) – Not needed by this kind of inferencer.
result_out_dir (str) – Output directory of image. Defaults to ‘’.

Returns

Result of visualize

Return type

List[np.ndarray]

postprocess(preds: mmagic.apis.inferencers.base_mmagic_inferencer.PredType, imgs: Optional[List[numpy.ndarray]] = None) → Union[mmagic.apis.inferencers.base_mmagic_inferencer.ResType, Tuple[mmagic.apis.inferencers.base_mmagic_inferencer.ResType, numpy.ndarray]][source]¶: Postprocess is not needed in this inferencer.

mmagic.apis.inferencers¶

Package Contents¶

Classes¶

`mmagic.apis.inferencers`¶