mmagic.models.editors.real_basicvsr

Package Contents

Classes

RealBasicVSR	RealBasicVSR model for real-world video super-resolution.
RealBasicVSRNet	RealBasicVSR network structure for real-world video super-resolution.

class mmagic.models.editors.real_basicvsr.RealBasicVSR(generator, discriminator=None, gan_loss=None, pixel_loss=None, cleaning_loss=None, perceptual_loss=None, is_use_sharpened_gt_in_pixel=False, is_use_sharpened_gt_in_percep=False, is_use_sharpened_gt_in_gan=False, is_use_ema=False, train_cfg=None, test_cfg=None, init_cfg=None, data_preprocessor=None)

Bases: mmagic.models.editors.real_esrgan.RealESRGAN

RealBasicVSR model for real-world video super-resolution.

Ref: Investigating Tradeoffs in Real-World Video Super-Resolution, arXiv

Parameters

• generator (dict) – Config for the generator.

• discriminator (dict, optional) – Config for the discriminator. Default: None.

• gan_loss (dict, optional) – Config for the gan loss. Note that the loss weight in gan loss is only for the generator.

• pixel_loss (dict, optional) – Config for the pixel loss. Default: None.

• cleaning_loss (dict, optional) – Config for the image cleaning loss. Default: None.

• perceptual_loss (dict, optional) – Config for the perceptual loss. Default: None.

• is_use_sharpened_gt_in_pixel (bool, optional) – Whether to use the image sharpened by unsharp masking as the GT for pixel loss. Default: False.

• is_use_sharpened_gt_in_percep (bool, optional) – Whether to use the image sharpened by unsharp masking as the GT for perceptual loss. Default: False.

• is_use_sharpened_gt_in_gan (bool, optional) – Whether to use the image sharpened by unsharp masking as the GT for adversarial loss. Default: False.

• train_cfg (dict) – Config for training. Default: None. You may change the training of gan by setting: disc_steps: how many discriminator updates after one generate update; disc_init_steps: how many discriminator updates at the start of the training. These two keys are useful when training with WGAN.

• test_cfg (dict) – Config for testing. Default: None.

• init_cfg (dict, optional) – The weight initialized config for BaseModule. Default: None.

• data_preprocessor (dict, optional) – The pre-process config of BaseDataPreprocessor. Default: None.

extract_gt_data(data_samples)

extract gt data from data samples.

Parameters

data_samples (list) – List of DataSample.

Returns

Extract gt data.

Return type

Tensor

g_step(batch_outputs, batch_gt_data)

G step of GAN: Calculate losses of generator.

Parameters

• batch_outputs (Tensor) – Batch output of generator.

• batch_gt_data (Tuple[Tensor]) – Batch GT data.

Returns

Dict of losses.

Return type

dict

train_step(data: List[dict], optim_wrapper: mmengine.optim.OptimWrapperDict) → Dict[str, torch.Tensor]

Train step of GAN-based method.

Parameters

• data (List[dict]) – Data sampled from dataloader.

• optim_wrapper (OptimWrapper) – OptimWrapper instance used to update model parameters.

Returns

A dict of tensor for logging.

Return type

Dict[str, torch.Tensor]

forward_train(batch_inputs, data_samples=None)

Forward Train.

Run forward of generator with return_lqs=True

Parameters

• batch_inputs (Tensor) – Batch inputs.

• data_samples (List[DataSample]) – Data samples of Editing. Default:None

Returns

Result of generator.

(outputs, lqs)

Return type

Tuple[Tensor]

class mmagic.models.editors.real_basicvsr.RealBasicVSRNet(mid_channels=64, num_propagation_blocks=20, num_cleaning_blocks=20, dynamic_refine_thres=255, spynet_pretrained=None, is_fix_cleaning=False, is_sequential_cleaning=False)

Bases: mmengine.model.BaseModule

RealBasicVSR network structure for real-world video super-resolution.

Support only x4 upsampling.

Paper:

Investigating Tradeoffs in Real-World Video Super-Resolution, arXiv

Parameters

• mid_channels (int, optional) – Channel number of the intermediate features. Default: 64.

• num_propagation_blocks (int, optional) – Number of residual blocks in each propagation branch. Default: 20.

• num_cleaning_blocks (int, optional) – Number of residual blocks in the image cleaning module. Default: 20.

• dynamic_refine_thres (int, optional) – Stop cleaning the images when the residue is smaller than this value. Default: 255.

• spynet_pretrained (str, optional) – Pre-trained model path of SPyNet. Default: None.

• is_fix_cleaning (bool, optional) – Whether to fix the weights of the image cleaning module during training. Default: False.

• is_sequential_cleaning (bool, optional) – Whether to clean the images sequentially. This is used to save GPU memory, but the speed is slightly slower. Default: False.

forward(lqs, return_lqs=False)

Forward function for BasicVSR++.

Parameters

• lqs (tensor) – Input low quality (LQ) sequence with shape (n, t, c, h, w).

• return_lqs (bool) – Whether to return LQ sequence. Default: False.

Returns

Output HR sequence.

Return type

Tensor