mmagic.models.editors.basicvsr.basicvsr_net
¶
Module Contents¶
Classes¶
BasicVSR network structure for video super-resolution. |
|
Residual blocks with a convolution in front. |
|
SPyNet network structure. |
|
Basic Module for SPyNet. |
- class mmagic.models.editors.basicvsr.basicvsr_net.BasicVSRNet(mid_channels=64, num_blocks=30, spynet_pretrained=None)[source]¶
Bases:
mmengine.model.BaseModule
BasicVSR network structure for video super-resolution.
Support only x4 upsampling.
- Paper:
BasicVSR: The Search for Essential Components in Video Super-Resolution and Beyond, CVPR, 2021
- Parameters
mid_channels (int) – Channel number of the intermediate features. Default: 64.
num_blocks (int) – Number of residual blocks in each propagation branch. Default: 30.
spynet_pretrained (str) – Pre-trained model path of SPyNet. Default: None.
- check_if_mirror_extended(lrs)[source]¶
Check whether the input is a mirror-extended sequence.
If mirror-extended, the i-th (i=0, …, t-1) frame is equal to the (t-1-i)-th frame.
- Parameters
lrs (tensor) – Input LR images with shape (n, t, c, h, w)
- compute_flow(lrs)[source]¶
Compute optical flow using SPyNet for feature warping.
Note that if the input is an mirror-extended sequence, ‘flows_forward’ is not needed, since it is equal to ‘flows_backward.flip(1)’.
- Parameters
lrs (tensor) – Input LR images with shape (n, t, c, h, w)
- Returns
- Optical flow. ‘flows_forward’ corresponds to the
flows used for forward-time propagation (current to previous). ‘flows_backward’ corresponds to the flows used for backward-time propagation (current to next).
- Return type
tuple(Tensor)
- class mmagic.models.editors.basicvsr.basicvsr_net.ResidualBlocksWithInputConv(in_channels, out_channels=64, num_blocks=30)[source]¶
Bases:
mmengine.model.BaseModule
Residual blocks with a convolution in front.
- Parameters
in_channels (int) – Number of input channels of the first conv.
out_channels (int) – Number of channels of the residual blocks. Default: 64.
num_blocks (int) – Number of residual blocks. Default: 30.
- class mmagic.models.editors.basicvsr.basicvsr_net.SPyNet(pretrained)[source]¶
Bases:
mmengine.model.BaseModule
SPyNet network structure.
- The difference to the SPyNet in [tof.py] is that
more SPyNetBasicModule is used in this version, and
no batch normalization is used in this version.
- Paper:
Optical Flow Estimation using a Spatial Pyramid Network, CVPR, 2017
- Parameters
pretrained (str) – path for pre-trained SPyNet. Default: None.
- compute_flow(ref, supp)[source]¶
Compute flow from ref to supp.
Note that in this function, the images are already resized to a multiple of 32.
- Parameters
ref (Tensor) – Reference image with shape of (n, 3, h, w).
supp (Tensor) – Supporting image with shape of (n, 3, h, w).
- Returns
Estimated optical flow: (n, 2, h, w).
- Return type
Tensor
- forward(ref, supp)[source]¶
Forward function of SPyNet.
This function computes the optical flow from ref to supp.
- Parameters
ref (Tensor) – Reference image with shape of (n, 3, h, w).
supp (Tensor) – Supporting image with shape of (n, 3, h, w).
- Returns
Estimated optical flow: (n, 2, h, w).
- Return type
Tensor