mmagic.models.editors.flavr.flavr_net

Module Contents

Classes

FLAVRNet	PyTorch implementation of FLAVR for video frame interpolation.
Encoder	Encoder of FLAVR.
Decoder	Decoder of FLAVR.
UpConv3d	A conv block that bundles conv/SEGating/norm layers.
Conv3d	A conv block that bundles conv/SEGating/norm layers.
BasicStem	The default conv-batchnorm-relu stem of FLAVR.
BasicBlock	Basic block of encoder in FLAVR.
SEGating	Gating of SE attention.

class mmagic.models.editors.flavr.flavr_net.FLAVRNet(num_input_frames, num_output_frames, mid_channels_list=[512, 256, 128, 64], encoder_layers_list=[2, 2, 2, 2], bias=False, norm_cfg=None, join_type='concat', up_mode='transpose', init_cfg=None)

Bases: mmengine.model.BaseModule

PyTorch implementation of FLAVR for video frame interpolation.

Paper:

FLAVR: Flow-Agnostic Video Representations for Fast Frame Interpolation

Ref repo: https://github.com/tarun005/FLAVR

Parameters

• num_input_frames (int) – Number of input frames.

• num_output_frames (int) – Number of output frames.

• mid_channels_list (list[int]) – List of number of mid channels. Default: [512, 256, 128, 64]

• encoder_layers_list (list[int]) – List of number of layers in encoder. Default: [2, 2, 2, 2]

• bias (bool) – If True, adds a learnable bias to the conv layers. Default: True

• norm_cfg (dict | None) – Config dict for normalization layer. Default: None

• join_type (str) – Join type of tensors from decoder and encoder. Candidates are concat and add. Default: concat

• up_mode (str) – Up-mode UpConv3d, candidates are transpose and trilinear. Default: transpose

• init_cfg (dict, optional) – Initialization config dict. Default: None.

forward(images: torch.Tensor)

Forward function.

Parameters

images (Tensor) – Input frames tensor with shape (N, T, C, H, W).

Returns

Output tensor.

Return type

out (Tensor)

class mmagic.models.editors.flavr.flavr_net.Encoder(block, layers, stem_layer, mid_channels_list, norm_cfg, bias)

Bases: torch.nn.Module

Encoder of FLAVR.

Parameters

• block (nn.Module) – Basic block of encoder.

• layers (str) – List of layers in encoder.

• stem_layer (nn.Module) – stem layer (conv first).

• mid_channels_list (list[int]) – List of mid channels.

• norm_cfg (dict | None) – Config dict for normalization layer. Default: None

• bias (bool) – If True, adds a learnable bias to the conv layers. Default: True

forward(x)

Forward function.

Parameters

x (Tensor) – Input tensor).

Returns

Output tensors.

Return type

tuple(Tensor)

_make_layer(block, mid_channels, num_blocks, norm_cfg, stride=1, temporal_stride=None)

Make layers by stacking the blocks.

_initialize_weights()

Init weights for models.

class mmagic.models.editors.flavr.flavr_net.Decoder(join_type, up_mode, mid_channels_list=[512, 256, 128, 64], batchnorm=False)

Bases: torch.nn.Module

Decoder of FLAVR.

Parameters

• join_type (str) – Join type of tensors from decoder and encoder. Candidates are concat and add. Default: concat

• up_mode (str) – Up-mode UpConv3d, candidates are transpose and trilinear. Default: transpose

• mid_channels_list (list[int]) – List of mid channels. Default: [512, 256, 128, 64]

• batchnorm (bool) – Whether contains BatchNorm3d. Default: False.

forward(xs)

Forward function.

Parameters

xs (Tensor) – Input tensor).

Returns

Output tensor.

Return type

dx_out (Tensor)

_join_tensors(x1, x2)

Concat or Add two tensors.

Parameters

• x1 (Tensor) – The first input tensor.

• x2 (Tensor) – The second input tensor.

class mmagic.models.editors.flavr.flavr_net.UpConv3d(in_channels, out_channels, kernel_size, stride, padding, up_mode='transpose', batchnorm=False)

Bases: torch.nn.Module

A conv block that bundles conv/SEGating/norm layers.

Parameters

• in_channels (int) – Number of channels in the input feature map. Same as that in nn._ConvNd.

• out_channels (int) – Number of channels produced by the convolution. Same as that in nn._ConvNd.

• kernel_size (int | tuple[int]) – Size of the convolving kernel. Same as that in nn._ConvNd.

• stride (int | tuple[int]) – Stride of the convolution. Same as that in nn._ConvNd.

• padding (int | tuple[int]) – Zero-padding added to both sides of the input. Same as that in nn._ConvNd.

• up_mode (str) – Up-mode UpConv3d, candidates are transpose and trilinear. Default: transpose.

• batchnorm (bool) – Whether contains BatchNorm3d. Default: False.

forward(x)

Forward function.

class mmagic.models.editors.flavr.flavr_net.Conv3d(in_channels, out_channels, kernel_size, stride=1, padding=0, bias=True, batchnorm=False)

Bases: torch.nn.Module

A conv block that bundles conv/SEGating/norm layers.

Parameters

• in_channels (int) – Number of channels in the input feature map. Same as that in nn._ConvNd.

• out_channels (int) – Number of channels produced by the convolution. Same as that in nn._ConvNd.

• kernel_size (int | tuple[int]) – Size of the convolving kernel. Same as that in nn._ConvNd.

• stride (int | tuple[int]) – Stride of the convolution. Same as that in nn._ConvNd. Default: 1.

• padding (int | tuple[int]) – Zero-padding added to both sides of the input. Same as that in nn._ConvNd.

• bias (bool) – If True, adds a learnable bias to the conv layer. Default: True

• batchnorm (bool) – Whether contains BatchNorm3d. Default: False.

forward(x)

Forward function.

class mmagic.models.editors.flavr.flavr_net.BasicStem(out_channels=64, bias=False, norm_cfg=None)

Bases: mmcv.cnn.ConvModule

The default conv-batchnorm-relu stem of FLAVR.

Parameters

• out_channels (int) – Number of output channels. Default: 64

• bias (bool) – If True, adds a learnable bias to the conv layer. Default: False

• norm_cfg (dict | None) – Config dict for normalization layer. Default: None.

class mmagic.models.editors.flavr.flavr_net.BasicBlock(in_channels, mid_channels, stride=1, norm_cfg=None, bias=False, downsample=None)

Bases: torch.nn.Module

Basic block of encoder in FLAVR.

Parameters

• in_channels (int) – Number of channels in the input feature map.

• mid_channels (int) – Number of middle channels.

• stride (int | tuple[int]) – Stride of the first convolution. Default: 1.

• norm_cfg (dict | None) – Config dict for normalization layer. Default: None.

• bias (bool) – If True, adds a learnable bias to the conv layers. Default: True

• downsample (None | torch.nn.Module) – Down-sample layer. Default: None.

expansion = 1

forward(x)

Forward function.

Parameters

xs (Tensor) – Input tensor).

Returns

Output tensor.

Return type

out (Tensor)

class mmagic.models.editors.flavr.flavr_net.SEGating(in_channels)

Bases: torch.nn.Module

Gating of SE attention.

Parameters

in_channels (int) – Number of channels in the input feature map.

forward(x)

Forward function.

Parameters

x (Tensor) – Input tensor).

Returns

Output tensors.

Return type

Tensor