Welcome to MMagic’s documentation!

Languages: English | 简体中文

MMagic (Multimodal Advanced, Generative, and Intelligent Creation) is an open-source AIGC toolbox for professional AI researchers and machine learning engineers to explore image and video processing, editing and generation.

MMagic supports various foundamental generative models, including:

• Unconditional Generative Adversarial Networks (GANs)

• Conditional Generative Adversarial Networks (GANs)

• Internal Learning

• Diffusion Models

• And many other generative models are coming soon!

MMagic supports various applications, including:

• Text-to-Image

• Image-to-image translation

• 3D-aware generation

• Image super-resolution

• Video super-resolution

• Video frame interpolation

• Image inpainting

• Image matting

• Image restoration

• Image colorization

• Image generation

• And many other applications are coming soon!

MMagic is based on PyTorch and is a part of the OpenMMLab project. Codes are available on GitHub.

Documentation

Community

• Contributing guidance
• MMagic projects

Get Started

• Overview
• Installation
• Quick run

User Guides

• Tutorial 1: Learn about Configs in MMagic
• Tutorial 2: Prepare datasets
• Tutorial 3: Inference with pre-trained models
• Tutorial 4: Train and test in MMagic
• Tutorial 5: Using metrics in MMagic
• Tutorial 6: Visualization
• Tutorial 7: Useful tools
• Tutorial 8: Deploy models in MMagic

Advanced Guides

• Evaluator
  • Evaluation Metrics and Evaluators
  • prepare_metrics
  • prepare_samplers
  • Evaluation process of an evaluator
• Data Structure
• Data pre-processor
  • The position of the data preprocessor in the training pipeline.
• Data flow
  • Overview of dataflow
  • Data flow between dataset and model
  • Data flow between model output and visualizer

How To

• How to design your own models
  • Overview of models in MMagic
  • An example of SRCNN
  • An example of DCGAN
  • References
• How to prepare your own datasets
  • Supported Data Format
  • Design a new dataset
• How to design your own data transforms
  • Data pipelines in MMagic
  • Supported transforms in MMagic
  • Extend and use custom pipelines
• How to design your own loss functions
  • Introduction to supported losses
  • Design a new loss function
  • Available losses

FAQ

• Frequently asked questions

Model Zoo

• Overview
• text2video
  • Summary
  • AnimateDiff (2023)
• inpainting
  • Summary
  • Stable Diffusion (2022)
  • AOT-GAN (TVCG’2021)
  • DeepFillv2 (CVPR’2019)
  • PConv (ECCV’2018)
  • DeepFillv1 (CVPR’2018)
  • Global&Local (ToG’2017)
• video super-resolution
  • Summary
  • RealBasicVSR (CVPR’2022)
  • BasicVSR++ (CVPR’2022)
  • IconVSR (CVPR’2021)
  • BasicVSR (CVPR’2021)
  • TDAN (CVPR’2020)
  • TOFlow (IJCV’2019)
  • EDVR (CVPRW’2019)
• conditional gans
  • Summary
  • BigGAN (ICLR’2019)
  • SAGAN (ICML’2019)
  • SNGAN (ICLR’2018)
• video interpolation
  • Summary
  • CAIN (AAAI’2020)
  • FLAVR (arXiv’2020)
  • TOFlow (IJCV’2019)
• text2image
  • Summary
  • FastComposer (2023)
  • Control Net (2023)
  • ViCo (2023)
  • Stable Diffusion XL (2023)
  • Disco Diffusion (2022)
  • DreamBooth (2022)
  • Stable Diffusion (2022)
  • Textual Inversion (2022)
• controlnet_animation
  • Summary
  • Controlnet Animation (2023)
• image2image
  • Summary
  • CycleGAN (ICCV’2017)
  • Pix2Pix (CVPR’2017)
• unconditional gans
  • Summary
  • StyleGANv3 (NeurIPS’2021)
  • Positional Encoding in GANs (CVPR’2021)
  • StyleGANv2 (CVPR’2020)
  • StyleGANv1 (CVPR’2019)
  • PGGAN (ICLR’2018)
  • GGAN (ArXiv’2017)
  • WGAN-GP (NeurIPS’2017)
  • LSGAN (ICCV’2017)
  • Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks (ICLR’2016)
• deblurring
  • Summary
  • Restormer (CVPR’2022)
  • DeblurGAN-v2 (ICCV’2019)
• image super-resolution
  • Summary
  • SwinIR (ICCVW’2021)
  • LIIF (CVPR’2021)
  • GLEAN (CVPR’2021)
  • Real-ESRGAN (ICCVW’2021)
  • TTSR (CVPR’2020)
  • DIC (CVPR’2020)
  • RDN (CVPR’2018)
  • ESRGAN (ECCVW’2018)
  • EDSR (CVPR’2017)
  • SRGAN (CVPR’2016)
  • SRCNN (TPAMI’2015)
• diffusers pipeline
  • Summary
  • Diffusers Pipeline (2023)
• matting
  • Summary
  • GCA (AAAI’2020)
  • IndexNet (ICCV’2019)
  • DIM (CVPR’2017)
• draggan
  • Summary
  • DragGAN (SIGGRAPH’2023)
• 3d-aware generation
  • Summary
  • EG3D (CVPR’2022)
• image generation
  • Summary
  • Guided Diffusion (NeurIPS’2021)
• colorization
  • Summary
  • Instance-aware Image Colorization (CVPR’2020)
• image restoration
  • Summary
  • NAFNet (ECCV’2022)
• deraining
  • Summary
  • Restormer (CVPR’2022)
• denoising
  • Summary
  • Restormer (CVPR’2022)
• internal learning
  • Summary
  • SinGAN (ICCV’2019)
• image denoising
  • Summary
  • SwinIR (ICCVW’2021)
• jpeg compression artifact reduction
  • Summary
  • SwinIR (ICCVW’2021)

Dataset Zoo

• Overview
• Preparing DF2K_OST Dataset
• Preparing RealSRSet Dataset
• Preparing VideoLQ Dataset
• Preparing GLEAN Dataset
• Preparing CelebA-HQ Dataset
• Preparing GoPro Dataset
• Preparing Vimeo90K Dataset
• Preparing Paired Dataset for Pix2pix
• Preparing SIDD Dataset
• Preparing DPDD Dataset
• Preparing UDM10 Dataset
• Preparing Unpaired Dataset for CycleGAN
• Preparing HIDE Dataset
• Preparing Paris Street View Dataset
• Preparing Vid4 Dataset
• Preparing NTIRE21 decompression Dataset
• Preparing Deraining Dataset
• Preparing REDS Dataset
• Preparing Denoising Dataset
• Preparing Composition-1k Dataset
• Preparing RealBlur Dataset
• Unconditional GANs Datasets
• Preparing SPMCS Dataset
• Preparing Places365 Dataset
• Preparing LIVE1 Dataset
• Preparing Vimeo90K-triplet Dataset
• Preparing Classic5 Dataset
• Preparing DIV2K Dataset

Changelog

• Changelog

API Reference

• mmagic.apis.inferencers
  • Package Contents
• mmagic.structures
  • Package Contents
• mmagic.datasets
  • Package Contents
• mmagic.datasets.transforms
  • Package Contents
• mmagic.evaluation
  • Package Contents
• mmagic.visualization
  • Package Contents
• mmagic.engine.hooks
  • Package Contents
• mmagic.engine.optimizers
  • Package Contents
• mmagic.engine.runner
  • Package Contents
• mmagic.engine.schedulers
  • Package Contents
• mmagic.models.archs
  • Package Contents
• mmagic.models.base_models
  • Package Contents
• mmagic.models.losses
  • Package Contents
• mmagic.models.data_preprocessors
  • Package Contents
• mmagic.models.editors
  • Package Contents
• mmagic.utils
  • Package Contents

Migration from MMEdit 0.x

• Overview
• Migration of Runtime Settings
• Migration of Model Settings
• Migration of Evaluation and Testing Settings
• Migration of Schedule Settings
• Migration of Data Settings
• Migration of Distributed Training Settings
• Migration of Optimizers
• Migration of Visualization
• Migration of AMP Training

Device Support

• NPU (HUAWEI Ascend)

Switch Language

• English
• 简体中文

Indices and tables

• Index

• Module Index

• Search Page