Video Generation
268 papers with code • 15 benchmarks • 14 datasets
( Various Video Generation Tasks. Gif credit: MaGViT )
Benchmarks
These leaderboards are used to track progress in Video Generation
| Trend | Dataset | Best Model | Paper | Code | Compare |
|---|---|---|---|---|---|
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
Libraries
Use these libraries to find Video Generation models and implementationsDatasets
UCF101
Kinetics
MSR-VTT
WebVid
LAION-400M
Kinetics-600
How2Sign
BAIR Robot Pushing
Most implemented papers
FreeNoise: Tuning-Free Longer Video Diffusion via Noise Rescheduling
With the availability of large-scale video datasets and the advances of diffusion models, text-driven video generation has achieved substantial progress.
VideoCrafter1: Open Diffusion Models for High-Quality Video Generation
The I2V model is designed to produce videos that strictly adhere to the content of the provided reference image, preserving its content, structure, and style.
Latte: Latent Diffusion Transformer for Video Generation
maxin-cn/Latte
•
•
We propose a novel Latent Diffusion Transformer, namely Latte, for video generation.
Train Sparsely, Generate Densely: Memory-efficient Unsupervised Training of High-resolution Temporal GAN
Training of Generative Adversarial Network (GAN) on a video dataset is a challenge because of the sheer size of the dataset and the complexity of each observation.
Video Generation from Single Semantic Label Map
This paper proposes the novel task of video generation conditioned on a SINGLE semantic label map, which provides a good balance between flexibility and quality in the generation process.
VirtualConductor: Music-driven Conducting Video Generation System
ChenDelong1999/VirtualConductor
•
•
In this demo, we present VirtualConductor, a system that can generate conducting video from any given music and a single user's image.
Diffusion Models: A Comprehensive Survey of Methods and Applications
This survey aims to provide a contextualized, in-depth look at the state of diffusion models, identifying the key areas of focus and pointing to potential areas for further exploration.
Make-A-Video: Text-to-Video Generation without Text-Video Data
lucidrains/make-a-video-pytorch
•
•
We propose Make-A-Video -- an approach for directly translating the tremendous recent progress in Text-to-Image (T2I) generation to Text-to-Video (T2V).
Phenaki: Variable Length Video Generation From Open Domain Textual Description
lucidrains/phenaki-pytorch
•
•
To the best of our knowledge, this is the first time a paper studies generating videos from time variable prompts.
Scalable Adaptive Computation for Iterative Generation
We show how to leverage recurrence by conditioning the latent tokens at each forward pass of the reverse diffusion process with those from prior computation, i. e. latent self-conditioning.
