LumiVid: HDR Video Generation via Latent Alignment with Logarithmic Encoding

Video

Overview video

Narrated overview

Results

SDR → HDR at normal exposure. Click a tile to expand.

Stills

HDR frames

SDR → HDR on still imagery. Click a tile to expand.

Compare methods

How each method was rendered

LumiVid: ACEScct EXR → Rec.709 × 2^EV → BT.2020 HLG. Tedla et al.: official 17-frame 704×1280 recipe; holds its last frame while the other method continues. HDRTVNet++: frame-wise, stacked to video. DiffHDR: official recipe, Rec.709-linear EXRs. SDR stays Rec.709 SDR. Missing files show a red tag.

Ablations

Camera augmentations

No augmentation, blur only, and the full model.

LumiVid is the paper model. The VAE decoder never runs the video DiT: it can lift exposure, and it does not restore highlight detail or hold time. Pull to EV −3 for highlights and EV +3 for shadows.

Abstract

High dynamic range (HDR) video captures continuous scene radiance essential for professional color grading and post-production workflows, yet generative video diffusion models remain bound to 8-bit standard dynamic range (SDR) formats. Existing adaptation methods either alter the VAE latent space or rely on auxiliary inputs like luminance masks, text prompts, or external reference images. However, relying on auxiliary inputs introduces manual overhead, makes the exact prompt hard to guess, and yields less stable results that often fail to remain consistent with the original footage. We present LumiVid, a framework that generates HDR video directly from a single SDR input, without requiring masks, text prompts, or external reference images. Key to our method is mapping the HDR video into a logarithmic color space representation, aligning high-magnitude radiance with the existing latent space of a frozen video VAE without losing luminance details. Fine-tuning a dual-canvas LoRA adapter on just 300 synthetic paired clips enables data-efficient SDR-to-HDR translation while preserving temporal consistency. To prevent shortcut pixel copying during training and force the model to infer missing radiance and details from its learned priors, we introduce camera-mimicking condition degradations on the SDR reference input. Extensive evaluations show that LumiVid outperforms state-of-the-art baselines by +5.5 dB PU21-PSNR on cinema-grade ARRI footage, producing high-fidelity half-float EXR video with exceptional re-exposure latitude across extreme exposure pulls.

Method

SDR-to-HDR dual canvas LoRA

Training overview

ACEScct logarithmic encoding aligns HDR radiance with the frozen VAE input range.
Asymmetric clamp: SDR [0, 0.55] vs HDR [0, 1.0] (≈223× white)
Run the VAE encoder in the ACEScct representation to get HDR latents
Train dual canvas LoRA, VAE stays frozen
5000 steps on ~300 paired 49-frame Poly Haven orbit clips
Camera-mimicking degradations on the SDR input like contrast clipping, highlight/shadow blur, MP4 compression and joint exposure shifts, force the model to infer missing radiance

Inference overview

Rec.709 → ACEScg → ACEScct → VAE → DiT → decode → inverse ACEScct → 16-bit half-float EXR
SDR in, scene-linear HDR out — no masks, text prompts, or external images
Outputs float16 EXR scene-linear, a gradeable, compositable plate
Re-expose in post: EV stops are rendered from the EXR, not re-generated

LumiVid training diagram (Fig. 2): dual-canvas ACEScct-encoded SDR/HDR pairs train a rank-128 LoRA on a frozen video VAE and DiT — Fig. 2 — Training: dual-canvas ACEScct pairs, frozen VAE, LoRA only.

LumiVid inference diagram (Fig. 3): SDR Rec.709 through ACEScg and ACEScct into the frozen VAE and dual-canvas DiT, decoded to half-float EXR — Fig. 3 — Inference: SDR in, scene-linear half-float EXR out.