Tsinghua's LiveEdit brings text-prompt video editing to live streams
Video editing by text prompt has been an offline affair — you describe the change, then wait while the model chews through the whole clip. A team at Tsinghua University and Hong Kong University of Science and Technology wants it live instead.
A video editor that keeps up with a live feed just cleared the peer-review bar. LiveEdit, from Tsinghua and HKUST, takes continuously arriving video and applies plain-language edits chunk by chunk — recolor an object, swap a material, restyle a region — at 12.66 frames per second, fast enough for livestream effects, video calls, and AR overlays. The work was accepted to ECCV 2026, and the training code, test code, and models are all open-sourced. That combination matters: most "real-time" demos in this space die at the reproducibility step, and this one hands you everything.
The hard part is architectural. High-quality video diffusion models look at the entire clip at once, letting future frames anchor the past — fine for rendering a finished video, impossible when the future hasn't arrived yet. Simply cutting off the future makes attention drift and backgrounds flicker. LiveEdit retrains its way around it in three stages: first learn solid two-way editing, then force the model to reproduce those edits seeing only current and past frames, and finally distill 100 denoising steps down to 4 so each chunk renders almost instantly.
Speed also comes from not recomputing what didn't change. When you edit a jacket, the background is identical frame to frame, so LiveEdit predicts which regions the next chunk will touch from the previous chunk's differences, and reuses cached computations for untouched areas — about 70% of tokens in its test setup. The team found reuse works at the attention layers but blurs detail if applied everywhere, which is the kind of engineering honesty that suggests real experiments rather than a benchmark-chasing demo.
The numbers back it up. Editing 81 frames took 7.89 seconds end to end, and in a 20-person user study LiveEdit ranked top-three on 100% of ballots for matching the instruction, 87.5% for keeping the background intact, and 95.8% for overall quality — against methods that either run offline or wobble badly when streamed. The honest caveat: the approach assumes edited regions don't jump wildly between moments, so fast-moving targets or whole-frame restyling are still weak spots. It's built on Alibaba's open Wan2.1 1.3B video model, trained on 20,000 curated editing pairs.
Real-time generation keeps marching this direction — we noted last week when a playable world model ran 720p at 16 FPS on one RTX 5090. Streaming-first is becoming the default posture for generative video, not the exception.
What to watch: whether the region-cache trick survives chaotic handheld footage, and which conferencing or livestreaming platform ships diffusion editing before anyone expects it.
When a live video call can be quietly restyled mid-conversation, how will you know what the camera actually saw? Tell us in the comments.
Sources: LeiPhone · LiveEdit paper (arXiv) · LiveEdit code (GitHub)
