Video Use: Editing Video by Talking to an Agent, Not a Timeline

Video Use is an open-source agent skill that turns a folder of raw footage into a finished video. You describe what you want in a sentence; the agent transcribes the audio, plans the cuts against the transcript, and drives ffmpeg to render them. There is no timeline, and you never scrub.

Its central bet is unusual and, once you see it, obvious: video editing is mostly a text problem.

Key takeaways

• Word-level transcription is the source of truth. Cuts, trims and filler-word removal are decided on text, not pixels.
• Images are consulted grudgingly — filmstrips and waveforms rendered only at decision points, rather than feeding every frame to a model.
• Every edit becomes an ffmpeg filter chain, including a 30ms audio fade at each cut so joins do not pop.
• Animated overlays are delegated to parallel sub-agents using whichever renderer suits the shot.
• It re-inspects its own render at the cut boundaries before handing the file over.
• It needs ffmpeg, shell access and a paid transcription key. The quality ceiling is the transcript's accuracy.

What problem does it solve?

A two-minute product demo takes a competent editor several hours. Almost none of that is craft. It is scrubbing for every "umm", trimming the four seconds where you reached for the mouse, matching the colour of the clip shot near a window against the one shot at night, keying the captions, exporting, noticing a mistake, exporting again.

For a small team or an agency, that cost is why the demo video does not exist. The founder records something, looks at the timeline, and quietly decides the feature can launch without a video.

The obvious AI fix does not work. Feed the frames to a vision model and you are shipping thirty thousand images to answer a question about pacing. The project states the arithmetic plainly: roughly forty-five million tokens of noise. Nobody is running that per edit.

How does Video Use work?

Two layers, and the split between them is the whole design.

1. The transcript layer. Audio is transcribed with word-level timestamps and speaker identity, then packed into around twelve kilobytes of structured markdown. Every editorial decision — what to cut, what to keep, where a sentence really ends — happens here. Twelve kilobytes against forty-five million tokens is the entire argument.
2. The visual layer, on demand. When the agent needs to see something — is this cut point mid-gesture? is the speaker still in frame? — it renders a composite: a filmstrip of nearby frames, a waveform, word labels. A handful of images at the moments that matter, rather than everything.

The decisions are then compiled into ffmpeg filter chains. One detail worth admiring: it applies a 30ms audio fade at every cut. That is the difference between an edit that sounds edited and one that does not, and it is exactly the kind of thing a general model would not think to do unprompted.

After rendering, it inspects its own output at the cut boundaries — a self-review pass before delivery. Session state lives in a project file, so a second session picks up the editorial context rather than starting from scratch.

What does the workflow look like?

Setup is a clone, a symlink into your skills directory, and dependencies:

git clone https://github.com/browser-use/video-use ~/Developer/video-use
ln -sfn ~/Developer/video-use ~/.claude/skills/video-use
cd ~/Developer/video-use
uv sync
brew install ffmpeg
cp .env.example .env

Then the editing session itself is almost anticlimactic:

cd /path/to/your/videos
claude
# > edit these into a launch video

From there you direct it in prose. Tighter. Lose the intro. Keep the bit where the test passes. Put subtitles on. The agent works from the transcript, so "cut everything before he says deploy" is a query it can actually answer.

What it is good at, and what it is not

Alternatives worth knowing

• Descript — the commercial incarnation of the same idea, and a mature one: edit the transcript, the video follows. Polished, hosted, paid, and closed. If you want this workflow without a terminal, this is the answer.
• Remotion — video as React components. A completely different proposition: brilliant for programmatic, templated, data-driven video, useless for cutting footage of a human talking.
• auto-editor — a small Python tool that removes silence. Does one tenth of what Video Use does, with one hundredth of the setup. Often that one tenth is the whole job.
• ffmpeg by hand — still the substrate under all of the above. If your edit is deterministic and repeatable, a shell script is more reliable than any agent.

Limitations to plan around

• Transcription quality is the ceiling. Heavy accents, crosstalk, poor audio and technical jargon all degrade the transcript, and every downstream decision inherits that.
• It needs shell access to your footage, which is a permission decision, not just a setup step.
• Transcription is a paid API. Small per video, real across a back catalogue.
• The install path assumes macOS. Everything it needs exists elsewhere, but you will adapt the commands.
• It is a young project. Keep your source footage; never let it edit your only copy.

Conclusion

Video Use is worth studying even if you never edit a video, because the insight generalises. Faced with an expensive modality, it found the cheap representation that carries most of the signal, and only paid for pixels where text genuinely could not answer the question.

That pattern — find the compact representation, escalate selectively — is the same one behind querying a code graph instead of reading files. Different domain, identical move.

Want product videos without hiring an editor?

DI Solutions builds the automation around tools like this — pipelines that turn recordings into clips, subtitles and localised cuts on a schedule. Talk to our engineers about what that looks like for your content.

Reference links

• Video Use on GitHub
• ffmpeg documentation