The challenge
Turning hours of long-form video into short clips is slow, repetitive work: download, transcribe, find the moments, cut, reframe for vertical, add captions, review, submit. Doing it by hand does not scale, but removing people from the loop entirely produces clips nobody would publish.
The pipeline also had to survive the reality of media processing: a crash after 30 of 100 clips, a transcription service timing out, a queue server restarting. Redoing finished work on every failure was not acceptable.
Approach
Every unit of work is a row in PostgreSQL with an explicit state; Redis and BullMQ only move work around. If Redis is lost entirely, nothing is lost but time. Each pipeline stage checks whether its output already exists and returns immediately if it does, so re-enqueueing is always safe and a crash resumes from where it stopped rather than from the start.
Failures are never thrown away. A stage writes the reason and whether it is retryable to the row before it fails, so both the retry logic and the operator dashboard can see exactly what happened.
Browser automation for the submission side runs at a human-like pace with randomised delays, one session per account, logging every attempt and taking a screenshot on failure. Two consecutive failures of the same automation raise a red banner for the operator. Posting to video platforms through automated browsers is deliberately out of scope; official APIs or a person do that.
Architecture
- NestJS modular monolith with BullMQ producers and consumers per stage and a sweeper that catches work stuck in transient states.
- Python workers for transcription with faster-whisper and for face-tracked reframing, and ffmpeg for cutting and encoding.
- A Remotion render service with headless Chrome for subtitles and hook overlays.
- Playwright automation for scraping campaign sources, submitting clips and reconciling results.
- A Next.js operator dashboard that starts batches from a URL, shows every source and clip moving through its states, exposes failure reasons and retries failed clips.
Results
The system is in active development with the core pipeline running end to end: batches move from URL to reviewed clips through idempotent, resumable stages, and the operator sees every failure with its reason. Throughput and time-saved figures will be published once the client has been running it in production for a full cycle.