How Chitrio is built
One topic in, an upload-ready YouTube video out - script, voice, visuals, word-timed captions, thumbnail, upload. Every launch video we published was made by the product it describes. This page is the engineering story: the pipeline, the real economics, the measurements, and the bugs that actually shipped.
.NET 8FFmpegHangfireEF CoreReactImageSharpGPT-4oWhisper + forced alignmentFlux / gpt-image-1LTX / Kling / Veo
The pipeline, end to end
1 · ScriptA topic becomes structured scene JSON - narration, a cinematographer-style visual brief, a headline - with exact word budgets per scene. Long scripts are generated in chunks because the model has a real per-response output ceiling (~660 words, measured) that no max_tokens setting raises.
2 · VoiceTTS per scene (budget or premium voices, 12 languages). Character voices can be designed from a text description - fully synthetic, nobody real - three auditions, keep one. Anything involving a real person's voice or likeness requires an ownership + consent attestation, recorded.
3 · VisualsA still per scene ($0.003), then a per-scene decision: leave it as a directed Ken Burns move (free), or animate it with an image-to-video engine at $0.02–$1.20 per clip. This one decision is most of the margin.
4 · Quality gateA small vision model reviews three frames of every generated clip for the glitches budget engines actually produce - deformed hands, garbled lettering. Rejected clips are deleted, refunded, and the scene falls back to Ken Burns. A failed video is worse than a static scene.
5 · CaptionsWord-level timings by forced alignment against the narration we authored - never transcription of our own synthesized speech. Burned as karaoke-style ASS subtitles.
6 · AssemblyFFmpeg filter graphs: headline banners composed in ImageSharp and overlaid (text is never baked into a moving frame or it warps), crossfades, music with sidechain-style ducking, branding, watermarks - with a corner-occupancy system so overlays never sit on a presenter's face.
7 · PackageThumbnail (A/B variants), SEO metadata, vertical Short cut from the same workspace, optional auto-upload to YouTube with real per-video tags.
The economics, measured
Real per-unit AI costs (verified against provider billing, not list prices):
| Unit | Real cost | Notes |
|---|---|---|
| Scene still (Flux) | $0.003 | premium stills (gpt-image-1) ~$0.016 at low quality |
| Narration | $0.03 / 1k chars | premium voices ~$0.10 / 1k |
| Motion clip - budget (LTX) | $0.02 | fast, faithful to the source image; weak on faces |
| Motion clip - balanced (Kling v3) | ~$0.42 | holds human faces intact; more photoreal than 2.1 |
| Motion clip - premium (Veo) | $1.20 | most dynamic individual clips |
| Lip-synced presenter clip | ~$1.41–2.10 | your own photo or an invented character (nobody real) - consent attested & recorded; the priciest unit in the product |
| Typical 5-min explainer | ~$0.28 total | cache + stock + per-scene engine choice |
| Re-render after an edit | ~$0.05 | content-addressed cache re-buys only what changed |
Credits are integers on purpose - everything money-shaped here is integer minor units, because the refund reconciliation is the one place a rounding bug is unforgivable.
The engine comparison nobody expected
Same script, two live renders, identical duration. Motion measured as mean frame-to-frame luma delta. Budget engine on every scene vs premium on key scenes: whole-video motion 1.02 vs 0.99 - a 3% difference for 3× the credits and 22× the cash. Premium clips individually are 2.6× more dynamic; they just cover a third of the video, and the effects cancel.
A second finding inverted the price ladder entirely: given the same source image, the mid-priced engine drifted furthest from it - re-lit and restructured the scene into a different photograph - while the cheapest held composition faithfully. For brand consistency, price does not buy fidelity.
Both findings shipped as product: engine choice is per-scene (premium on the hook where retention is decided, budget elsewhere), and a zero-cost tier - a transparent character sprite with an idle-bob overlay - exists because the cheapest presenter is a cached PNG, not a $2 lip-sync clip.
Things that bit me
FFmpeg's amix normalises by 1/n - and n was my scene count.Two stacked mixes attenuated an 11-scene video ~20 dB, worse the longer the video. Months of “why is this quiet” with no error anywhere. Defaults that scale with your input are the ones that hurt, because the bug grows with success.
A chained xfade's memory grows with the video.Input i decodes immediately but isn't consumed until the sum of all preceding durations, so raw 1080p frames pile up: measured 459 MB at 2 clips → 1418 MB at 12, linear, ~+87 MB/clip. The kernel OOM-killed three real renders at 90%. Fix: batch the chain, join batches with the concat demuxer; the trade is a hard cut at each seam.
I destroyed 11 encrypted API keys by recreating a container.The data-protection key ring lived inside it. Worse, the decrypt failure was swallowed and read as “not configured” - indistinguishable from unset. Now: state on host volumes, and a health check that compares stored secrets against decryptable ones. That check later caught five dead secrets invisible to every other signal.
Whisper hallucinates on silence - and I billed for it.A silent upload returned one confident word; the pipeline translated it, synthesized it, and charged a credit. “Transcript not empty” is not “transcript real”: measure the audio level before the first paid call, sanity-check transcript coverage after.
The emoji font saga took three fixes.(1) The installed emoji font was bitmap-only; the compositor draws outlines - nothing rendered while “is it installed” passed for four deploys. (2) A can-this-font-draw-this guard tested the emoji font alone; the renderer draws primary-plus-fallbacks - check passed, boxes stayed. (3) The finale: a static field initializer ORDER bug - the fallback list was built before the emoji font field initialized, so the font was silently absent from the exact list the draw uses, while every check tested runtime-built chains that contained it. Presence is not capability; testing a component is not testing the path; and the artifact to measure is the one the failing code actually reads.
My ORM's generated migration would have silently muted every customer.Adding a non-nullable MusicVolume column, it scaffolded defaultValue: 0 - and AddColumn backfills the default into every existing row. Read generated migrations; look at defaultValue on any column whose zero means “off”. The same rule later caught a feature scaffolded off for every existing workspace and a cost column that would have stored every sub-cent AI call as $0.00.
My tests passed because every test used a fresh account.The asset cache is tenant-scoped; a fresh workspace has an empty cache, so the reuse path - the entire feature - never ran once under test while a formatting bug sat in exactly that path. Ask what state your fixture is missing: a fresh account is a specific, unrepresentative one.
A 403 that had nothing to do with permissions.YouTube refuses comments on private videos with an error that blames authorization. Four deploys of scope-chasing later, the fix was flipping one test upload to unlisted. When every attempt fails identically, suspect the variable you deliberately held fixed - and never ship an error message asserting a cause you haven't measured.
Questions, answered properly
Ask anything technical. Signing up is the price of a question - you get a real answer, we get your email. Fair trade, stated plainly.
Why not train or fine-tune your own video model?
Unit economics. A generated clip costs $0.02–$1.20 from providers who amortise training across thousands of customers; the differentiated work is everything around the model - choosing per scene which engine (or none) is worth paying for, caching so edits are nearly free, and catching bad output before a customer sees it. Orchestration compounds; a home-trained model would be worse and more expensive on day one and a distraction every day after.
How do captions get word-level timing?
Forced alignment, not transcription. The narration text is authored, so transcribing our own synthesized speech means paying a model to guess words we already have - and the guesses caused real bugs (a character name misheard 30 times; Hindi captioned in Urdu script). Alignment is handed the words and returns their timings. Cheaper, faster (~0.8s vs 6–12s), and two whole bug classes stop being expressible.
Why FFmpeg instead of a hosted video-editing API?
Measured cost structure. Encoding is ~77% of a render's wall time, which means the pipeline's economics live in filter graphs we control: Ken Burns moves, karaoke ASS captions, audio ducking, transitions, overlays - all zero marginal cost once written. A hosted editor would put a per-minute price on exactly the part that scales. The price is that FFmpeg's sharp edges become your sharp edges (see the war stories).
How do you keep a recurring character looking the same across scenes?
Not by asking nicely. The script model was instructed to repeat the character's look verbatim in every scene prompt - measured compliance was 0 of 2 scenes. The fix is render-time injection: the canonical description is prepended to every scene's image prompt by the pipeline itself, so an edited, re-rolled or paraphrased prompt still carries the character. Instructions decay; injection is deterministic.
What was the single most expensive default?
FFmpeg's amix normalize. It scales every input by 1/n, and two stacked mixes (music under voice, sound design over the timeline) attenuated an 11-scene video by roughly 20 dB - worse the longer the video, because n was the scene count. Months of 'why is this quiet' with no error anywhere. normalize=0, explicit gains and a limiter; measured back to -18 dB with no clipping.
How does the caching work?
Content-addressed at the asset level: voiceover audio is keyed by (voice, language, text), images by their prompt and dimensions, motion clips by the producing engine plus the source image key. Editing one narration line re-renders the video but only re-buys that one line's audio - measured on a real project: the re-render used 0 new voice characters, AI cost fell 69%. It is also why the product edits inputs and re-renders instead of patching finished files like a browser NLE.
Is the premium video model worth it?
Not the way you'd assume - this was measured, and the prediction driving the test was wrong. Same script rendered twice: budget engine on every scene versus premium on key scenes came out at 1.02 vs 0.99 whole-video motion (mean frame-to-frame luma delta) for 3× the credits and 22× the cash. Premium clips individually are 2.6× more dynamic; they just cover a third of the scenes. The useful finding: the optimum is per-scene - premium on the hook where retention is decided, budget elsewhere - so that is what the product ships.
How is multi-tenancy enforced?
EF Core global query filters keyed on the tenant claim, so isolation is the default and forgetting it is hard. The sharp edge is IgnoreQueryFilters(): it drops the WHOLE filter, including a soft-delete condition that lives alongside the tenant clause - which once meant a deleted video's public share page kept serving. The rule now: every IgnoreQueryFilters call restates every condition the filter was carrying, and background jobs always pass an explicit tenant id.
How do you test video output? Asserting on pixels sounds miserable.
Never 'build succeeded', and rarely raw pixel asserts. The toolkit: frame extraction and actually looking (caught fonts that were installed but drew nothing), dB measurement (caught the amix bug), and discriminating pairs - design the check so the old code would fail it. Example: an idle-bob animation was proven by comparing frames at the bob's peak (nonzero diff) AND at its full period (byte-identical, exactly where sin returns to zero) - deterministic phase means the diff is the motion, not noise.
Nine languages - what actually made that hard?
Fonts and script identity, not translation. A Devanagari font with no Latin glyphs turned every brand name into tofu boxes, so font choice became per-STRING (script is a property of the text, not the render), with glyph-level fallback for mixed-script lines. Then Whisper transcribed Hindi narration in Urdu script - same spoken language, different alphabet, non-deterministic. Fix: never let a model auto-detect something you already know; the language is pinned on every call.
Ask a technical question
Asking needs a (free) account - that's the trade for a real answer.
Sign up free & ask