The content pipeline — built, and how it actually runs

This is the one fully-built pipeline. It takes a piece of content from a source to a classified row in the library. Everything below was verified against the running code on 17 August 2026 — stage list, routing, and gates all read from the files that execute them.

The path, in plain words

1. A source is scanned. One dispatcher decides which collector handles a link — five platforms each have their own collector (Instagram, TikTok, Twitter/X, and the Meta and LinkedIn ad libraries), and websites, newsrooms, and vendor portfolios each route to theirs.
2. The piece is ingested with the facts that need no judgment: where it came from, what kind of file it is, which page surface. The channel is derived by a database rule — never guessed.
3. The fifteen-stage spine runs: queued · download · normalize · extract frames · transcribe · identify music · classify · analyze · embed · index · extract scenes · validate · post-process · synthesize · classify scenes. Media-heavy stages run only for the formats that need them.
4. The analyzer is format-aware: text, image, and video each get the prompt their route resolves to (see routing & prompts).
5. Everything written resolves onto the registry — exact match, then alias, then close match, then flagged for review. A human's correction is never overwritten.
6. Two gates decide what publishes. A confidence floor parks anything the analyzer is unsure about. Above it, the entity's automation setting decides: fully autonomous work publishes, supervised work waits for a person, and onboarding work always parks for review.

What keeps it safe

• The master switch is a single database flag, and it is off. Nothing runs on a schedule until it is deliberately turned on.
• Every model call logs its cost to one table, so spend is always answerable with one query.
• The scene-indexing stage is manual-only by standing decision — it runs when a person approves it, never automatically.

The honest edges

• Scale: the spine has processed a fraction of the video library; most videos have no moments yet. That is scheduled work, not a defect.
• One brand dominates: most of the library is a single brand's content, because collection has focused there while the model settles.

Where to look next

• Routing & prompts — how a piece gets its prompt.
• Continuous monitoring — what re-scans the sources.
• The state of the data — live counts, generated.