Skip to main content

Watch a video go live

This is the whole publish path as one scene: the admin at the desk, the API, Cloud Storage, the queue, the single ephemeral worker, the readiness gate, the device that plays the result and the bell that tells the admin it happened.

Flow · ingest

How a video becomes live, and how the worker disappears

A real-world view of the publish path: the admin, the API, storage, the queue, one ephemeral VM and the devices. Every block is clickable; the journey runs in eleven steps.

The journey
RESTdirect upload · 8 MiB chunksqueue/transcodeclaim jobrenditionsresolutionConsumer → readinessvideo.live · SSEidle 5 min / 24 h → self-deletevideo.liveNotification bellvideo.live · SSECloud Storageimages · input · outputThe admingstv-admin-webADMIN APIAdmin APIgstv-admin-apiQUEUE · RTDBTranscode queuequeue/transcodeHLS LADDERWORKER · <env>Worker VMgstv-vps-<env>-workerReadiness gateevaluateVideoReadinessFirestoresource of truthREADY?videos/{id}the catalogue recordThe audienceTV · tablet · phone
Zoom 100 percent
Step 1 / 11

The admin opens Add video in gstv-admin-web. Four steps: video info, two artwork crops, trailer and feature, then upload.

REST direct to GCS RTDB queue control loop SSE

How to use it​

  • Step through the journey. The eleven chips run in order: wizard, sign, upload, artwork, save, VM up, boot, transcode, ready, live, shutdown. Play walks them automatically; Back and Next move one at a time.
  • Click any object for its plain-language story and its technical facts — ports, key files, what it owns, and how to know it is healthy. Every block carries its documentation link.
  • Zoom, fullscreen, export. The controls in the corner zoom in and out, fit the scene, open it full screen, and download it as SVG or PNG. Keyboard: +, -, 0, F, Esc.
  • Parallax. Move the pointer across the scene; the floor and the machines shift at different rates to show depth. Reduce-motion settings turn the movement and the fan off.

The eleven steps, in terms of what changes on disk​

#StepWhat becomes true
01WizardThe admin has filled video info, two artwork crops, a trailer and a feature. Nothing is written yet.
02Signstaging/{fileId} exists with a seven-day expiresAt; signUpload returns a signed PUT (images) or a resumable session (video).
03UploadFour objects land directly in the buckets, in 8 MiB chunks: thumbnail, thumbnailPreview, trailer, video. The API never carries bytes.
04ArtworkcompleteUpload has already produced <prefix>/<fileId>.webp at quality 82 and deleted the source. The crop rules live in two repos and are changed together.
05SavesaveVideo verifies every object, creates videos/{id}, writes flags/shutdownCancel, and pushes queue/transcode/{pushId}.
06VM upThe scheduler sees a non-empty queue and ensures exactly one gstv-vps-<env>-worker exists. It creates and deletes; it never stops or starts.
07Bootstartup-script.sh asserts metadata, environment identity, buckets, the baked key, the RTDB host and the timers. Any failure deletes the VM. Then gcsfuse mounts input and output.
08TranscodeOne job at a time — trailer first, then feature. The master playlist.m3u8 is written before the renditions; each finished rung emits a queue/resolutionEvents child.
09ReadyevaluateVideoReadiness runs one transaction requiring transcode.status === "completed", a non-empty trailerResolution and thumbnail.thumbnailUrl. Only then is staging removed and isReady set.
10Livenotifications holds video.live; the SSE stream toasts the bell and the catalogue's onSnapshot listener picks the document up without a refresh.
11ShutdownFive idle minutes, or the 24-hour cap, and the worker deletes its own instance. The queue child is removed only after success, so a crash is safe to retry.

Why the worker is drawn as a tower that disappears​

The VM is not a service. It has no uptime to protect, no rolling deploy and no health endpoint the platform polls. It is a machine that exists for the duration of a queue, and the queue is the only thing keeping it alive:

  • Fixed name per environment — gstv-vps-<env>-worker, filtered on labels.env, so the scheduler can always answer "is there a worker?" without listing every instance.
  • Create and delete only. The scheduler never stops or starts a VM; a stopped VM is a half-state that neither the queue nor the operator can reason about.
  • Two exits. Idle for five minutes (gstv-idle-threshold-ms=300000), or 24 hours of wall-clock time. Both end in self-deletion, not in a shutdown callback the API has to trust.

That is also why the 212-VM incident happened: the create gate counted a queue nobody could drain. A worker is only as ephemeral as its queue is finite.

Deploy the consumer first

When a change touches both the worker and the API, deploy the worker image first. A producer running ahead of its consumer is an infinite work signal, not a stuck job — and this scene's sixth step will keep creating machines forever.

Where failures show up in the scene​

FailureWhat the scene would showWhat the code does
A chunk upload is interruptedThe upload arc stalls between the admin and the bucketThe resumable session resyncs on 308; nothing is enqueued until saveVideo verifies all four objects.
The image conversion failsArtwork never finishes; save never runs422 with the error in staging; re-upload. 409 means the source changed mid-conversion.
The worker dies mid-jobThe VM blinks out; the queue keeps the childThe child is removed only after success, so the next VM claims it again.
Readiness is refusedThe gate stays closed; the title card never turns liveThe transaction requires all three conditions; markVideoFailed writes video.failed instead.

Related: Upload to publish has the full contract, and Notifications explains the SSE stream behind the bell.