Data stores
Three stores, three jobs, no overlap:
| Store | Job | Never used for |
|---|---|---|
| Firestore | Source of truth: users, catalogue, money, audit | Queues, ephemeral state |
| Firebase Realtime Database | Queues, worker status, pairing sessions | A second catalogue |
| Google Cloud Storage | Bytes: artwork, masters, HLS output | Queryable metadata |
What is stored where
Firestore owns truth, RTDB owns movement, Cloud Storage owns bytes.
The same system, laid out automatically. Use the controls to zoom, open it full screen, or download it for a deck.
Firestore
Constants live in src/db-core/paths.ts; helpers in src/db-core/doc.ts (toApiDoc synthesises
_id from the document ID and drops __v). Region nam5, native mode.
| Collection | Doc ID | Notes |
|---|---|---|
users | 32-hex id | Profile, firebaseUid, Stripe fields, fcmTokens, points, activeUntil |
emails | lowercased email | {userId} — login lookup and unique-email guard |
videos | legacy id | Metadata, counters, isReady, readinessState, transcode, deletion, searchTokens[] |
series | legacy id | totalSeasons, totalEpisodes, isReady |
seasons | seasonDocId(seriesId, seasonNumber) | seriesId, seasonNumber, totalEpisodes, isReady |
plans | legacy Mongo _id | priority, prices, features; _id preserved in responses |
payments | auto ID | Provider reference, amount (number in new writes), method, date |
videoFeedback | auto ID | {eventName, videoId, date, deviceId, duration, progression} |
admins | legacy ObjectId string | bcrypt password, role, isActive. The doc ID keeps old JWTs valid. |
auditLogs | auto ID | {adminId, adminEmail, action, targetType, targetId, details, ip, userAgent, at} |
staging | fileId from addVideo | Upload progress, per-kind image state, expiresAt +7 days |
notifications | auto ID | Admin bell log; append-only, no TTL |
paymentRefs | Flutterwave txRef | Correlation record, expiresAt +7 days |
webhookEvents | provider event key | Create-only dedupe, expiresAt +24 hours |
meta/bootstrap | bootstrap | Gates one-time seeding of plans and admins |
MyList | firebaseUid | Canonical items[{videoId, addedAt}]; reads also merge legacy movies[] |
viewEvents, viewDedup | auto / ${firebaseUid}_${videoId} | Top-10 aggregation; 30-minute dedupe |
User subcollections
| Path | Doc ID |
|---|---|
users/{uid}/likes, dislikes, views, shares, downloads | videoId |
users/{uid}/playlists/{name} and .../videos/{videoId} | playlist name, videoId |
users/{uid}/subscriptions/{autoId} | auto |
users/{uid}/devices/{deviceId} | deviceId |
users/{uid}/pointsHistory, locations | auto |
users/{uid}/progress/{videoId} | videoId |
Two rules that break builds
The Firestore client is never configured with ignoreUndefinedProperties. A write that
passes an undefined optional field throws. Build the payload without the absent keys, as the
signup path does. Adding the setting is a deliberate cross-API change, not a local fix.
_id is synthesised from the document ID. The videos, series, plans and admins
collections deliberately keep legacy Mongo IDs. Re-keying a document changes every URL and
invalidates every issued admin JWT.
Indexes and TTL
Eight composite indexes cover season/series/episode queries on videos and seasons, plus a
videos.videoId field override for collection-group queries. TTL policies for staging,
paymentRefs and webhookEvents are still outstanding, and notifications has none — it grows
without bound.
Realtime Database
RTDB is the control plane. Admin SDK access bypasses rules; client rules deny queue,
vpsStatus and flags.
| Path | Writer | Reader |
|---|---|---|
queue/transcode/{pushId} | admin API saveVideo | worker |
queue/transcodeFailed/{pushId} | worker | admin API consumer, ops |
queue/resolutionEvents/{pushId} | worker | admin API consumer |
queue/resolutionEventsFailed/{pushId} | admin API | ops |
queue/imageJobs, imageJobsFailed, imageEvents | legacy only | worker image lane, admin API |
vpsStatus/processing | worker | admin API (a string "true"/"false", not a boolean) |
vpsStatus/currentVideo, currentImageJob | worker | ops |
vpsStatus/heartbeat | worker | dashboard, ops |
vpsStatus/idleSince, shutdownPending | worker | admin API |
flags/shutdownCancel | admin API on enqueue | worker |
tv-pairing/sessions/{code} | TV, phone, webview | TV, phone, webview |
Every queue is a claim queue: consumed with child_added, removed after successful
handling. There is no lease. Two consumers on one queue process everything twice — which is why
the admin API is single-instance.
Dead letters (*Failed) are retained up to 100 entries and are what the notification feed turns
into video.failed messages.
Cloud Storage
Three buckets per environment, differing only by the -test / -prod suffix. Both APIs validate
the suffix against APP_ENV at boot.
| Bucket | Holds | Consumed by |
|---|---|---|
| images | Artwork. Sources are converted to WebP at ingest at <prefix>/<fileId>.webp; sources are deleted after conversion. | Admin web, clients via thumbnailUrl |
| input | Raw masters uploaded by the browser. Worker reads, then deletes. | Worker |
| output | HLS ladder per title and per rendition, plus trailer renditions. | gstv-multimedia through gcsfuse |
The worker and the streaming server see the buckets as directories thanks to gcsfuse. That
is the reason both containers need /dev/fuse and SYS_ADMIN, and the reason a local
npm start without a mount does not behave like production.
Media and metadata stay separate
A video's metadata and its bytes are joined by one string: the title's id.
- Firestore says which rungs exist (
transcode), and whether the title is live. - GCS holds the actual playlists and segments under
<videoId>/<resolution>/. - The URL contract (
/stream/videos/<videoId>/<resolution>/playlist.m3u8) is the only place the two meet.
That is why deleting a title is a cascade across all three stores, and why a partial delete leaves orphan bytes rather than a broken document.
Next: Notifications — how a background outcome reaches a human.