Skip to content

Service Topology ​

Who talks to whom, over what, and with which credential. This is the page to read before adding a service or debugging a cross-service failure.

Communication matrix ​

FromToTransportAuthPurpose
MobileapiHTTPS RESTAuthorization: Bearer <JWT>Everything
MobileesuSocket.IO (WSS)JWT in the handshakeRealtime
MobileLiveKitWebRTCLiveKit access token minted by the APIVoice rooms
MobileMuxHLS / RTMPPublic playback ID; stream key for broadcastLivestreaming
WebapiHTTPS REST—Public data
apiesuHTTP POST /admin/emit/*X-Realtime-Admin-TokenFan out realtime events
esuapiHTTP GET /api/v1/{dm,room}/:id/authzForwards the user's JWTAuthorize a join
ogunesuHTTP POST /admin/emit/*X-Realtime-Admin-TokenMedia-ready notification
apiRedisBullMQpasswordEnqueue media jobs
ogunRedisBullMQpasswordConsume media jobs
esuRedisSocket.IO Redis adapterpasswordCross-pod pub/sub
apiRedisioredispasswordFeed / trending cache
all servicesPostgreSQLPrismaconnection stringData
RevenueCatapiHTTPS webhookshared secretPurchase events
MuxapiHTTPS webhooksigning secretAsset and livestream state
anansiStripeHTTPS APIsecret keyTop-ups and creator transfers

The interservice channel ​

esu exposes an admin router at /admin, guarded by a shared token (apps/esu/src/interservice/adminRoutes.ts):

ts
router.use(requireAdminToken);        // checks X-Realtime-Admin-Token

router.post("/emit/dm/message/new", …);
router.post("/emit/dm/read", …);
router.post("/emit/room/message/new", …);
router.post("/emit/channel/room/created", …);
router.post("/emit/notification/new", …);
router.post("/emit/media/processing/update", …);

Each handler does one thing: emit a Socket.IO event into a room. No database access, no authorization beyond the token.

The API's helper adds the /admin prefix itself (apps/api/src/lib/interservice.ts):

ts
fetch(`${process.env.REALTIME_SERVICE_URL}/admin${path}`, …)

Ogun's helper is identical (apps/ogun/src/utils/interservice.ts) — keep the two copies in sync.

Both normalise the base URL before use, so REALTIME_SERVICE_URL works whether or not it already carries a trailing /admin:

ts
const base = (process.env.REALTIME_SERVICE_URL ?? "").replace(/\/+$/, "").replace(/\/admin$/, "");
fetch(`${base}/admin${path}`, …)

Prefer the bare service root (http://playpalz-esu:4010) in new configuration.

Previously: REALTIME_SERVICE_URL meant two different things

The helpers used to disagree about whether the env value included /admin, while the manifests set one value for both. That was correct for the API and wrong for ogun, so the "your photo finished processing" event silently 404'd in production; locally the polarity reversed and the API POSTed to /admin/admin/…. .catch(() => {}) at every call site hid both.

Two things stop this recurring: the normalisation above, and the helpers now logging and throwing on a non-2xx response instead of returning it unexamined.

Realtime room naming ​

esu groups sockets into rooms (apps/esu/src/rooms/roomNames.ts). Every emit targets one:

RoomContainsUsed for
user:<userId>Every socket for one userNotifications, media-ready, presence-targeted events
dm:<conversationId>Participants of a direct conversationMessages, read receipts, typing
room:<roomId>Members currently in a channel roomMessages, typing
channel:<channelId>Channel membersRoom created / updated, member joined / left

A socket joins user:<id> automatically on connect. All other joins are explicit and authorized via the callback to the API.

Why esu calls back into the API ​

It would be faster for esu to query Postgres directly for "is this user allowed in this conversation". It does not, on purpose:

  • Access rules for channels involve subscription state, blocks, and membership. Duplicating them would mean two implementations that drift.
  • esu stays free of business logic, so it can be restarted, scaled, or replaced without coordinating a data-model change.

The cost is one HTTP round trip per join. That is acceptable — joins are rare compared to messages.

Scaling characteristics ​

ServiceScales byConstraint
apiHorizontal, statelessPostgres connection count
esuHorizontal via the Redis adapterIngress needs sticky sessions for long-polling — hence the ESUSESSION cookie on the socket ingress
ogunHorizontal, more workers = more throughputCPU-bound (Sharp); Mux is rate-limited
anansiSingle effective runnerGuarded by a Redis distributed lock; a second replica would double-pay creators
igdb-heartbeatSingleCron
webHorizontal, stateless—

anansi and igdb-heartbeat must not be scaled up casually

Both are cron-driven. anansi uses a Redis lock to make duplicate runs safe, but that lock is the only thing standing between you and duplicate Stripe transfers. Treat replica counts for those two deployments as a deliberate decision, not a knob.

Failure modes ​

If this diesWhat breaksWhat keeps working
apiEverythingNothing meaningful
esuLive message delivery, typing, presenceSending and reading messages (the API persists them; clients see them on refresh)
ogunThumbnails and video transcodingUploads still succeed; media stays pending and processes when the worker returns
anansiPayout runsThe product; earnings still accrue in the database
igdb-heartbeatGame catalog freshnessEverything
RedisQueue, realtime fan-out, cachingDegraded but mostly functional — cache misses fall through to Postgres
PostgreSQLEverythingNothing

The useful property here is that esu and ogun are both deferrable: losing them degrades the experience but does not lose data.

Internal documentation — PlayPalz platform