Run on more than one instance

Preview The background jobs (@kitr/jobs) and file uploads (@kitr/uploads) steps use packages that are not published to npm yet. Try them from a kitr checkout with KITR_PREVIEW=1; every other step uses released packages.

One instance is the right way to start, and a kitr app runs on one without any of this. The day you add a second, for uptime during deploys or for load, everything a package kept in the memory of its process starts to disagree: Ada's live update reaches only the clients on her instance, a rate limit lets a client through three times, a cached report cleared on one instance is still served by the other, a nightly job runs once per instance, and an upload saved to one disk is a 404 on the next request.

This guide takes acme-projects, the app from Build a full-stack app deployed in Deploy to production, and moves each of those into shared storage: Redis for events, cache entries, counters and the job queue, and an S3-compatible bucket for files. Each package keeps its in-memory default for development, so kitr dev still needs nothing but Node. At the end, two instances run side by side on your machine and you watch an event cross from one to the other.

Before you start

You need acme-projects building and starting with pnpm kitr start site, as at the end of Deploy to production. You need a Redis server. With Docker, this starts one:

docker run -d --name acme-redis -p 6379:6379 redis:7

Managed Redis works the same way: Upstash, Redis Cloud, ElastiCache, Fly's or Railway's Redis. Copy its connection string; a rediss:// URL turns on TLS.

kitr's Redis adapters take an ioredis client, which you install once:

pnpm add ioredis

Add the connection string to .env, and to the environment of every instance in production:

REDIS_URL=redis://localhost:6379

Find what lives in one process

Every package that keeps state starts with a store in memory, and each has a shared replacement behind the same interface. The code that uses the package does not change, only the option that picks the store:

Sessions need nothing: a signed cookie or a database session reads the same on any instance. The database is already shared. Logs and error reports leave the process as they are written.

Connect to Redis once

Every adapter takes a client you create, so the app opens one connection and shares it. Put it in its own module:

// server/redis.ts
import { Redis } from 'ioredis'

/** The app's one Redis connection, shared by the cache, rate limits, jobs and realtime. */
export const redis = new Redis(process.env.REDIS_URL ?? 'redis://localhost:6379', {
  // Fail a command after one retry instead of queueing it while Redis is down.
  maxRetriesPerRequest: 1,
})

The realtime adapter also needs a connection in subscriber mode, which can run no other command; it opens that one itself with redis.duplicate(). Everything else shares redis. ioredis pipelines commands over one connection, so a single client handles a busy instance.

Close the connection when the server stops, so a deploy does not cut a command in half. A provider's shutdown runs after in-flight requests have finished:

// server/providers/redis_provider.ts
import { defineProvider } from 'kitr'
import { redis } from '#server/redis'
import { realtimeAdapter } from '#server/realtime_adapter'

export default defineProvider({
  name: 'redis',
  async shutdown() {
    await realtimeAdapter.close?.()
    await redis.quit()
  },
})

Add it last to providers in kitrrc.ts, so every other provider has shut down first.

Share live updates

Grace's projects page subscribes to teamProjectsChannel, and Ada's action emits created on it. With two instances, Grace's socket may be on instance A while Ada's request is served by instance B. Create the realtime adapter:

// server/realtime_adapter.ts
import { createRedisRealtimeAdapter } from '@kitr/realtime/server'
import { redis } from '#server/redis'

export const realtimeAdapter = createRedisRealtimeAdapter(redis, { prefix: 'acme-projects:rt:' })

Pass it to the socket link in server/routes.ts:

// server/routes.ts
// ...
import { teamProjectsChannel } from '#app/channels/projects'
import { realtimeAdapter } from '#server/realtime_adapter'

export const routes = defineRoutes([api, realtime], (router) =>
  router.group('/api', (r) =>
    r
      // ...endpoints as before
      .ws('/ws', { channels: [teamProjectsChannel], adapter: realtimeAdapter }),
  ),
)

The action that emits stays as it was. On instance B, emit('created', project) publishes the event on the Redis channel acme-projects:rt:projects:{"teamId":"team_acme"}. Instance A subscribed to that channel when Grace's socket subscribed to the topic, so it receives the event and writes it to her socket. Each instance subscribes only to the topics its own clients watch, and drops a subscription when the last of them leaves.

A WebSocket stays on the instance it opened on, so the load balancer needs no sticky sessions. The SSE transport does not either: a control POST that lands on another instance than its stream is forwarded through Redis. use(channel, params).size and connections still count this instance only. Realtime has the details.

Share the cache

A cached report on instance A is a miss on instance B, and invalidateTags() on B leaves A serving the stale entry. Point the cache at Redis:

// config/cache.ts
import { createRedisCacheDriver, defineConfig } from '@kitr/cache'
import { redis } from '#server/redis'

export default defineConfig({
  driver: createRedisCacheDriver(redis),
  prefix: 'acme-projects:',
  defaultTtl: 300,
})

Every remember(), get() and invalidateTags() now reads and writes the same entries. Redis expires them itself. clear() deletes only keys under the driver's base (kitr:cache: by default), never the whole Redis, so the queue and the counters sharing it are safe.

remember() still shares one run of a slow function per instance, not per cluster: a cold miss on three instances at once runs it three times. For a report that takes seconds, warm it from a scheduled job instead of on the first request.

In development, keep the memory driver by choosing on the environment:

// config/cache.ts
import { createMemoryCacheDriver, createRedisCacheDriver, defineConfig } from '@kitr/cache'
import { redis } from '#server/redis'

export default defineConfig({
  driver: process.env.REDIS_URL ? createRedisCacheDriver(redis) : createMemoryCacheDriver(),
  prefix: 'acme-projects:',
})

Share rate limits

With three instances, each counts on its own, so a client gets three times the limit, and a deploy resets every counter. Give the shield a Redis store, and trust the load balancer's X-Forwarded-For so each visitor gets their own counter:

// config/shield.ts
import { createRedisRateLimitStore, defineConfig } from '@kitr/shield'
import { redis } from '#server/redis'

export default defineConfig({
  headers: {},
  rateLimit: {
    max: 300,
    windowMs: 60_000,
    trustProxy: true,
    store: createRedisRateLimitStore(redis, { prefix: 'acme-projects:rl:' }),
  },
  csrf: { ignore: ['/webhooks'] },
})

Each window is one Redis key that expires with the window. A rateLimit() on a single route in server/routes.ts takes the same store as its store option. Only set trustProxy when every request passes through a proxy you control; Shield explains why.

Share the job queue

With the memory driver, a job dispatched on instance A runs on A or not at all, a restart loses the queue, and a schedule: '0 3 * * *' job runs once per instance. Move the queue to Redis:

// config/jobs.ts
import { createRedisJobsDriver, defineConfig } from '@kitr/jobs'
import { redis } from '#server/redis'
import { generateInvoice } from '#server/jobs/generate_invoice'
import { pruneSessions } from '#server/jobs/prune_sessions'

export default defineConfig({
  jobs: [generateInvoice, pruneSessions],
  driver: createRedisJobsDriver(redis, { prefix: 'acme-projects:jobs:', visibilityTimeout: 10 * 60_000 }),
})

Every instance's worker now pulls from one queue, and each job runs on exactly one of them. A worker that dies mid-job leaves the job reserved; after visibilityTimeout it goes back to the queue and another worker runs it, so keep the timeout above your longest job. Scheduled jobs queue once per run: before queuing prune_sessions at 03:00, each worker takes a Redis lock named after the job and the minute, and only the one that gets it queues the run. Give every instance the same TZ, because cron reads local time.

To keep jobs off the web instances, run separate workers with kitr jobs:work and set worker: false on the web servers; Jobs shows both.

Store uploads in a bucket

The disk driver writes to storage/uploads on the instance that received the file, so the next request for it on another instance is a 404, and a container restart deletes it. Store files in an S3-compatible bucket through unstorage instead:

pnpm add unstorage

// config/uploads.ts
import { defineConfig } from '@kitr/uploads'
import { createUnstorageUploadsDriver } from '@kitr/uploads/server'
import { createStorage } from 'unstorage'
import s3Driver from 'unstorage/drivers/s3'

const bucket = createStorage({
  driver: s3Driver({
    accessKeyId: process.env.S3_ACCESS_KEY_ID!,
    secretAccessKey: process.env.S3_SECRET_ACCESS_KEY!,
    endpoint: process.env.S3_ENDPOINT!,
    bucket: 'acme-projects-uploads',
    region: 'auto',
  }),
})

export default defineConfig({
  driver: createUnstorageUploadsDriver(bucket, { publicPath: '/api/uploads' }),
  maxSize: 10 * 1024 * 1024,
})

Cloudflare R2, AWS S3, MinIO and any other S3-compatible service work with the same driver. Files are still served through /api/uploads/:key, so the bucket stays private.

Run two instances side by side

Build once, then start two instances on different ports against the same Redis:

pnpm kitr build
KITR_SITE_PORT=3000 pnpm kitr start site &
KITR_SITE_PORT=3001 pnpm kitr start site &

Open http://localhost:3000/projects in one browser window as Grace and http://localhost:3001/projects in another as Ada, and create "Apollo launch" in Ada's window.

Grace's list shows "Apollo launch" without a reload, although her socket is on port 3000 and Ada's form was served by 3001. Watch the event pass through Redis while you do it:

docker exec -it acme-redis redis-cli PSUBSCRIBE 'acme-projects:rt:*'

1) "pmessage"
2) "acme-projects:rt:*"
3) "acme-projects:rt:projects:{\"teamId\":\"team_acme\"}"
4) "{\"t\":\"ev\",\"ch\":\"projects\",\"p\":{\"teamId\":\"team_acme\"},\"ev\":\"created\",\"d\":{...}}"

In production, put the instances behind your host's load balancer. Fly.io, Railway, Render and Cloud Run scale a service by instance count; Kubernetes by replicas. Point the health check at /healthz, which config/server.ts serves, so an instance only gets traffic once it is up.

Plan for Redis going down

Redis is now part of the app. What each package does when it cannot reach it:

Use a managed Redis with a replica and automatic failover for production, and keep maxRetriesPerRequest low so requests fail fast instead of hanging while it fails over. Separate Redis databases or prefixes keep apps that share one Redis apart: every adapter takes a prefix or base.

Next steps

• Realtime: the adapter, SSE forwarding and writing your own adapter.
• Cache, Shield and Jobs: each driver's options.
• Deploy to production: the Docker image and hosts these instances run on.
• Uploads: the unstorage driver and its options.
• ioredis: TLS, Sentinel and Cluster connections.