Repository navigation
Description
A REQ or EVENT that a client sends as soon as its WebSocket opens is often lost. The relay sends nothing back: no EOSE or CLOSED for the REQ, no OK for the EVENT. The client can’t tell that it was lost, so a subscription stays silent and a publish waits until it times out. Every client sends its first REQs and EVENTs on open, and a reconnecting client sends all of them at once, so this affects a large share of connections.
Cause
WebSocketServerAdapter.onConnection (src/adapters/web-socket-server-adapter.ts) awaits the connection rate-limit check before it creates the WebSocketAdapter:
if (await isRateLimited(remoteAddress, currentSettings)) { … }
this.webSocketsAdapters.set(client, this.createWebSocketAdapter([client, req, this]))
The WebSocketAdapter constructor is what attaches client.on('message', …). isRateLimited reads its counters from the rate limiter’s store (Redis by default), so the await takes a round trip. A frame that arrives in the meantime is emitted by ws to no listener, and Node’s EventEmitter drops it without an error. Whether a connection loses its first messages depends on whether the client’s frame or the store’s reply arrives first, which is why only some connections are affected.
The connection rate limits are on in default-settings.yaml (limits.connection.rateLimits), so a default install has the race.
To reproduce
nostream 3.11.0 (ghcr.io/cameri/nostream:latest) with Postgres and Redis, default settings, relay on port 7784. This Node 22 script opens connections one after another and sends a REQ delayMs after each open:
// node first-msg.mjs ws://127.0.0.1:7784 0 10
const [url, delayMs = '0', n = '6'] = process.argv.slice(2)
function once(i) {
return new Promise((resolve) => {
const ws = new WebSocket(url)
const t0 = Date.now()
let answered = null
ws.onopen = () => setTimeout(() => ws.send(JSON.stringify(['REQ', 'r' + i, { kinds: [1], limit: 1 }])), Number(delayMs))
ws.onmessage = (m) => { const msg = JSON.parse(m.data); if (!answered && ['EOSE', 'CLOSED', 'NOTICE'].includes(msg[0])) answered = ${msg[0]} after ${Date.now() - t0} ms }
setTimeout(() => { ws.close(); resolve(conn ${i}: ${answered ?? 'UNANSWERED'}) }, 3000)
})
}
for (let i = 0; i < Number(n); i++) console.log(await once(i))
SetupREQs answeredDefault settings, REQ sent on open2 of 6, and 7 of 10 in a second run (no connection was rate-limited or terminated)Default settings, REQ sent 1 s after open4 of 4Client’s address added to limits.connection.ipWhitelist, so the check returns without touching Redis8 of 8With the fix in the linked PR, default settings10 of 10EVENTs are affected the same way: in a test harness publishing on fresh connections, 2 of 6 EVENTs got no OK and were never stored.
Expected behavior
Every message the client sends after the socket opens is handled, or answered with a CLOSED/NOTICE/OK saying why not.
Environment
• nostream 3.11.0, Docker image ghcr.io/cameri/nostream:latest, Postgres 16, Redis 7
• Clients: a plain Node 22 WebSocket client (above) and Quartz’s Kotlin client
Activity
cameri commentedon Nov 3, 2026
@vitorpamplona thanks for the detailed issue
chappie-daemon commentedon Nov 4, 2026
Independent reproduction on a second deployment, with one extra measurement that separates the cause from look-alike symptoms.
Environment. Two different instances, both default settings with connection rate limits on: the reporter's relay at 3.11.0 and a separate container running 3.8.0. The 3.8.0 instance answered with the same script from the issue, unchanged, against
ws://<host>:8008:A second pass of 10 connections lost 1. Control with the REQ delayed 1000 ms after
onopen: 4 of 4 answered, every time.Dropped frame, live socket. For each connection I sent a REQ immediately and a second REQ on the same socket 600 ms later, and recorded which subids came back:
On the connection that lost its first REQ, the same socket answered a REQ 600 ms later — no reconnect, no second handshake. That rules out the two explanations which look identical from a client's side:
client.terminate()and left nothing to answer on;What remains is a socket that was open and served later frames, with the earliest frame emitted before anything was listening for it.
The code still has the ordering.
origin/mainat 3.12.0,src/adapters/web-socket-server-adapter.tsline 137:precedes
this.webSocketsAdapters.set(client, this.createWebSocketAdapter([client, req, this])), and the adapter's constructor is what attachesclient.on('message', …).No fix is proposed here — this comment adds measurements only.
Ferryx349 commentedon Nov 4, 2026
@vitorpamplona Thanks again, we’ve implemented the fix in #1045.
Could you please try it with your reproduction against 3.12.1 / the latest image, and let us know if it works for you now? Specifically, we’d like to confirm that a REQ sent immediately on open always gets an EOSE/CLOSED, and an EVENT gets an OK. Rate-limited connections should still be terminated as before.
Any remaining failures would be very helpful.
vitorpamplona commentedon Nov 4, 2026
Yes, the fix works.
Ferryx349 commentedon Nov 4, 2026
Thank You!!