Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions .changeset/peering-on-demand.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
---
"nostream": minor
---

feat(peering): rate-limit on-demand requests from authenticated peers

Authenticated peers can request events outside their standing feed with a
normal `REQ`, `COUNT`, or ad-hoc `NEG-OPEN` session. Those requests continue to
use peer subscription and NIP-77 limits instead of client limits, and are now
bounded per peer pubkey by `limits.peer.request.rateLimits` (default 300
requests/minute across `REQ`, `COUNT`, and `NEG-OPEN`).

Refused `REQ`/`COUNT` calls receive `CLOSED` with a `rate-limited:` reason and
retry hint; refused `NEG-OPEN` sessions receive `NEG-ERR`. The limiter fails
closed when its store is unavailable, and unidentified clients are unaffected.
Per-peer request/event accounting and admin-console visibility are left to the
dedicated admin-console follow-up.
2 changes: 2 additions & 0 deletions CONFIGURATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -197,6 +197,8 @@ The settings below are listed in alphabetical order by name. Please keep this ta
| limits.message.rateLimits[].rate | Maximum number of messages during period. |
| limits.peer.message.rateLimits[].period | Rate limit period in milliseconds for authenticated peering peers. Separate from `limits.message`, so a peer feed never spends client message budget. |
| limits.peer.message.rateLimits[].rate | Maximum peer messages during period. Defaults to 6000/min. |
| limits.peer.request.rateLimits[].period | Rate limit period in milliseconds for on-demand peer `REQ`, `COUNT`, and `NEG-OPEN` requests. Counted per peer pubkey. |
| limits.peer.request.rateLimits[].rate | Maximum on-demand peer requests during period. Defaults to 300/min. |
| limits.peer.subscription.maxFilters | Maximum number of filters per peer subscription. Defaults to 100. Disabled when set to zero. |
| limits.peer.subscription.maxFilterValues | Maximum number of values allowed in each peer filter. Defaults to 10000. Disabled when set to zero. |
| limits.peer.subscription.maxLimit | Maximum REQ `limit` for peers. Defaults to 0 (no cap), so a peer can request a full feed. |
Expand Down
19 changes: 19 additions & 0 deletions docs/PEERING.md
Original file line number Diff line number Diff line change
Expand Up @@ -160,6 +160,25 @@ outbound sync path: the sync worker reconciles only the peer's advertised
scopes and drops fetched events that fall outside them. Inbound feed scope
enforcement remains part of the sync work that is still in progress.

## On-demand requests

Beyond the standing feed and periodic reconciliation, an authenticated peer may
ask for specific events with an ordinary `REQ`, `COUNT`, or an ad-hoc `NEG-OPEN`
session. This covers an event outside the advertised scope — for example the
root of a thread, or a deeper backfill — without changing either relay's feed
configuration.

These requests are served under the peer's normal subscription and NIP-77
limits, not client limits, and are additionally throttled per peer pubkey by
`limits.peer.request.rateLimits`. The default budget is 300 requests per minute
across `REQ`, `COUNT`, and `NEG-OPEN`; a refused `REQ`/`COUNT` receives
`CLOSED` with a `rate-limited:` reason and a retry hint, and a refused
`NEG-OPEN` receives `NEG-ERR`. The limiter fails closed if its store is
unavailable.

The same request pattern from a non-peer connection is handled as ordinary
client traffic and is not given the peer budget.

Only the public half of the keypair is ever visible on the wire. Inbound AUTH is
verified by `AuthMessageHandler`: event id, signature, timestamp, challenge, and
relay host. Outbound AUTH proves this relay's identity to the peer; it does not
Expand Down
8 changes: 8 additions & 0 deletions resources/default-settings.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -430,6 +430,14 @@ limits:
- description: 6000 peer messages/min
period: 60000
rate: 6000
request:
# On-demand REQ/COUNT/NEG-OPEN requests, counted per peer pubkey rather
# than per IP. Separate from the generic message bucket so a feed's raw
# traffic does not consume the on-demand budget.
rateLimits:
- description: 300 peer on-demand requests/min
period: 60000
rate: 300
message:
# period is milliseconds (60000 = one minute)
rateLimits:
Expand Down
2 changes: 2 additions & 0 deletions src/@types/adapters.ts
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,8 @@ export type IWebSocketAdapter = EventEmitter & {
drainAndClose(reason?: string): void
/** NIP-77 sessions of this connection. */
getNegentropySessions(): INegentropySessions
/** Removes a subscription and aborts any in-flight read still serving it. */
endSubscription(subscriptionId: string): void
}

export interface ICacheAdapter {
Expand Down
6 changes: 6 additions & 0 deletions src/@types/settings.ts
Original file line number Diff line number Diff line change
Expand Up @@ -170,6 +170,7 @@ export interface ClientLimits {
export interface PeerLimits {
subscription?: ClientSubscriptionLimits
message?: MessageLimits
request?: PeerRequestLimits
}

export interface MessageRateLimit extends RateLimit {
Expand All @@ -181,6 +182,11 @@ export interface MessageLimits {
ipWhitelist?: string[]
}

/** Bounds on on-demand peer requests (REQ, COUNT, NEG-OPEN). */
export interface PeerRequestLimits {
rateLimits?: RateLimit[]
}

export interface ConnectionLimits {
rateLimits: RateLimit[]
ipWhitelist?: string[]
Expand Down
2 changes: 1 addition & 1 deletion src/adapters/web-socket-adapter.ts
Original file line number Diff line number Diff line change
Expand Up @@ -417,7 +417,7 @@ export class WebSocketAdapter extends EventEmitter implements IWebSocketAdapter
return createNoticeMessage('rate limited')
}

private endSubscription(subscriptionId: SubscriptionId): void {
public endSubscription(subscriptionId: SubscriptionId): void {
this.abortInFlightSubscriptionReads(subscriptionId)
if (this.subscriptions.has(subscriptionId)) {
this.onUnsubscribed(subscriptionId)
Expand Down
8 changes: 4 additions & 4 deletions src/factories/message-handler-factory.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ import {
} from '../@types/repositories'
import { RedisAdapter } from '../adapters/redis-adapter'
import { getCacheClient } from '../cache/client'
import { getMasterDbClient } from '../database/client'
import { AuthMessageHandler } from '../handlers/auth-message-handler'
import { CountMessageHandler } from '../handlers/count-message-handler'
import { EventMessageHandler } from '../handlers/event-message-handler'
Expand All @@ -18,9 +19,8 @@ import { NegMsgMessageHandler } from '../handlers/neg-msg-message-handler'
import { NegOpenMessageHandler } from '../handlers/neg-open-message-handler'
import { SubscribeMessageHandler } from '../handlers/subscribe-message-handler'
import { UnsubscribeMessageHandler } from '../handlers/unsubscribe-message-handler'
import { eventStrategyFactory } from './event-strategy-factory'
import { getMasterDbClient } from '../database/client'
import { UserSubscriptionRepository } from '../repositories/user-subscription-repository'
import { eventStrategyFactory } from './event-strategy-factory'
import { rateLimiterFactory } from './rate-limiter-factory'
import { createSettings } from './settings-factory'
import { wotGraphServiceFactory } from './wot-graph-service-factory'
Expand Down Expand Up @@ -85,11 +85,11 @@ export const messageHandlerFactory =
case MessageType.CLOSE:
return new UnsubscribeMessageHandler(adapter)
case MessageType.COUNT:
return new CountMessageHandler(adapter, eventRepository, createSettings)
return new CountMessageHandler(adapter, eventRepository, createSettings, rateLimiterFactory)
case MessageType.AUTH:
return new AuthMessageHandler(adapter, createSettings)
case MessageType.NEG_OPEN:
return new NegOpenMessageHandler(adapter, eventRepository, createSettings)
return new NegOpenMessageHandler(adapter, eventRepository, createSettings, rateLimiterFactory)
case MessageType.NEG_MSG:
return new NegMsgMessageHandler(adapter, createSettings)
case MessageType.NEG_CLOSE:
Expand Down
29 changes: 22 additions & 7 deletions src/handlers/count-message-handler.ts
Original file line number Diff line number Diff line change
@@ -1,17 +1,20 @@
import { equals, uniqWith } from 'ramda'

import { IWebSocketAdapter } from '../@types/adapters'
import { Factory } from '../@types/base'
import { IMessageHandler } from '../@types/message-handlers'
import { CountMessage } from '../@types/messages'
import { IEventRepository } from '../@types/repositories'
import { Settings } from '../@types/settings'
import { SubscriptionFilter, SubscriptionId } from '../@types/subscription'
import { IRateLimiter } from '../@types/utils'
import { WebSocketAdapterEvent } from '../constants/adapter'
import { createLogger } from '../factories/logger-factory'
import { countFilterValues } from '../utils/filter'
import { createClosedMessage, createCountResultMessage } from '../utils/messages'
import { isCountAuthorized } from '../utils/nip42'
import { subscriptionLimitsFor } from '../utils/peer-limits'
import { getPeerRequestRetryAfterMs } from '../utils/peer-requests'

const debug = createLogger('count-message-handler')

Expand All @@ -20,22 +23,37 @@ export class CountMessageHandler implements IMessageHandler {
private readonly webSocket: IWebSocketAdapter,
private readonly eventRepository: IEventRepository,
private readonly settings: () => Settings,
private readonly rateLimiter?: Factory<IRateLimiter>,
) {}

public async handleMessage(message: CountMessage): Promise<void> {
const queryId = message[1]
const countEnabled = this.settings().nip45?.enabled ?? true
if (!countEnabled) {
this.webSocket.emit(WebSocketAdapterEvent.Message, createClosedMessage(queryId, 'COUNT is disabled by relay configuration'))
this.webSocket.emit(
WebSocketAdapterEvent.Message,
createClosedMessage(queryId, 'COUNT is disabled by relay configuration'),
)
return
}

const peerPubkey = this.webSocket.getPeerPubkey()
if (peerPubkey && this.rateLimiter) {
const retryAfterMs = await getPeerRequestRetryAfterMs(this.rateLimiter, this.settings(), peerPubkey, 'COUNT')
if (retryAfterMs !== undefined) {
this.webSocket.emit(
WebSocketAdapterEvent.Message,
createClosedMessage(queryId, 'rate-limited: peer request limit exceeded', String(retryAfterMs)),
)
return
}
}

// Some clients send the same filter more than once. Bound the raw list
// before deduplicating it, since uniqWith() is O(n^2) and the schema no
// longer caps filter counts so peer limits can be larger.
const incomingFilters = message.slice(2) as SubscriptionFilter[]
const maxFilters =
subscriptionLimitsFor(this.settings(), this.webSocket.getPeerPubkey() !== undefined)?.maxFilters ?? 0
const maxFilters = subscriptionLimitsFor(this.settings(), peerPubkey !== undefined)?.maxFilters ?? 0
if (maxFilters > 0 && incomingFilters.length > maxFilters) {
this.webSocket.emit(
WebSocketAdapterEvent.Message,
Expand Down Expand Up @@ -68,10 +86,7 @@ export class CountMessageHandler implements IMessageHandler {
}

private canCount(queryId: SubscriptionId, filters: SubscriptionFilter[]): string | undefined {
const subscriptionLimits = subscriptionLimitsFor(
this.settings(),
this.webSocket.getPeerPubkey() !== undefined,
)
const subscriptionLimits = subscriptionLimitsFor(this.settings(), this.webSocket.getPeerPubkey() !== undefined)
const maxFilters = subscriptionLimits?.maxFilters ?? 0

if (maxFilters > 0 && filters.length > maxFilters) {
Expand Down
41 changes: 31 additions & 10 deletions src/handlers/neg-open-message-handler.ts
Original file line number Diff line number Diff line change
@@ -1,18 +1,21 @@
import { IWebSocketAdapter } from '../@types/adapters'
import { createNegErrMessage, createNegMsgMessage } from '../utils/messages'
import { getNip77Limits, isNip77Active, NEGENTROPY_FRAME_SIZE_LIMIT } from '../utils/nip77'
import { INegentropySessions } from '../@types/negentropy'
import { Negentropy, NegentropyStorageVector } from '../utils/negentropy'
import { IEventRepository, ReconciliationItem } from '../@types/repositories'
import { Factory } from '../@types/base'
import { IMessageHandler } from '../@types/message-handlers'
import { NegOpenMessage, OutgoingMessage } from '../@types/messages'
import { INegentropySessions } from '../@types/negentropy'
import { IEventRepository, ReconciliationItem } from '../@types/repositories'
import { Settings } from '../@types/settings'
import { SubscriptionFilter, SubscriptionId } from '../@types/subscription'
import { IRateLimiter } from '../@types/utils'
import { WebSocketAdapterEvent } from '../constants/adapter'
import { countFilterValues } from '../utils/filter'
import { createLogger } from '../factories/logger-factory'
import { countFilterValues } from '../utils/filter'
import { createNegErrMessage, createNegMsgMessage } from '../utils/messages'
import { Negentropy, NegentropyStorageVector } from '../utils/negentropy'
import { isReconciliationAuthorized } from '../utils/nip42'
import { getNip77Limits, isNip77Active, NEGENTROPY_FRAME_SIZE_LIMIT } from '../utils/nip77'
import { subscriptionLimitsFor } from '../utils/peer-limits'
import { getPeerRequestRetryAfterMs } from '../utils/peer-requests'

const logger = createLogger('neg-open-message-handler')

Expand All @@ -25,6 +28,7 @@ export class NegOpenMessageHandler implements IMessageHandler {
private readonly webSocket: IWebSocketAdapter,
private readonly eventRepository: IEventRepository,
private readonly settings: () => Settings,
private readonly rateLimiter?: Factory<IRateLimiter>,
) {}

public async handleMessage(message: NegOpenMessage): Promise<void> {
Expand All @@ -49,6 +53,9 @@ export class NegOpenMessageHandler implements IMessageHandler {

const { maxEventsPerSession, maxSessionsPerConnection, sessionIdleTimeoutMs } = getNip77Limits(settings)

// Claim the id before the first await. A NEG-CLOSE (or a replacement
// NEG-OPEN) that arrives while the limiter is pending then owns the id,
// and this handler must not open a session after it.
const token = sessions.reserve(subscriptionId, maxSessionsPerConnection)
if (!token) {
this.reject(
Expand All @@ -59,6 +66,23 @@ export class NegOpenMessageHandler implements IMessageHandler {
return
}

const peerPubkey = this.webSocket.getPeerPubkey()
if (peerPubkey && this.rateLimiter) {
const retryAfterMs = await getPeerRequestRetryAfterMs(this.rateLimiter, settings, peerPubkey, 'NEG-OPEN')
if (retryAfterMs !== undefined) {
sessions.endRead(token)
if (sessions.isCurrent(subscriptionId, token)) {
this.reject(sessions, subscriptionId, `rate-limited: retry after ${retryAfterMs} ms`)
}
return
}
}

if (!sessions.isCurrent(subscriptionId, token)) {
sessions.endRead(token)
return
}

let items: ReconciliationItem[]
try {
// One row past the limit tells us the set is too big without loading all of it.
Expand Down Expand Up @@ -123,10 +147,7 @@ export class NegOpenMessageHandler implements IMessageHandler {
initialMessage: Buffer,
): string | undefined {
const settings = this.settings()
const subscriptionLimits = subscriptionLimitsFor(
settings,
this.webSocket.getPeerPubkey() !== undefined,
)
const subscriptionLimits = subscriptionLimitsFor(settings, this.webSocket.getPeerPubkey() !== undefined)

const maxFilterValues = subscriptionLimits?.maxFilterValues ?? 0
if (maxFilterValues > 0 && countFilterValues(filter) > maxFilterValues) {
Expand Down
26 changes: 24 additions & 2 deletions src/handlers/subscribe-message-handler.ts
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ import { Settings } from '../@types/settings'
import { SubscribeMessage } from '../@types/messages'
import { WebSocketAdapterEvent } from '../constants/adapter'
import { subscriptionLimitsFor } from '../utils/peer-limits'
import { getPeerRequestRetryAfterMs } from '../utils/peer-requests'

const logger = createLogger('subscribe-message-handler')

Expand Down Expand Up @@ -53,11 +54,32 @@ export class SubscribeMessageHandler implements IMessageHandler, IAbortable {
const subscriptionId = message[1]
this.activeSubscriptionId = subscriptionId
const incomingFilters = message.slice(2) as SubscriptionFilter[]
const peerPubkey = this.webSocket.getPeerPubkey()

if (peerPubkey) {
const retryAfterMs = await getPeerRequestRetryAfterMs(this.rateLimiter, this.settings(), peerPubkey, 'REQ')
Comment thread
Ferryx349 marked this conversation as resolved.
Resolved

// CLOSE aborted this handler while the limiter was pending. A newer REQ
// may already own the id; this stale result must not touch it.
if (this.abortController.signal.aborted) {
return
}

if (retryAfterMs !== undefined) {
// A reused subscription id must not keep its old filters/read alive
// after we tell the peer this request was refused.
this.webSocket.endSubscription(subscriptionId)
Comment thread
Ferryx349 marked this conversation as resolved.
Resolved
this.webSocket.emit(
WebSocketAdapterEvent.Message,
createClosedMessage(subscriptionId, 'rate-limited: peer request limit exceeded', String(retryAfterMs)),
)
return
Comment thread
Ferryx349 marked this conversation as resolved.
Resolved
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Resolved
}
}

// Bound before uniqWith(): deduplicating an unbounded list is O(n^2). The
// schema no longer caps filter counts so peer limits can be larger.
const maxFilters =
subscriptionLimitsFor(this.settings(), this.webSocket.getPeerPubkey() !== undefined)?.maxFilters ?? 0
const maxFilters = subscriptionLimitsFor(this.settings(), peerPubkey !== undefined)?.maxFilters ?? 0
if (maxFilters > 0 && incomingFilters.length > maxFilters) {
const reason = `Too many filters: Number of filters per susbscription must be less then or equal to ${maxFilters}`
logger('subscription %s with %o rejected: %s', subscriptionId, incomingFilters, reason)
Expand Down
5 changes: 2 additions & 3 deletions src/handlers/unsubscribe-message-handler.ts
Original file line number Diff line number Diff line change
@@ -1,12 +1,11 @@
import { IMessageHandler } from '../@types/message-handlers'
import { IWebSocketAdapter } from '../@types/adapters'
import { IMessageHandler } from '../@types/message-handlers'
import { UnsubscribeMessage } from '../@types/messages'
import { WebSocketAdapterEvent } from '../constants/adapter'

export class UnsubscribeMessageHandler implements IMessageHandler {
public constructor(private readonly webSocket: IWebSocketAdapter) {}

public async handleMessage(message: UnsubscribeMessage): Promise<void> {
this.webSocket.emit(WebSocketAdapterEvent.Unsubscribe, message[1])
this.webSocket.endSubscription(message[1])
}
}
Loading
Loading