Skip to content

Reader: concurrent scan with a confined arena per worker thread #505

Open

Description

@dfa1

Parked design for a concurrent reader scan. Follow-up to #474 (writer side, shipped).

Reference behavior

  • Rust (vortex-layout/src/scan/scan_builder.rs): the scan is split into row-range tasks (SplitBy), each spawned onto the session runtime, up to concurrency * num_workers in flight (default 4 per worker). ordered (default) uses .buffered(n), otherwise .buffer_unordered(n). A per-split map function runs on the spawned task.
  • .NET (Vorticity): MaxDegreeOfParallelism per session or per scan, MaxConcurrentReads for I/O, shared SegmentCache, concurrent scans over one open file.
  • vortex-java today: ScanIterator is single-threaded, every arena is ofConfined.

Proposed shape

A confined Arena per worker thread, so the callback runs on the worker and the caller never touches decoded memory from another thread:

scan.parallel(executor)            // executor + bounded in-flight window on ScanOptions
    .map(chunk -> r)               // runs on the worker; chunk and its arena close on return
  • R must not reference chunk memory (aggregates, copied-out values); Javadoc states the contract.
  • Results delivered ordered (Rust buffered, default) or unordered (buffer_unordered).
  • The file mapping becomes ofShared (one close at the end), since workers must read it. Alternative: each worker maps its own view.
  • scan() / ScanIterator / Chunk lifetime rules stay unchanged.

Pros

  • Rust-shaped (map on the spawned task, ordered/unordered, bounded window), so parity with the reference.
  • Confined arenas keep the cheap alloc/close path; Arena.ofShared().close() needs a cross-thread handshake per chunk.
  • No change to the existing single-threaded API or its safety checks.
  • Same bounded-window idea as the writer's pending queue / MAX_IN_FLIGHT_SEGMENTS.
  • Overlaps decode with consumption; wide files and slow consumers benefit most.

Cons

  • No ordered, parallel, zero-copy Iterator<Chunk> for a consumer thread: a chunk decoded on a worker cannot be read elsewhere. That case would need a shared per-chunk arena (slower close, and relaxes the "second next() while a Chunk is open is rejected" rule).
  • Callback API is less ergonomic than an iterator; callers who must keep data have to copy out.
  • Mapping arena goes shared; any cross-thread use after close() becomes a hazard to test (ADR 0003 security contract still applies on worker threads: only VortexException may escape).
  • Worker exceptions must be propagated and in-flight work cancelled/drained on failure (the writer's discardPending equivalent).
  • Needs A/B on all 7 real files (RealFileScanBenchmark, interleaved): memory-bandwidth-bound scans may not scale, and oversubscribing the pool gains nothing.

Out of scope / not planned

  • Per-column parallel decode within a chunk: not a Rust pattern, so it would be a deliberate divergence needing an explicit decision.
  • Parallel I/O for VortexHttpReader (concurrent range requests): independent of this, worth its own ticket if pursued.
  • Writer follow-ups (parallel scheme trials in CascadingCompressor.chooseBest): comment on Writer: compress columns/chunks concurrently #474.

Activity

  1. added
    enhancementNew feature or request
    PARKEDdo not yet merge
    and removed
    PARKEDdo not yet merge
    on Oct 9, 2026
  2. dfa1 commented on Oct 24, 2026

    @dfa1
    OwnerAuthor

    Revised design (2026-10-24): parallel decode-ahead behind the existing iterator

    Context: ParquetVsVortexReadBenchmark compares Hardwood's multi-core page decode (a fixed pool of availableProcessors() threads plus two virtual threads per column) with our single-threaded scan. Two taxi columns, single fork: on one CPU (-XX:ActiveProcessorCount=1) Vortex 163 vs Parquet 106 ops/s; on all CPUs Vortex 143 vs Parquet 165. The wall-clock gap is concurrency, not decode cost.

    Rust's default is an ordered stream whose splits are decoded on spawned tasks (.buffered(n)), which is an iterator with decode-ahead. So instead of a callback API, keep scan() / ScanIterator and make it decode ahead:

    1. "No ordered, parallel, zero-copy Iterator<Chunk>": measure Arena.ofShared() allocate+close against a chunk's decode (milliseconds) before ruling it out. If the close is cheap, use a shared arena per chunk. If not, one shared slab per in-flight slot, bump-allocated by the worker and reset when the consumer closes the chunk (one close per scan, a fresh arena for an oversized chunk). The "close a Chunk before the next next()" rule stays, since workers decode into other slots.
    2. "Callback API": there is none. ScanOptions.withExecutor(executor) (like WriteOptions#executor(), Writer: compress columns/chunks concurrently #474) turns on decode-ahead; existing code is unchanged. map(chunk -> r) is optional for aggregations.
    3. "Shared mapping, use-after-close": FFM's shared-arena close makes a late access throw IllegalStateException, never read freed memory. ScanIterator.close() / VortexReader.close() cancel and drain in-flight decodes before closing the mapping. Worker boundary maps anything else to VortexException (ADR 0003); a test closes the reader mid-scan.
    4. "Exceptions and cancellation": the writer's bounded futures queue: on failure cancel and drain the rest, next() rethrows the first error as VortexException.
    5. "May not scale": opt-in, window defaults to the executor's parallelism. Ship gate: RealFileScanBenchmark A/B on all seven files (sequential vs parallel), plus single-core numbers; one worker already overlaps decode with consumption.

    Order: shared-arena close cost first, then decode-ahead behind withExecutor, then the A/B and the taxi two-column read against Parquet on all cores.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions