Skip to content

perf(identity): bound per-connect procfs work in binary identity resolution - #5213

Merged
FrostGod merged 1 commit into
ai-openshell:mainfrom
EmilienM:perf/5182-binary-identity-procfs/EmilienM
Nov 6, 2026
Merged

FrostGod merged 1 commit into
ai-openshell:mainfrom
EmilienM:perf/5182-binary-identity-procfs/EmilienM

Conversation

@EmilienM

@EmilienM EmilienM commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Cut the procfs and hashing work openshell-binary-identity does per external connect, all of it synchronous on the sandbox broker thread while the workload's connect(2) stays blocked.

Related Issue

Closes #5182

Changes

  • Cap /proc/<pid>/cmdline reads at 128 KiB; one extra byte makes the truncation predicate exact, and the truncated tail drops its last argument so no partial fragment reaches cmdline_paths
  • Deduplicate cmdline paths with a hash set instead of a quadratic Vec::contains scan, keeping the insertion order consumers compare
  • Evict one digest-cache entry at capacity instead of clearing all 1024
  • Read /proc/<pid>/status once per ancestry hop instead of twice
  • Hash executables with a 64 KiB buffer instead of 8 KiB

Release Notes

Binary identity resolution now bounds its per-connect procfs work: long command lines and digest-cache churn no longer add unbounded cost per mediated connect. Large uncached executables can still delay a connect while being hashed; this change reduces that cost.

Testing

  • Checks appropriate to the affected code and behavior pass

  • Unit tests added/updated (if applicable)

  • E2E tests added/updated (if applicable)

  • cargo test -p openshell-binary-identity: 20 tests, up from 5 (long-cmdline child fixture, cap boundary matrix, dedup order, eviction at capacity, status parser, streaming-vs-one-shot digest equivalence)

  • cargo test -p openshell-sandbox network_broker, -p openshell-supervisor-network, -p openshell-sandbox-backend

  • Workspace clippy and rustfmt command sets, run directly (the mise tasks' lockfile check hits a local uv config conflict)

  • mise run e2e:podman:ci

  • Buffer benchmark (same-run, release mode, page-cached 100 MiB file, 15 rounds): 64 KiB vs 8 KiB is about 1.28x throughput, 68.9 vs 88.2 ms per hash

Argv beyond the 128 KiB cap no longer appears in snapshot validation, OPA denial strings, or identity logs. cmdline_paths is diagnostic context and never authorizes (pinned by the supervisor-network spoofing tests), so this only limits forensic visibility of very long command lines.

Checklist

  • Follows Conventional Commits
  • Commits are signed off (DCO)
  • Architecture docs updated (if applicable)

…lution

Cut the procfs and hashing work openshell-binary-identity pays for every
external connect, all of it synchronous on the sandbox broker thread
while the workload's connect(2) stays blocked:

- Cap /proc/<pid>/cmdline reads at 128 KiB so a long command line can
  no longer grow per-connect work without bound. Reading one byte past
  the cap keeps the truncation predicate exact, and the truncated tail
  drops its last argument so no partial fragment reaches cmdline_paths.
- Deduplicate the collected cmdline paths with a hash set instead of a
  quadratic Vec scan, preserving the insertion order consumers compare.
- Evict one digest-cache entry at capacity instead of clearing the
  whole cache, so a capacity crossing no longer forces a full-file
  rehash of every live executable.
- Read /proc/<pid>/status once per ancestry hop instead of twice, so
  the NSpid check and the parent PID come from one kernel generation.
- Hash executables with a 64 KiB buffer instead of 8 KiB; a same-run
  release-mode comparison over a page-cached 100 MiB file measures
  about 1.28x throughput (68.9 ms vs 88.2 ms per hash).

Digests, fail-closed Option contracts, and cmdline_paths for
untruncated command lines are unchanged; streaming digests stay
byte-identical, pinned by an equivalence test against the one-shot
facade hash.

Closes ai-openshell#5182

Signed-off-by: Emilien Macchi <emacchi@redhat.com>
@copy-pr-bot

copy-pr-bot Bot commented Nov 3, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@mrunalp

mrunalp commented Nov 3, 2026

Copy link
Copy Markdown
Contributor

/ok to test 9675864

@FrostGod

FrostGod commented Nov 4, 2026

Copy link
Copy Markdown
Contributor

Please soften the release note: large uncached executables can still delay connections while being hashed; this change reduces that cost.

@EmilienM

EmilienM commented Nov 4, 2026

Copy link
Copy Markdown
Contributor Author

Please soften the release note: large uncached executables can still delay connections while being hashed; this change reduces that cost.

done

@mrunalp mrunalp added the retired:area:sandbox Retired; do not apply. See #5151 for item-specific migration to the current taxonomy. label Nov 4, 2026
@FrostGod
FrostGod added this pull request to the merge queue Nov 6, 2026
Merged via the queue into ai-openshell:main with commit af8182f Nov 6, 2026
88 of 91 checks passed
@FrostGod
FrostGod deleted the perf/5182-binary-identity-procfs/EmilienM branch November 6, 2026 22:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

retired:area:sandbox Retired; do not apply. See #5151 for item-specific migration to the current taxonomy.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug: unbounded per-connect procfs work in binary identity resolution

3 participants