Skip to content

test(e2e): port Docker and Podman driver feature tests from e2e/rust - #5695

Draft
matthewgrossman wants to merge 15 commits into
test/5655-port-network-l7-policy/matthewgrossmanfrom
test/5676-port-docker-podman-features/matthewgrossman
Draft

matthewgrossman wants to merge 15 commits into
test/5655-port-network-l7-policy/matthewgrossmanfrom
test/5676-port-docker-podman-features/matthewgrossman

Conversation

@matthewgrossman

@matthewgrossman matthewgrossman commented Nov 18, 2026 •

Copy link
Copy Markdown
Member

Summary

Ports the e2e/rust tests that need a container engine to testkit suites, so they run on the Docker and Podman rows instead of being gated to one engine. Draft: some legacy tests stay until the drops in #5297 are decided.

Related Issue

Closes #5676
Part of #5281

Stack: main <- #5665 <- #5695 (#5691 has merged) (base test/5655-port-network-l7-policy/matthewgrossman). The PR holds only its own 14 commits.

Helpers from #5691 (now on main) used: sandbox_clients_support::{connect, http, http_raw} with Request, ReadEnd::Closed and Http::status(); host_server_support::{TEST_HOST, HttpRecorder, TcpServer}; testkit::python_image() with .command(["sleep", "infinity"]); policy::allow and policy::base_with with policy::binaries_yaml(PYTHON_BINARIES). Left inline: the driver-config main commands, which must run as the main process (the OCI test checks workspace preparation before exec sleep infinity), and the bind test's filesystem-only policy YAML, which has no binaries: list.

Changes

  • tests/suites/drivers/{docker,podman}: stop is terminal and ephemeral cleanup leaves no containers, one test per driver. Podman also checks the supervisor JWT exp. drivers/docker is a new suite with its CI rows.
  • features/resource-limits: checks that gateway info reports CPU and memory limit_supported, then reads cpu.max and memory.max.
  • features/ocsf: supervisor stderr OCSF JSON records. Covers opt-in, clean stdout, gateway logs keeping only the shorthand, Network/HTTP/SSH Activity classes (denied connect, L7-denied POST to an in-process host server, exec sessions), container and image attribution, version, and OCSF schema validation.
  • features/driver-config: existing volume, OCI workspace with a nested volume, and bind mount, on a new <base>-driver-config tmachine environment.
  • features/host-gateway: response_eof.rs runs the three closing responses in one sandbox and checks the read ends with ReadEnd::Closed. host_server_reachable_after_stop_start checks the host server again after stop and start.
  • features/transparent-tcp: native TCP resolves through policy DNS to a synthetic address and reaches an echo fixture, also by the numeric synthetic address after its mapping expires, as an IPv4-mapped IPv6 address, and on the endpoint's second port. A wrong port, the fixture's real address, and an unlisted host are denied, nothing denied reaches the fixture, and exec still works afterwards. Rows: Docker rootful, Podman rootful and rootless. On both engines the fixture runs on the host behind host.openshell.internal. A fixture container on the Podman network can't be used: since RFC 0012 (feat(isolation): implement the RFC 0012 sandbox architecture #2942) the Podman supervisor shares the host's network namespace, so policy DNS asks the host's resolver, which doesn't know the network's aliases. The legacy Podman variant (fixture container, TCP port 53, transparent listener port 15001) never ran in CI, because transparent_tcp was excluded from the tmachine archive.
  • Helpers: container-engine state, copy, log and volume helpers; testkit Context::gateway_info and SandboxBuilder::log_level. Engine::copy_from reads cp <id>:<path> - as a tar stream, because under rootful Podman (sudo -n podman cp) a copy to a host path is owned by root and the test can't read it.
  • features/ocsf sets the sandbox log level to info. The shorthand layer logs at info; the Podman driver leaves the supervisor at its warn default, while the Docker driver passes the gateway's level, so without it Podman's supervisor stderr has no OCSF NET:OPEN line.
  • New shared clients and fixtures:
    • sandbox_clients_support::resolve returns what getaddrinfo answered (Resolve::{Addresses, Failed}).
    • sandbox_clients_support::Python runs connect, http, http_raw and resolve under a chosen interpreter with extra environment, such as a copied python3 with PYTHONHOME. The free functions use the image's python3.
  • Removes driver_config_volume.rs, ocsf_console.rs and podman_resource_limits.rs, and the EOF test from host_gateway_alias.rs, plus the now-unused bollard dependency. Removes the ported native TCP test from transparent_tcp.rs. local_driver_token_restart.rs keeps only its VM gateway restart test until PR6, and leaves the Podman CI list.

Defaults applied

  • The driver-config environment sets allow_driver_config = true and enable_bind_mounts = true, and disables resource admission. Only this environment disables it: admission rejects host bind mounts, so the bind-mount test can't run with it on. Test volumes still carry the openshell.ai/sandbox-attachable and workspace labels.
  • The bind-mount test uses testkit::python_image(), because the default image has no sandbox user. The bind source is a temp directory under $HOME.
  • features/resource-limits has no k3s row yet. The VM driver reports limit_supported=false, so no VM issue was filed.

Coverage that is new on a driver

  • Resource limits on Docker.
  • OCSF stderr records on Podman.
  • OCI workspace with a nested volume on Podman.
  • Driver lifecycle (stop is terminal, ephemeral cleanup) on Docker.
  • HTTP response EOF and stop/start reachability on Docker, Podman rootless, and k3s/k3s-ha. They were Podman-only or Docker-only before.
  • Native TCP policy DNS runs in the tmachine CI rows; before, transparent_tcp was excluded from the tmachine archive.

Pending drops (Decision D in #5297)

These stay in e2e/rust until approved:

  • ocsf_jsonl_conformance.rs, for its /var/log file-sink check. Its other checks now live in features/ocsf.
  • host_gateway_alias::sandbox_reaches_host_openshell_internal_via_host_gateway_alias.
  • podman_host_gateway.rs, for its reachability half and the nameserver 127.0.0.53 check. Its stop/start half now lives in features/host-gateway.
  • transparent_tcp::rootless_podman_musl_getaddrinfo_uses_udp_policy_dns.
  • transparent_tcp::rootless_podman_private_dns_pins_do_not_cross_binary_rules. It needs a fixture at a private address reached by a *.openshell.test name. On Podman the host-networked supervisor's resolver can't resolve a network alias, and on Docker the fixture would sit behind host.openshell.internal, which the supervisor trusts as the host gateway for any matching rule. It never ran in CI.

Testing

  • cargo fmt --check on testkit, support, drivers/docker, drivers/podman, features, conformance and e2e/rust.

  • cargo clippy --locked --all-targets -- -D warnings on the same workspaces.

  • Unit tests in tests/support (including the new Python command and resolve parser) and the features/ocsf record parser.

  • uv run tasks/scripts/check_cargo_lockfile_alignment.py and tasks/scripts/check-cargo-lockfiles.sh. Every commit resolves with cargo metadata --locked.

  • Run end to end locally against Docker Desktop with e2e/with-docker-gateway.sh: features/host-gateway (3 tests), features/ocsf, features/resource-limits, features/transparent-tcp and drivers/docker pass. features/driver-config and the Podman rows rely on CI.

  • Podman rows may flake on ContainerExited until fix(server): keep the reconcile sweep from failing a sandbox mid-start #5265 merges.

  • Checks appropriate to the affected code and behavior pass

  • Unit tests added/updated (if applicable)

  • E2E tests added/updated (if applicable)

Checklist

  • Follows Conventional Commits
  • Commits are signed off (DCO)
  • Architecture docs updated (if applicable)

@copy-pr-bot

copy-pr-bot Bot commented Nov 18, 2026

Copy link
Copy Markdown

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@github-actions github-actions Bot added the state:accepted A maintainer decided OpenShell should pursue this issue label Nov 18, 2026
@matthewgrossman
matthewgrossman force-pushed the test/5655-port-network-l7-policy/matthewgrossman branch from 251835b to 3214f39 Compare November 18, 2026 18:34
@matthewgrossman
matthewgrossman force-pushed the test/5676-port-docker-podman-features/matthewgrossman branch from 4c8dfc9 to 4579361 Compare November 18, 2026 18:42
@matthewgrossman

Copy link
Copy Markdown
Member Author

/ok to test 4579361

@matthewgrossman
matthewgrossman force-pushed the test/5676-port-docker-podman-features/matthewgrossman branch from 4579361 to cd9c8f2 Compare November 18, 2026 18:51
@matthewgrossman

Copy link
Copy Markdown
Member Author

/ok to test cd9c8f2

@matthewgrossman
matthewgrossman force-pushed the test/5655-port-network-l7-policy/matthewgrossman branch from 3214f39 to f99bacb Compare November 18, 2026 19:08
@matthewgrossman

Copy link
Copy Markdown
Member Author

/ok to test 0a29574

@matthewgrossman
matthewgrossman force-pushed the test/5676-port-docker-podman-features/matthewgrossman branch from cd9c8f2 to 0a29574 Compare November 18, 2026 19:11
@matthewgrossman
matthewgrossman force-pushed the test/5655-port-network-l7-policy/matthewgrossman branch from f99bacb to afcf2f4 Compare November 18, 2026 23:27
@matthewgrossman

Copy link
Copy Markdown
Member Author

/ok to test f188601

@matthewgrossman

Copy link
Copy Markdown
Member Author

/ok to test f054e20

@matthewgrossman

Copy link
Copy Markdown
Member Author

/ok to test 0211cf1

@matthewgrossman
matthewgrossman force-pushed the test/5676-port-docker-podman-features/matthewgrossman branch from f054e20 to 0211cf1 Compare November 20, 2026 00:34
@matthewgrossman

Copy link
Copy Markdown
Member Author

/ok to test 1d05357

@github-actions

Copy link
Copy Markdown

Label test:e2e applied for 1d05357. Open the existing run and click Re-run all jobs to execute with the label set. The run will execute the standard E2E suite after building the required gateway, sandbox, and supervisor images once. The matching required CI gate status on this PR will flip green automatically once the run finishes.

…container-engine

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
…leanup

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
… and ephemeral cleanup

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
…ker and Podman

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
…onfig on Docker and Podman

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
… host-gateway

The EOF check runs its three responses in one sandbox and reads the close through the shared http_raw client. The stop/start check is the restart half of podman_host_gateway, which stays in e2e/rust until its DNS relay check is decided.

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
…ocsf

Merges the class, attribution and schema checks of ocsf_jsonl_conformance into the stderr test. The sandbox denies a TCP connect and an L7 POST to an in-process host server, and the exec sessions add SSH Activity. ocsf_jsonl_conformance stays in e2e/rust for its /var/log file-sink check until that drop is decided.

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
…hosen Python to sandbox-clients

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
@matthewgrossman

Copy link
Copy Markdown
Member Author

/ok to test 47c28fb

@matthewgrossman
matthewgrossman force-pushed the test/5676-port-docker-podman-features/matthewgrossman branch from 1d05357 to 47c28fb Compare November 20, 2026 07:11
…transparent-tcp

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
…en_restart

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
…rites the shorthand

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
@matthewgrossman

Copy link
Copy Markdown
Member Author

/ok to test b82164b

@matthewgrossman
matthewgrossman force-pushed the test/5676-port-docker-podman-features/matthewgrossman branch from 47c28fb to b82164b Compare November 20, 2026 16:22
@matthewgrossman matthewgrossman added test:e2e Requires end-to-end coverage and removed test:e2e Requires end-to-end coverage labels Nov 20, 2026
…test/5676-port-docker-podman-features/matthewgrossman

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

state:accepted A maintainer decided OpenShell should pursue this issue test:e2e Requires end-to-end coverage

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant