Skip to content

test(oidc-auth): port the OIDC authorization tests to testkit - #5787

Draft
matthewgrossman wants to merge 74 commits into
mainfrom
test/5731-port-oidc-auth-tests/matthewgrossman
Draft

matthewgrossman wants to merge 74 commits into
mainfrom
test/5731-port-oidc-auth-tests/matthewgrossman

Conversation

@matthewgrossman

@matthewgrossman matthewgrossman commented Nov 20, 2026 •

Copy link
Copy Markdown
Member

Summary

Port the OIDC tests in e2e/rust/tests/oidc_pkce.rs to 13 testkit tests in tests/suites/features/oidc-auth. They run in a new <base>-oidc tmachine environment that reuses the Keycloak setup.

Related Issue

Closes #5731
Part of #5281

This branch merges #5780 (the gateway_overrides role and environment contract table, for #5728) and #5657 (provider test port). Merge those first; this PR's diff shrinks to its own commits after they land.

Changes

  • <base>-oidc environment (tests/tmachine): the oidc_gateway role applies an oidc override block through gateway_overrides. It sets [openshell.gateway.oidc] to trust the Keycloak realm and sets [openshell.gateway.mtls_auth] operator_enabled = true in place of enabled. It also re-signs testkit's client certificate as an operator (OU=operator), so testkit's own connection sets up and cleans up without signing in as a user. The suite's setup fails with a pointer to the role if the certificate signs in as a user.
    The block also sets [openshell.gateway.tls] to the paths generate-certs writes. config preflight (the unit's first ExecStartPre) accepts operator_enabled only with an explicit client CA, and doesn't pick up the local TLS files the packaged gateway otherwise finds on its own.
  • keycloak-support: User, Session (a signed-in user's SDK client and CLI), browser_login (the PKCE flow with a recorded xdg-open) and token-claim helpers. These are shaped so test(e2e): migrate the Python E2E suites to Rust #5624's oidc_authz.rs can move onto them.
  • testkit: Cli::without_env and Cli::in_workspace; Workspace::track_sandbox, track_provider and import_profile.
  • oidc-auth suite: 13 tests following the issue's mapping table. Per-user clients verify the gateway against its CA and send only a bearer token (no --gateway-insecure, no client certificate). Denials are checked by gRPC code and message. One CLI test checks the remediation command a workspace user sees.
  • CI: one row, ubuntu-docker-rootful-oidc with deb.
  • oidc_pkce.rs keeps only the 4 tests decision D drops: admin_can_delete_workspace, second_workspace_user_cannot_read_first_users_workspace, workspace_admin_can_create_sandbox and workspace_admin_can_delete_sandbox. The e2e:oidc-pkce tasks stay.

Notes for review:

  • The gateway has no built-in provider profiles, so tests import a small test profile into their workspace, or globally for the profile-resolve test, and delete it on cleanup.
  • The CLI-only "requires admin privileges" check is dropped; denials are asserted on the gateway's own messages (role 'openshell-admin' required, not a member of workspace '<ws>', and so on).
  • CI client variables and client_credentials_grant are left to test(e2e): migrate the Python E2E suites to Rust #5624.

Testing

  • cargo clippy -D warnings and cargo fmt for testkit, keycloak-support and the features workspace
  • oidc_pkce.rs compiles clean of new warnings with --features e2e-oidc-pkce
  • E2E tests added/updated
  • ubuntu-docker-rootful-oidc CI row passes (waiting on /ok to test)

Checklist

  • Follows Conventional Commits
  • Commits are signed off (DCO)

create_provider and install_provider return a builder that creates the
provider when awaited, so tests can chain .config(key, value) for
provider create --config. Existing callers are unchanged.

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
Move provider_files and provider_auto_create from e2e/rust into a
providers conformance suite. The auto-create test imports a run-unique
profile instead of relying on a globally imported claude-code profile,
so it no longer changes shared gateway state.

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
Requests go to the gateway's endpoint with the service URL's host as the
Host header, which is how the gateway routes service traffic. Over TLS
the client trusts the gateway CA and presents the registered gateway's
client certificate, through openshell-crypto's rustls configuration.

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
… suite

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
…mage

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
…er-tests/matthewgrossman

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
…lder::providers

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
…olicy set helpers

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
…vers to host-server

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
…ewrite tests

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
… tests

These tests now run in the conformance suite and the routable-host-ip
feature package.

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
…suites

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
…er limit

The gateway rejects sandbox names longer than 19 characters, which leaves
5 for a suffix after test-<run-id>-. The static credential binding test
also needs the Python workload image, which has the sandbox user that the
base policy runs as.

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
Drops the include of a conformance client script from another workspace.

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
Replace conformance's support module and generic client scripts with
testkit's run_python, last_json, policy readers, providers and
binaries_yaml, and the connect, http and http_raw probes. Protocol and
specialized clients move to the sandbox-clients scripts module.

The pipelined request tests now count every response in the reply; the
403 follows the first response's body with no line break, so the old
line-based count missed it. The policy update tests use the Python
image, which has the sandbox user the base policy runs as.

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
@github-actions github-actions Bot added the state:accepted A maintainer decided OpenShell should pursue this issue label Nov 20, 2026
@github-actions

Copy link
Copy Markdown

… arrives on loopback

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
…settings

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
…an empty file

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
Add the authority key identifier to server certificates, which strict
verifiers such as Python 3.13 require, time out TLS handshakes, report CA
load failures as such, and keep test CA files in temporary directories.
The contract table now names environment-chosen ports
OPENSHELL_TEST_<SERVICE>_PORT, matching HttpRecorder's port option.

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
@matthewgrossman

Copy link
Copy Markdown
Member Author

/ok to test c29be49

@matthewgrossman matthewgrossman added the test:e2e Requires end-to-end coverage label Nov 20, 2026
@github-actions

Copy link
Copy Markdown

Label test:e2e applied for c29be49. Open the existing run and click Re-run all jobs to execute with the label set. The run will execute the standard E2E suite after building the required gateway, sandbox, and supervisor images once. The matching required CI gate status on this PR will flip green automatically once the run finishes.

…eflight

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
…ers/matthewgrossman' into test/5731-port-oidc-auth-tests/matthewgrossman

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>

# Conflicts:
#	TESTING.md
#	tests/support/README.md
#	tests/support/host-server/src/http.rs
#	tests/support/host-server/src/lib.rs
#	tests/support/host-server/src/tls.rs
#	tests/tmachine/ansible/playbooks/drivers/podman/userns.yaml
#	tests/tmachine/ansible/roles/gateway_overrides/defaults/main.yaml
#	tests/tmachine/ansible/roles/gateway_overrides/tasks/main.yaml
…tthewgrossman' into test/5731-port-oidc-auth-tests/matthewgrossman

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
…uth-tests/matthewgrossman

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
@matthewgrossman

Copy link
Copy Markdown
Member Author

/ok to test 8b9ac69

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
@matthewgrossman

Copy link
Copy Markdown
Member Author

/ok to test 6740fa3

…file

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
@matthewgrossman

Copy link
Copy Markdown
Member Author

/ok to test 82b5c06

…k-l7-policy/matthewgrossman

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
…uth-tests/matthewgrossman

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
…/matthewgrossman' into test/5728-share-environment-helpers/matthewgrossman

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
…ers/matthewgrossman' into test/5731-port-oidc-auth-tests/matthewgrossman

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

state:accepted A maintainer decided OpenShell should pursue this issue test:e2e Requires end-to-end coverage

Projects

None yet

Development

Successfully merging this pull request may close these issues.

test(e2e): port OIDC authorization tests from e2e/rust to features/oidc-auth

1 participant