Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions .agents/skills/debug-inference/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -406,3 +406,14 @@ When you report back, state:
2. Whether gateway topology is part of the problem
3. The most likely root cause
4. The exact fix commands the user should run

## Codex subscription standalone search

When Responses inference works but native search reports no completed receipt,
check the custom Codex provider's `supports_standalone_web_search` capability and
the matched supervisor/router support for `POST /v1/alpha/search`. Search has its
own buffered JSON protocol and cannot use the Responses path/body rewrite.
A policy denial for this exact path can come from an older inference interceptor;
a broader sandbox network rule does not fix it. Stop unchanged retries and
report the required component update. Never use another account, copied OAuth
tokens, or direct sandbox egress as a fallback.
10 changes: 10 additions & 0 deletions architecture/gateway.md
Original file line number Diff line number Diff line change
Expand Up @@ -784,3 +784,13 @@ system entry instead of pretending to delete package-manager owned state.
streams must be re-established by supervisors.
- User-facing behavior changes must update published docs in `docs/`; this file
should only record stable architecture.

### Codex subscription standalone search

The attached Codex subscription route advertises Responses and standalone web
search separately. The supervisor recognises only `POST /v1/alpha/search` for
search, uses the selected attached route and buffers its JSON response. The
router maps it to the pinned Codex origin's `/alpha/search`, preserving the
search body and gateway-owned authorization/account headers. Responses retain
their `/responses` override and model rewrite. This adds no direct sandbox
egress or alternate credential route.
5 changes: 4 additions & 1 deletion crates/openshell-core/src/inference.rs
Original file line number Diff line number Diff line change
Expand Up @@ -420,7 +420,10 @@ mod tests {
codex.default_base_url,
subscription_oauth::OPENAI_CODEX_INFERENCE_BASE_URL
);
assert_eq!(codex.protocols, &["openai_responses"]);
assert_eq!(
codex.protocols,
&["openai_responses", "openai_standalone_search"]
);
assert!(codex.base_url_config_keys.is_empty());
assert!(codex.requires_provider_attachment);

Expand Down
3 changes: 2 additions & 1 deletion crates/openshell-core/src/subscription_oauth.rs
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,8 @@ pub const XAI_GROK_SCOPES: &[&str] = &[
"grok-cli:access",
];

pub const OPENAI_CODEX_INFERENCE_PROTOCOLS: &[&str] = &["openai_responses"];
pub const OPENAI_CODEX_INFERENCE_PROTOCOLS: &[&str] =
&["openai_responses", "openai_standalone_search"];
pub const XAI_GROK_INFERENCE_PROTOCOLS: &[&str] = &["openai_chat_completions", "model_discovery"];
pub const OPENAI_CODEX_ROUTE_HEADERS: &[(&str, &str)] = &[("originator", "openshell")];
pub const XAI_GROK_ROUTE_HEADERS: &[(&str, &str)] = &[];
Expand Down
134 changes: 133 additions & 1 deletion crates/openshell-router/src/backend.rs
Original file line number Diff line number Diff line change
Expand Up @@ -331,6 +331,19 @@ fn prepare_backend_request(
body: bytes::Bytes,
stream_response: bool,
) -> Result<(reqwest::RequestBuilder, String), RouterError> {
let standalone_search = path.split('?').next() == Some("/v1/alpha/search");
if standalone_search
&& (method != "POST"
|| !route
.protocols
.iter()
.any(|p| p == "openai_standalone_search"))
{
return Err(RouterError::Unauthorized(
"standalone search requires its advertised protocol and POST method".to_string(),
));
}

// For AWS Bedrock routes the model id is encoded in the URL path
// (`/model/{modelId}/invoke[-with-response-stream]`), not in the
// JSON body. The caller's path can carry any model id; rewrite it
Expand Down Expand Up @@ -403,7 +416,13 @@ fn prepare_backend_request(
// path) and inject "anthropic_version" (required in the body, not a header).
// Non-JSON bodies pass through unchanged; model rewrite and version injection
// are silently skipped. Such bodies would be rejected by the upstream anyway.
let body = rewrite_inference_request_body(route, body)?;
// Search has its own schema. In particular it is not a Responses request:
// preserve the model identity and settings carried by its search context.
let body = if standalone_search {
body
} else {
rewrite_inference_request_body(route, body)?
};
builder = builder.body(body);

Ok((builder, url))
Expand Down Expand Up @@ -870,6 +889,17 @@ fn build_provider_url(
stream_response: bool,
) -> String {
let base = route.endpoint.trim_end_matches('/');
// The subscription's fixed /responses override must not capture search.
// Keep the upstream origin pinned and grant-derived headers/auth unchanged.
if is_openai_codex_subscription_route(route)
&& route
.protocols
.iter()
.any(|p| p == "openai_standalone_search")
&& protocol_path.split('?').next() == Some("/v1/alpha/search")
{
return format!("{base}/alpha/search");
}
match (&route.request_path_override, route.model_in_path) {
// Vertex AI publisher endpoint: model in URL path with suffix
// e.g. .../publishers/anthropic/models/claude-3-5-sonnet@20241022:rawPredict
Expand Down Expand Up @@ -1384,6 +1414,108 @@ mod tests {
);
}

#[tokio::test]
async fn standalone_search_round_trips_buffered_json_without_model_rewrite() {
let server = MockServer::start().await;
let body =
br#"{"id":"search-session","model":"model-id-from-search-context","commands":{"search_query":[{"q":"OpenShell"}]},"settings":{"search_context_size":"low"},"telemetry_attributes":{"model_slug":"test-model"}}"#;
let response = serde_json::json!({"results":[{"url":"https://example.com/source"}]});
Mock::given(method("POST"))
.and(path("/v1/alpha/search"))
.and(header("authorization", "Bearer sk-test"))
.respond_with(ResponseTemplate::new(200).set_body_json(&response))
.expect(1)
.mount(&server)
.await;
let route = test_route(
&server.uri(),
&["openai_standalone_search"],
AuthHeader::Bearer,
);
let result = proxy_to_backend(
&reqwest::Client::new(),
&route,
"openai_standalone_search",
"POST",
"/v1/alpha/search",
vec![],
bytes::Bytes::from_static(body),
)
.await
.unwrap();
assert_eq!(result.status, 200);
assert_eq!(
serde_json::from_slice::<serde_json::Value>(&result.body).unwrap(),
response
);
assert_eq!(server.received_requests().await.unwrap()[0].body, body);
}

#[test]
fn standalone_search_rejects_unadvertised_routes_and_wrong_methods() {
for (protocols, method) in [
(vec!["openai_responses"], "POST"),
(vec!["openai_standalone_search"], "GET"),
] {
let route = test_route("https://api.openai.com/v1", &protocols, AuthHeader::Bearer);
assert!(matches!(
prepare_backend_request(
&reqwest::Client::new(),
&route,
method,
"/v1/alpha/search",
&[],
bytes::Bytes::from_static(b"{}"),
false
),
Err(RouterError::Unauthorized(_))
));
}
}

#[test]
fn codex_standalone_search_preserves_body_and_gateway_identity() {
let mut route = test_route(
openshell_core::subscription_oauth::OPENAI_CODEX_INFERENCE_BASE_URL,
&["openai_responses", "openai_standalone_search"],
AuthHeader::Bearer,
);
route.request_path_override = Some("/responses".to_string());
route.default_headers = vec![
("chatgpt-account-id".into(), "bound-account".into()),
("originator".into(), "openshell".into()),
];
let body = bytes::Bytes::from_static(
br#"{ "id":"search-session", "model":"model-id-from-search-context", "commands":{"search_query":[{"q":"OpenShell"}]}, "settings":{"search_context_size":"low"}, "telemetry_attributes":{"model_slug":"test-model"} }"#,
);
let headers = vec![
("authorization".into(), "Bearer hostile-token".into()),
("chatgpt-account-id".into(), "hostile-account".into()),
("originator".into(), "hostile-originator".into()),
];
let (builder, url) = prepare_backend_request(
&reqwest::Client::new(),
&route,
"POST",
"/v1/alpha/search",
&headers,
body.clone(),
false,
)
.unwrap();
assert_eq!(url, "https://chatgpt.com/backend-api/codex/alpha/search");
let request = builder.build().unwrap();
assert_eq!(request.body().unwrap().as_bytes().unwrap(), body.as_ref());
assert_eq!(request.headers()["authorization"], "Bearer sk-test");
assert_eq!(request.headers()["chatgpt-account-id"], "bound-account");
assert_eq!(request.headers()["originator"], "openshell");
// Responses retain their existing fixed upstream path.
assert_eq!(
build_provider_url(&route, &route.model, "/v1/responses", true),
"https://chatgpt.com/backend-api/codex/responses"
);
}

#[test]
fn codex_subscription_body_strips_unsupported_responses_fields() {
let mut route = test_route(
Expand Down
26 changes: 26 additions & 0 deletions crates/openshell-supervisor-network/src/l7/inference.rs
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,15 @@ pub fn default_patterns() -> Vec<InferenceApiPattern> {
kind: "responses".to_string(),
framing: ResponseFraming::Streaming,
},
// Codex executes standalone web search through its selected provider.
// The result is one JSON document, never an SSE inference stream.
InferenceApiPattern {
method: "POST".to_string(),
path_glob: "/v1/alpha/search".to_string(),
protocol: "openai_standalone_search".to_string(),
kind: "web_search".to_string(),
framing: ResponseFraming::Buffered,
},
InferenceApiPattern {
method: "POST".to_string(),
path_glob: "/v1/embeddings".to_string(),
Expand Down Expand Up @@ -505,6 +514,23 @@ mod tests {
assert_eq!(result.unwrap().protocol, "openai_chat_completions");
}

#[test]
fn detect_standalone_search_is_exact_and_buffered() {
let patterns = default_patterns();
let pattern = detect_inference_pattern("POST", "/v1/alpha/search", &patterns)
.expect("standalone search must use the attached inference route");
assert_eq!(pattern.protocol, "openai_standalone_search");
assert!(pattern.is_buffered());
for path in [
"/v1/alpha/search/extra",
"/v1/alpha/searching",
"/admin/search",
] {
assert!(detect_inference_pattern("POST", path, &patterns).is_none());
}
assert!(detect_inference_pattern("GET", "/v1/alpha/search", &patterns).is_none());
}

#[test]
fn detect_openai_responses() {
let patterns = default_patterns();
Expand Down
41 changes: 41 additions & 0 deletions crates/openshell-supervisor-network/src/proxy.rs
Original file line number Diff line number Diff line change
Expand Up @@ -9601,6 +9601,47 @@ network_policies:
/// `Content-Length`, never through the chunked streaming path whose
/// truncation handlers would append an SSE `proxy_stream_error` frame into
/// the JSON body. This guards the framing classification for the protocol.
#[tokio::test]
async fn inference_standalone_search_returns_one_buffered_json_document() {
let router = openshell_router::Router::new().unwrap();
let mut route = model_discovery_inference_route("mock://search-fixture".to_string());
route.protocols = vec!["openai_standalone_search".to_string()];
let ctx = InferenceContext::new(
crate::l7::inference::default_patterns(),
router,
vec![route],
vec![],
);
let body = r#"{"id":"search","model":"search-context-model","commands":{"search_query":[{"q":"OpenShell"}]}}"#;
let request = format!(
"POST /v1/alpha/search HTTP/1.1\r\nHost: inference.local\r\nContent-Length: {}\r\n\r\n{}",
body.len(),
body
);
let (client, mut server) = tokio::io::duplex(65536);
let (mut reader, mut writer) = tokio::io::split(client);
let task =
tokio::spawn(async move { process_inference_keepalive(&mut server, &ctx, 443).await });
writer.write_all(request.as_bytes()).await.unwrap();
writer.shutdown().await.unwrap();
let mut response = Vec::new();
reader.read_to_end(&mut response).await.unwrap();
let text = String::from_utf8(response).unwrap();
let (headers, body) = text.split_once("\r\n\r\n").unwrap();
assert!(headers.starts_with("HTTP/1.1 200"));
assert!(!headers.to_ascii_lowercase().contains("transfer-encoding"));
assert!(
headers
.to_ascii_lowercase()
.contains(&format!("content-length: {}", body.len()))
);
assert!(serde_json::from_str::<serde_json::Value>(body).is_ok());
assert!(matches!(
task.await.unwrap().unwrap(),
InferenceOutcome::Routed
));
}

#[tokio::test]
async fn inference_model_discovery_served_buffered_with_content_length() {
let listener = TcpListener::bind("127.0.0.1:0").await.unwrap();
Expand Down
16 changes: 16 additions & 0 deletions docs/providers/openai-codex-subscription.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,22 @@ OpenAI hosts remain governed by the sandbox's ordinary network policy; the
subscription grant is usable only through `inference.local` and only while this
exact provider is attached.

## Standalone web search

The attached subscription route supports Codex Responses requests and the exact
`POST /v1/alpha/search` standalone-search endpoint. Search is forwarded to the
pinned Codex backend's `/alpha/search` endpoint with gateway-owned account and
authorization headers. It returns buffered JSON; it is not a Responses request
and its body preserves the model identity and settings supplied by Codex search
context.

A custom Codex provider must declare `supports_standalone_web_search = true`
when standalone search is enabled. The gateway and supervisor must both include
this routing support. An older supervisor may reject the search path even while
normal inference works; repeating the request or allowing the public internet
in sandbox policy does not repair that route. Report the capability failure and
have the operator update the matched gateway/supervisor components.

## Refresh and failure behavior

The gateway refreshes the access token before expiry and binds refreshed tokens
Expand Down
Loading