Skip to content

feat(voice): start and end live voice sessions from backend functions - #372

Draft
talge-a11y wants to merge 2 commits into
mainfrom
feat/voice-sessions
Draft

talge-a11y wants to merge 2 commits into
mainfrom
feat/voice-sessions

Conversation

@talge-a11y

@talge-a11y talge-a11y commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Adds base44.asServiceRole.voice with createSession() and endSession(), for live voice sessions between an app's users and an AI agent that uses the app's tools through an MCP profile. Backend: base44-dev/apper#37995.

A backend function passes the browser's WebRTC SDP offer, an MCP profile ID, and instructions, and returns the SDP answer to the browser. Audio then flows directly between the browser and the voice model. The module follows the browser module: same service-role auth, and the same signed session IDs. There's no end() shortcut on the returned session, because the call runs in the browser after the function returns. A later request ends the session with endSession().

const session = await base44.asServiceRole.voice.createSession({
  sdp,
  mcpProfileId: VOICE_PROFILE_ID,
  voiceInstructions: 'You are the shop assistant. Speak briefly.',
  agentInstructions: 'Use the app tools to look up orders.',
});
return Response.json({ sessionId: session.sessionId, sdp: session.sdp });

Key decisions

  • Draft MCP profile in the preview. The backend picks the draft profile from Base44-Functions-Version, which the service-role client doesn't send. createSession() adds it to that request only, next to the on-behalf-of user token, as the actors module does for its token mint. Other service-role requests are unchanged, and a test covers that.
  • No backend limits in the SDK. The module passes every value through unchanged. Models, voices, and message roles are plain strings, and the JSDoc gives no limits, value lists, durations, or prices. A backend validation change never needs an SDK release. Supported values belong on a voice guide page in the docs site.
  • AI Gateway model IDs, such as gpt_6_luna. voiceModel and agentModel are optional, so the backend can pick defaults.
  • No browser helper. The app's frontend handles WebRTC itself with built-in browser APIs.

Depends on backend changes to apper#37995

  • Accept AI Gateway model IDs instead of OpenAI names.
  • Make voice_model and agent_model optional with defaults.

Until those land, calls that omit the models, or pass gateway IDs, fail with a 422. The examples use gpt_live_1 as a placeholder for the voice model's ID.

Docs

The four voice types are added to the docs config, so the next docs run publishes the voice reference page. To keep it hidden until the feature flag is ready, remove them from types-to-expose.json. The JSDoc doesn't link a voice guide page yet.

Testing

  • npm test -w @base44/sdk: 453 passed, including 16 new voice tests. Type tests and tsc --noEmit are clean.
  • Removing the header fix makes both draft-profile header tests fail.
  • npm run create-docs -w @base44/sdk renders the voice page with both methods, their Throws sections, and examples.
  • Not run against a live voice session.

🤖 Generated with Claude Code

Adds base44.asServiceRole.voice with startSession() and endSession(),
modeled on the browser module. A backend function passes the browser's
WebRTC SDP offer, an MCP profile ID, and instructions, and returns the
SDP answer to the browser. Audio then flows directly between the
browser and the voice model.

- startSession() sends Base44-Functions-Version together with the
  on-behalf-of user token, so the app's owner or an editor gets the
  draft MCP profile in the app editor preview. The service-role client
  doesn't send that header by default, so it's added to this request
  only, as the actors module does for its token mint.
- The module passes every value through unchanged. Models, voices, and
  message roles are plain strings, the JSDoc gives no limits or value
  lists, and voiceModel and agentModel are optional. A backend
  validation change never needs an SDK release.
- Model IDs follow AI Gateway IDs, such as gpt_6_luna.

Backend: base44-dev/apper#37995.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

🚀 Package Preview Available!


Install this PR's preview build with npm:

npm i @base44-preview/sdk@0.8.59-pr.372.142c509

Prefer not to change any import paths? Install using npm alias so your code still imports @base44/sdk:

npm i "@base44/sdk@npm:@base44-preview/sdk@0.8.59-pr.372.142c509"

Or add it to your package.json dependencies:

{
  "dependencies": {
    "@base44/sdk": "npm:@base44-preview/sdk@0.8.59-pr.372.142c509"
  }
}

Preview published to npm registry — try new features instantly!

A voice session runs in the browser after the backend function
returns, so no code holds the session object when the user hangs up.
The end() shortcut had no caller. endSession() with the session ID,
called from a later request, is the only way to end a session early.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant