Repository navigation
feat(voice): start and end live voice sessions from backend functions - #372
Draft
talge-a11y wants to merge 2 commits into
Draft
talge-a11y wants to merge 2 commits into
talge-a11y wants to merge 2 commits into
Conversation
Adds base44.asServiceRole.voice with startSession() and endSession(), modeled on the browser module. A backend function passes the browser's WebRTC SDP offer, an MCP profile ID, and instructions, and returns the SDP answer to the browser. Audio then flows directly between the browser and the voice model. - startSession() sends Base44-Functions-Version together with the on-behalf-of user token, so the app's owner or an editor gets the draft MCP profile in the app editor preview. The service-role client doesn't send that header by default, so it's added to this request only, as the actors module does for its token mint. - The module passes every value through unchanged. Models, voices, and message roles are plain strings, the JSDoc gives no limits or value lists, and voiceModel and agentModel are optional. A backend validation change never needs an SDK release. - Model IDs follow AI Gateway IDs, such as gpt_6_luna. Backend: base44-dev/apper#37995. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
🚀 Package Preview Available!Install this PR's preview build with npm: npm i @base44-preview/sdk@0.8.59-pr.372.142c509Prefer not to change any import paths? Install using npm alias so your code still imports npm i "@base44/sdk@npm:@base44-preview/sdk@0.8.59-pr.372.142c509"Or add it to your {
"dependencies": {
"@base44/sdk": "npm:@base44-preview/sdk@0.8.59-pr.372.142c509"
}
}
Preview published to npm registry — try new features instantly! |
A voice session runs in the browser after the backend function returns, so no code holds the session object when the user hangs up. The end() shortcut had no caller. endSession() with the session ID, called from a later request, is the only way to end a session early. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
base44.asServiceRole.voicewithcreateSession()andendSession(), for live voice sessions between an app's users and an AI agent that uses the app's tools through an MCP profile. Backend: base44-dev/apper#37995.A backend function passes the browser's WebRTC SDP offer, an MCP profile ID, and instructions, and returns the SDP answer to the browser. Audio then flows directly between the browser and the voice model. The module follows the browser module: same service-role auth, and the same signed session IDs. There's no
end()shortcut on the returned session, because the call runs in the browser after the function returns. A later request ends the session withendSession().Key decisions
Base44-Functions-Version, which the service-role client doesn't send.createSession()adds it to that request only, next to theon-behalf-ofuser token, as the actors module does for its token mint. Other service-role requests are unchanged, and a test covers that.gpt_6_luna.voiceModelandagentModelare optional, so the backend can pick defaults.Depends on backend changes to apper#37995
voice_modelandagent_modeloptional with defaults.Until those land, calls that omit the models, or pass gateway IDs, fail with a 422. The examples use
gpt_live_1as a placeholder for the voice model's ID.Docs
The four voice types are added to the docs config, so the next docs run publishes the voice reference page. To keep it hidden until the feature flag is ready, remove them from
types-to-expose.json. The JSDoc doesn't link a voice guide page yet.Testing
npm test -w @base44/sdk: 453 passed, including 16 new voice tests. Type tests andtsc --noEmitare clean.npm run create-docs -w @base44/sdkrenders the voice page with both methods, their Throws sections, and examples.🤖 Generated with Claude Code