feat(realtime): unify initial state with modelName, image, and prompt

[AdirAmsalem]

Summary

• Replace the boolean isAvatarLive flag with a typed modelName: RealTimeModels parameter across the WebRTC stack, enabling extensible model-specific branching beyond a binary avatar/non-avatar check.
• Unify initial prompt delivery so all models send the prompt via WebSocket during the connection handshake (previously only live_avatar did this pre-handshake; other models sent it post-connection via setPrompt).
• Remove the avatar/avatarImage connect option entirely. Image is now passed as initialState.image alongside prompt, using the more capable imageToBase64 utility (supports Blob, File, URL, data URL, raw base64).
• In the WebRTC connection Phase 2 pre-handshake: if image is provided, send via setImageBase64() which carries prompt+enhance in the same message; otherwise fall back to sendInitialPrompt() for prompt-only.

Changes

API surface

• initialState.image (Blob | File | string) replaces avatar: { avatarImage } — works for all models, not just live_avatar
• Removed AvatarOptions type export from src/index.ts

Internal

• avatarImageBase64 → initialImage through WebRTCConfig, WebRTCManager, and WebRTCConnection
• Removed avatarOptionsSchema, AvatarOptions, sendAvatarImage() wrapper
• Phase 2 logic: image path sends via setImageBase64(image, {prompt, enhance}); prompt-only path uses sendInitialPrompt()
• modelName: RealTimeModels replaces isAvatarLive: boolean for model-specific branching

Breaking changes

• avatar option removed from realtime.connect() — use initialState: { image: ... } instead
• AvatarOptions type no longer exported

---

Note

Medium Risk
Changes the realtime connect API surface and the connection handshake sequencing for prompt/image, which can affect model startup behavior and break existing consumers using the removed avatar option.

Overview
Unifies realtime connection setup around initialState by adding initialState.image support and removing the avatar: { avatarImage } connect option (and the exported AvatarOptions type), updating docs/examples accordingly.

Refactors the WebRTC stack to pass a typed modelName (instead of isAvatarLive) and to always send the initial prompt during the WebSocket pre-handshake; if an initial image is provided it is sent via set_image with the prompt/enhance in the same message, otherwise it falls back to prompt-only. Tests and the SDK demo page are updated to cover/consume the new handshake behavior and initial image loading.

^{Written by Cursor Bugbot for commit 4a2b432. This will update automatically on new commits. Configure here.}

[pkg-pr-new]

Open in StackBlitz

npm i https://pkg.pr.new/DecartAI/sdk/@decartai/sdk@84

commit: 4a2b432