Skip to content

feat: add wasm thread control and runtime backend/quantization/stride settings #175

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
AAlp22 merged 2 commits into from
Feb 21, 2026
Merged

feat: add wasm thread control and runtime backend/quantization/stride settings #175

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
bb1b297
feat: add runtime backend/quantization/stride and wasm thread controls
ysdede Feb 21, 2026
d593d72
fix: address PR review issues
ysdede Feb 21, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 60 additions & 4 deletions src/App.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ import {
loadSettingsFromStorage,
saveSettingsToStorage,
} from './utils/settingsStorage';
import { clampWasmThreadsForDevice } from './utils/hardwareThreads';

// Singleton instances
let audioEngine: AudioEngine | null = null;
Expand Down Expand Up @@ -228,12 +229,22 @@ const App: Component = () => {
const persistedSettings = loadSettingsFromStorage();
const persistedGeneral = persistedSettings.general;
const persistedAudio = persistedSettings.audio;
const persistedModelId = persistedSettings.model?.selectedModelId;
const persistedModel = persistedSettings.model;
const persistedModelId = persistedModel?.selectedModelId;
const persistedUi = persistedSettings.ui;

if (persistedModelId && MODELS.some((model) => model.id === persistedModelId)) {
appStore.setSelectedModelId(persistedModelId);
}
if (persistedModel?.backend !== undefined) {
appStore.setModelBackendMode(persistedModel.backend);
}
if (persistedModel?.encoderQuant !== undefined) {
appStore.setEncoderQuant(persistedModel.encoderQuant);
}
if (persistedModel?.decoderQuant !== undefined) {
appStore.setDecoderQuant(persistedModel.decoderQuant);
}
if (persistedGeneral?.energyThreshold !== undefined) {
appStore.setEnergyThreshold(persistedGeneral.energyThreshold);
}
Expand All @@ -249,6 +260,12 @@ const App: Component = () => {
if (persistedGeneral?.streamingWindow !== undefined) {
appStore.setStreamingWindow(persistedGeneral.streamingWindow);
}
if (persistedGeneral?.frameStride !== undefined) {
appStore.setFrameStride(persistedGeneral.frameStride);
}
if (persistedGeneral?.wasmThreads !== undefined) {
appStore.setWasmThreads(clampWasmThreadsForDevice(persistedGeneral.wasmThreads));
}
if (persistedUi?.debugPanel?.visible !== undefined) {
appStore.setShowDebugPanel(persistedUi.debugPanel.visible);
}
Expand Down Expand Up @@ -297,6 +314,26 @@ const App: Component = () => {
}
});

// Keep quantization presets aligned with the backend mode (parakeet.js demo behavior).
let lastBackendMode: 'webgpu-hybrid' | 'wasm' | null = null;
createEffect(() => {
Copy link

@kiloconnect kiloconnect bot Feb 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: createEffect silently discards user-persisted quantization choices

This effect fires immediately on mount (and on every modelBackendMode change), which means any encoderQuant/decoderQuant values restored from localStorage at lines 251–255 are immediately overwritten. For example, a user who saved backend: 'wasm' + encoderQuant: 'fp32' will always have encoderQuant reset to 'int8' on startup.

The effect also prevents the user from independently choosing fp32 encoder on WASM — the UI dropdowns appear editable but the effect immediately reverts them.

Safest fix: Remove this effect and instead apply the preset only when the backend mode changes (not on initial mount), or only apply defaults when the user has not explicitly set a quant value. Alternatively, document that quant is always derived from backend and make the dropdowns read-only / hidden when the preset is active.

const backendMode = appStore.modelBackendMode();
if (lastBackendMode === null) {
lastBackendMode = backendMode;
return;
}
if (backendMode === lastBackendMode) return;

if (backendMode.startsWith('webgpu')) {
if (appStore.encoderQuant() !== 'fp32') appStore.setEncoderQuant('fp32');
if (appStore.decoderQuant() !== 'int8') appStore.setDecoderQuant('int8');
} else {
if (appStore.encoderQuant() !== 'int8') appStore.setEncoderQuant('int8');
if (appStore.decoderQuant() !== 'int8') appStore.setDecoderQuant('int8');
}
lastBackendMode = backendMode;
});
Comment on lines 317 to 335
Copy link

@coderabbitai coderabbitai bot Feb 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Reactive effect overrides user-selected quantization values.

In SolidJS, this effect tracks modelBackendMode(), encoderQuant(), and decoderQuant() as dependencies. If a user manually changes encoder quantization via the UI (e.g., selects int8 while in webgpu-hybrid mode), the effect re-fires and immediately reverts it to fp32. The quantization selects in SettingsPanel.tsx (lines 131–154) are only disabled during loading — they should also be disabled (or hidden) when the effect is controlling them, to avoid confusing UX where user selections are silently overridden.

🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@src/App.tsx` around lines 326 - 336, The reactive createEffect in App.tsx
currently forces encoder/decoder quant values based only on
appStore.modelBackendMode(), which causes user selections
(appStore.encoderQuant()/appStore.decoderQuant()) to be overwritten; change the
logic so the effect only enforces presets when a new backend mode is selected
AND the user isn’t actively controlling the quant settings (e.g., add a flag in
appStore like isQuantAutoControlled or check a new store field quantControlMode
=== 'auto'|'manual'), and update the SettingsPanel quant selects to disable/hide
when the effect is controlling them; specifically adjust the createEffect that
calls appStore.setEncoderQuant and setDecoderQuant to respect the new control
flag and update SettingsPanel.tsx (the encoder/decoder select controls) to
reflect that flag so users aren’t silently overridden.


let dragStart = { x: 0, y: 0 };
let posStart = { x: 0, y: 0 };

Expand Down Expand Up @@ -364,9 +401,14 @@ const App: Component = () => {
v4InferenceIntervalMs: appStore.v4InferenceIntervalMs(),
v4SilenceFlushSec: appStore.v4SilenceFlushSec(),
streamingWindow: appStore.streamingWindow(),
frameStride: appStore.frameStride(),
wasmThreads: appStore.wasmThreads(),
},
model: {
selectedModelId: appStore.selectedModelId(),
backend: appStore.modelBackendMode(),
encoderQuant: appStore.encoderQuant(),
decoderQuant: appStore.decoderQuant(),
},
audio: {
selectedDeviceId,
Expand Down Expand Up @@ -425,6 +467,7 @@ const App: Component = () => {
appStore.setModelProgress(p.progress);
appStore.setModelMessage(p.message || '');
if (p.file) appStore.setModelFile(p.file);
if (p.backend) appStore.setBackend(p.backend);
Copy link
Contributor

@sourcery-ai sourcery-ai bot Feb 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue (bug_risk): appStore.setBackend is called but there is no corresponding signal/setter in the store API.

createAppStore defines modelBackendMode, encoderQuant, decoderQuant, and wasmThreads, but no backend/setBackend pair is exported. This call will not type‑check (or will hit an undefined property at runtime). If you need to track the resolved runtime backend separately from modelBackendMode, add a dedicated signal (e.g. [runtimeBackend, setRuntimeBackend]) to createAppStore and export the setter.

Copy link

@kiloconnect kiloconnect bot Feb 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: appStore.setBackend updates the runtime backend signal (the actual backend used), but appStore.modelBackendMode (the requested backend) is never updated when a WebGPU fallback to WASM occurs. After a fallback, modelBackendMode() still returns 'webgpu-hybrid' while backend() returns 'wasm'. On the next save-to-storage cycle (line 402–415), backend: 'webgpu-hybrid' is persisted, so the user's setting is not corrected. Consider also updating modelBackendMode when a fallback is detected, or at minimum surfacing the mismatch in the UI.

};

workerClient.onModelStateChange = (s) => {
Expand Down Expand Up @@ -642,6 +685,7 @@ const App: Component = () => {
features: features.features,
T: features.T,
melBins: features.melBins,
frameStride: appStore.frameStride(),
timeOffset,
endTime: window.endFrame / 16000,
segmentId: `v4_${Date.now()}`,
Expand Down Expand Up @@ -1040,11 +1084,20 @@ const App: Component = () => {

const loadSelectedModel = async () => {
if (!workerClient) return;
if (appStore.modelState() === 'ready') return;
if (appStore.modelState() === 'loading') return;
Copy link

@kiloconnect kiloconnect bot Feb 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: The guard if (appStore.modelState() === 'ready') return; was removed to allow model reload. However, there is no guard against reloading while transcription is actively running (recordingState() === 'recording'). Reloading the model mid-transcription will tear down the worker and drop in-flight audio. Consider adding:

Suggested change
if (appStore.modelState() === 'loading') return;
if (appStore.modelState() === 'loading') return;

and separately checking appStore.recordingState() !== 'idle' before allowing reload, or at minimum showing a warning.

if (appStore.recordingState() === 'recording') {
console.warn('[App] Model reload blocked while recording is active');
return;
}
setShowContextPanel(true);
try {
await workerClient.initModel(appStore.selectedModelId());
await workerClient.initModel({
modelId: appStore.selectedModelId(),
cpuThreads: appStore.wasmThreads(),
backend: appStore.modelBackendMode(),
encoderQuant: appStore.encoderQuant(),
decoderQuant: appStore.decoderQuant(),
});
} catch (e) {
console.error('Failed to load model:', e);
appStore.setModelState('error');
Expand Down Expand Up @@ -1078,7 +1131,10 @@ const App: Component = () => {
if (!workerClient) return;
setShowContextPanel(true);
try {
await workerClient.initLocalModel(files);
await workerClient.initLocalModel(files, {
cpuThreads: appStore.wasmThreads(),
backend: appStore.modelBackendMode(),
});
} catch (e) {
console.error('Failed to load local model:', e);
}
Expand Down
79 changes: 77 additions & 2 deletions src/components/SettingsPanel.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ import { Component, For, Show } from 'solid-js';
import { appStore } from '../stores/appStore';
import { getModelDisplayName, MODELS } from './ModelLoadingOverlay';
import type { AudioEngine } from '../lib/audio/types';
import { getMaxHardwareThreads } from '../utils/hardwareThreads';

const formatInterval = (ms: number) => {
if (ms >= 1000) return `${(ms / 1000).toFixed(1)}s`;
Expand Down Expand Up @@ -34,6 +35,7 @@ export interface SettingsContentProps {
export const SettingsContent: Component<SettingsContentProps> = (props) => {
const isV4 = () => appStore.transcriptionMode() === 'v4-utterance';
const isV3 = () => appStore.transcriptionMode() === 'v3-streaming';
const maxWasmThreads = () => getMaxHardwareThreads();

const expandUp = () => props.expandUp?.() ?? false;
const section = () => props.section ?? 'full';
Expand Down Expand Up @@ -65,11 +67,11 @@ export const SettingsContent: Component<SettingsContentProps> = (props) => {
<button
type="button"
onClick={props.onLoadModel}
disabled={appStore.modelState() === 'ready' || appStore.modelState() === 'loading'}
disabled={appStore.modelState() === 'loading'}
class="flex items-center gap-1.5 px-3 py-1.5 text-sm font-medium text-[var(--color-earthy-muted-green)] hover:bg-[var(--color-earthy-sage)]/20 transition-colors disabled:opacity-50 disabled:cursor-not-allowed shrink-0"
>
<span class="material-symbols-outlined text-lg">power_settings_new</span>
{appStore.modelState() === 'ready' ? 'Loaded' : appStore.modelState() === 'loading' ? '...' : 'Load'}
{appStore.modelState() === 'ready' ? 'Reload' : appStore.modelState() === 'loading' ? '...' : 'Load'}
</button>
<Show when={props.onLocalLoad}>
<label class="flex items-center gap-1.5 px-3 py-1.5 text-sm font-medium text-[var(--color-earthy-soft-brown)] hover:bg-[var(--color-earthy-sage)]/20 transition-colors cursor-pointer shrink-0">
Expand All @@ -92,6 +94,59 @@ export const SettingsContent: Component<SettingsContentProps> = (props) => {
<p class="text-xs text-[var(--color-earthy-soft-brown)]">
{appStore.modelState() === 'ready' ? getModelDisplayName(appStore.selectedModelId()) : appStore.modelState()}
</p>
<div class="grid grid-cols-2 gap-x-4 gap-y-3 pt-1">
<div class="space-y-1">
<span class="text-[10px] font-bold uppercase tracking-widest text-[var(--color-earthy-soft-brown)]">Backend</span>
<select
class="w-full text-sm bg-transparent border-b border-[var(--color-earthy-sage)]/40 px-0 py-1.5 text-[var(--color-earthy-dark-brown)] focus:outline-none focus:border-[var(--color-earthy-muted-green)]"
value={appStore.modelBackendMode()}
onInput={(e) => appStore.setModelBackendMode((e.target as HTMLSelectElement).value as 'webgpu-hybrid' | 'wasm')}
disabled={appStore.modelState() === 'loading'}
>
<option value="webgpu-hybrid">WebGPU</option>
<option value="wasm">WASM</option>
</select>
</div>
<div class="space-y-1">
<span class="text-[10px] font-bold uppercase tracking-widest text-[var(--color-earthy-soft-brown)]">Stride</span>
<input
type="number"
min="1"
max="4"
step="1"
value={appStore.frameStride()}
onInput={(e) => {
const next = Number((e.target as HTMLInputElement).value);
if (Number.isFinite(next)) appStore.setFrameStride(Math.max(1, Math.min(4, Math.round(next))));
}}
class="w-full text-sm bg-transparent border-b border-[var(--color-earthy-sage)]/40 px-0 py-1.5 text-[var(--color-earthy-dark-brown)] focus:outline-none focus:border-[var(--color-earthy-muted-green)]"
/>
</div>
<div class="space-y-1">
<span class="text-[10px] font-bold uppercase tracking-widest text-[var(--color-earthy-soft-brown)]">Encoder</span>
<select
class="w-full text-sm bg-transparent border-b border-[var(--color-earthy-sage)]/40 px-0 py-1.5 text-[var(--color-earthy-dark-brown)] focus:outline-none focus:border-[var(--color-earthy-muted-green)]"
value={appStore.encoderQuant()}
onInput={(e) => appStore.setEncoderQuant((e.target as HTMLSelectElement).value as 'int8' | 'fp32')}
disabled={appStore.modelState() === 'loading'}
>
<option value="fp32">fp32</option>
<option value="int8">int8</option>
</select>
</div>
<div class="space-y-1">
<span class="text-[10px] font-bold uppercase tracking-widest text-[var(--color-earthy-soft-brown)]">Decoder</span>
<select
class="w-full text-sm bg-transparent border-b border-[var(--color-earthy-sage)]/40 px-0 py-1.5 text-[var(--color-earthy-dark-brown)] focus:outline-none focus:border-[var(--color-earthy-muted-green)]"
value={appStore.decoderQuant()}
onInput={(e) => appStore.setDecoderQuant((e.target as HTMLSelectElement).value as 'int8' | 'fp32')}
disabled={appStore.modelState() === 'loading'}
>
<option value="int8">int8</option>
<option value="fp32">fp32</option>
</select>
</div>
</div>
Comment on lines +97 to +149
Copy link

@coderabbitai coderabbitai bot Feb 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

New configuration grid is well-structured.

Backend, Stride, Encoder, and Decoder controls are correctly wired to the app store. One note: since the reactive effect in App.tsx overrides encoder/decoder quant based on backend mode, consider adding visual indication (e.g., disabled or a tooltip) when quantization values are auto-managed by the backend selection — otherwise users may be confused when their selection reverts.

🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@src/components/SettingsPanel.tsx` around lines 103 - 155, The encoder/decoder
selects can be changed by the user but are later overridden by the reactive
effect in App.tsx when certain backend modes are selected; update the UI to
reflect that by disabling the Encoder and Decoder <select> controls (the ones
bound to appStore.encoderQuant() and appStore.decoderQuant()) whenever
appStore.modelBackendMode() indicates the backend auto-manages quantization, and
add a short tooltip/title like "Managed by backend selection" to those disabled
controls; keep existing disabled behavior tied to appStore.modelState() ===
'loading' and ensure you don't remove the onInput handlers
(setEncoderQuant/setDecoderQuant) so they remain usable when not auto-managed.

<Show when={appStore.modelState() === 'loading'}>
<div class="space-y-1">
<div class="flex justify-between text-xs">
Expand Down Expand Up @@ -134,6 +189,26 @@ export const SettingsContent: Component<SettingsContentProps> = (props) => {

<Show when={showSliders()}>
<section class="grid grid-cols-2 gap-x-4 gap-y-3">
<div class="space-y-1.5 min-w-0">
<div class="flex justify-between items-center gap-2">
<span class="text-[10px] font-bold uppercase tracking-widest text-[var(--color-earthy-soft-brown)]">WASM threads</span>
<span class="text-sm text-[var(--color-earthy-dark-brown)] tabular-nums shrink-0">{appStore.wasmThreads()} / {maxWasmThreads()}</span>
</div>
<input
type="range"
min="1"
max={maxWasmThreads()}
step="1"
value={Math.min(appStore.wasmThreads(), maxWasmThreads())}
onInput={(e) => {
const next = parseInt(e.currentTarget.value, 10);
appStore.setWasmThreads(Math.max(1, Math.min(maxWasmThreads(), next)));
}}
class="debug-slider w-full h-2 rounded-full appearance-none cursor-pointer bg-[var(--color-earthy-sage)]/30"
/>
<div class="text-[9px] text-[var(--color-earthy-soft-brown)]">Applied on next model load/reload.</div>
</div>

<div class="space-y-1.5 min-w-0">
<div class="flex justify-between items-center gap-2">
<span class="text-[10px] font-bold uppercase tracking-widest text-[var(--color-earthy-soft-brown)]">Energy threshold</span>
Expand Down
Loading