Optimize Incremental Mel Calculation

[ysdede]

This PR optimizes the IncrementalMelProcessor in src/mel.js to significantly improve performance for streaming audio processing.

Key Changes

1. JsPreprocessor.computeRawMel: Added a startFrame parameter. When provided, the STFT/Mel loop starts from this frame index, leaving earlier frames as zeros (which are later filled from cache). This avoids recomputing frames that are already known.
2. IncrementalMelProcessor.process (Incremental Path): When a valid cache prefix is found (safeFrames > 0), computeRawMel is called with startFrame = safeFrames. This skips the heavy computation for the overlapping part of the window.
3. IncrementalMelProcessor.process (Full Path): Refactored the logic when no cache is available. Previously, it called process (compute features) AND computeRawMel (compute raw for cache), doubling the work. Now it computes raw mel once, normalizes it, and updates the cache.

Performance Impact

Benchmarks on tests/mel.test.mjs show significant improvements:

• Incremental Path (70% overlap): Reduced from 78.6ms to 26.1ms (~3x speedup). The cost is now proportional to the new audio only.
• Full Path (First call): Reduced from 153.6ms to 78.3ms (~2x speedup) by eliminating redundant computation.

These changes make the processor much more efficient for real-time applications.

---

PR created automatically by Jules for task 5930782711649377570 started by @ysdede

Summary by CodeRabbit

• New Features

  • Added support for flexible, partial audio feature extraction with configurable starting points

• Performance Improvements

  • Optimized Mel feature computation with enhanced caching and incremental processing for efficient audio analysis

[coderabbitai]

📝 Walkthrough

Walkthrough

The changes enable partial Mel feature extraction by adding an optional startFrame parameter to the computeRawMel method, allowing frame-wise computation to start at any offset. The IncrementalMelProcessor now explicitly separates rawMel computation from normalization, caching intermediate results for efficient incremental reuse.

Changes

Cohort / File(s)	Summary
Mel Processing Refactor src/mel.js	Updated computeRawMel to accept optional startFrame parameter (default 0) for partial frame processing. Refactored IncrementalMelProcessor.process to explicitly separate rawMel computation from normalization, implementing both full recomputation and incremental caching paths with intermediate result memoization.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Poem

🐰 Hop, hop—frames need not start at zero!
A startFrame hero saves the day,
Raw mel computed partial-way,
Normalized with flair and cache—
Incremental processing in a flash! 🎵✨

🚥 Pre-merge checks | ✅ 3

✅ Passed checks (3 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'Optimize Incremental Mel Calculation' directly and accurately summarizes the main change—improving performance of incremental Mel computation through a startFrame parameter and refactored processing logic.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing touches

• 📝 Generate docstrings

🧪 Generate unit tests (beta)

✅ Unit Test PR creation complete.

• Create PR with unit tests
• Commit unit tests in branch perf-incremental-mel-5930782711649377570
• Post copyable unit tests in a comment

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

src/mel.js (1)

556-578: ⚠️ Potential issue | 🟡 Minor

Potential out-of-bounds cache copy if the audio window shrinks between calls.

safeFrames is bounded by this._cachedFeaturesLen (from the previous call), but the new call's nFrames could be smaller if the current audio is shorter. In that case, the copy loop at lines 575–576 writes to rawMel[m * nFrames + t] where t can reach safeFrames - 1 >= nFrames, silently overwriting the start of the next mel bin's row in the flat array (or writing past the end for the last bin). In JavaScript typed arrays, indices beyond the buffer length are silently ignored, so there's no crash — but the resulting features would be silently corrupted.

If the streaming contract guarantees non-shrinking windows, a comment or assertion documenting that assumption would be valuable. Otherwise, clamp safeFrames against the new nFrames/featuresLen.

🛡️ Proposed fix — clamp safeFrames against the new frame count

     const { rawMel, nFrames, featuresLen } =
       this.preprocessor.computeRawMel(audio, safeFrames);
 
+    // Ensure safeFrames doesn't exceed new frame dimensions
+    const clampedSafeFrames = Math.min(safeFrames, featuresLen, nFrames);
+
     // Replace first safeFrames with cached values (they're identical)
-    if (safeFrames > 0 && this._cachedRawMel) {
+    if (clampedSafeFrames > 0 && this._cachedRawMel) {
       for (let m = 0; m < this.nMels; m++) {
         const srcBase = m * this._cachedNFrames;
         const dstBase = m * nFrames;
-        for (let t = 0; t < safeFrames; t++) {
+        for (let t = 0; t < clampedSafeFrames; t++) {
           rawMel[dstBase + t] = this._cachedRawMel[srcBase + t];
         }
       }
     }

Also update the return value to use clampedSafeFrames for the cachedFrames and newFrames counts.

🧹 Nitpick comments (2)

src/mel.js (2)

359-398: Consider adding a bounds check for startFrame.

If a caller passes a startFrame that exceeds nFrames, the loop silently produces an all-zero rawMel. A negative startFrame would cause offset = t * HOP_LENGTH to index into padded at negative offsets, producing NaN. While current internal callers always pass a non-negative, bounded value (via Math.max(0, ...)), a defensive clamp would protect this public method.

🛡️ Proposed defensive clamp

  computeRawMel(audio, startFrame = 0) {
+   startFrame = Math.max(0, Math.min(startFrame, Number.MAX_SAFE_INTEGER));
    const N = audio.length;

And after nFrames is computed:

    const nFrames = Math.floor((paddedLen - N_FFT) / HOP_LENGTH) + 1;
    const featuresLen = Math.floor(N / HOP_LENGTH);
+   startFrame = Math.min(startFrame, nFrames);

---

497-502: reset() and clear() are identical — consider removing one.

Both methods set the exact same four fields to the same default values. Having two public methods with the same behavior is confusing for consumers. Consider keeping one (e.g., reset()) and removing the other, or having one delegate to the other.

Also applies to: 606-611

[coderabbitai]

Note

Unit test generation is a public access feature. Expect some limitations and changes as we gather feedback and continue to improve it.

---

Generating unit tests... This may take up to 20 minutes.

[coderabbitai]

Note

Unit test generation is a public access feature. Expect some limitations and changes as we gather feedback and continue to improve it.

---

Generating unit tests... This may take up to 20 minutes.

[coderabbitai]

Failed to clone repository into sandbox.

[coderabbitai]

Note

Unit test generation is a public access feature. Expect some limitations and changes as we gather feedback and continue to improve it.

---

Generating unit tests... This may take up to 20 minutes.

[coderabbitai]

Failed to clone repository into sandbox.

[coderabbitai]

✅ Created PR with unit tests: #33

[ysdede]

Summary: Adds startFrame support and avoids double compute in incremental mel; full path now normalizes raw mel once.
Recommendation: Ready to merge.
Tests: npm test

[ysdede]

Handled in PR#34 (release branch). Closing superseded PR.

[google-labs-jules]

Handled in PR#34 (release branch). Closing superseded PR.

Understood. Acknowledging that this work is now superseded by PR#34 and stopping work on this task.