@ruvector/rvf-node: compression option in RvfOptions is accepted but silently ignored #225
Description
Summary
The @ruvector/rvf SDK (v0.2.0) accepts a compression parameter in RvfOptions (types.ts line 32, CompressionProfile = 'none' | 'scalar' | 'product'), and backend.ts correctly maps it to native format via mapCompressionToNative() (line 676-698). However, the N-API binding in @ruvector/rvf-node (v0.1.7) does not read this field from the JavaScript options object.
Steps to Reproduce
import { RvfDatabase } from '@ruvector/rvf';
// Create with scalar compression
const db = await RvfDatabase.create('test.rvf', {
dimensions: 384,
metric: 'cosine',
compression: 'scalar', // <-- this is accepted but has no effect
});
// Ingest 100 random vectors
const entries = [];
for (let i = 0; i < 100; i++) {
const vec = new Float32Array(384);
for (let j = 0; j < 384; j++) vec[j] = Math.random() - 0.5;
entries.push({ id: String(i), vector: vec });
}
await db.ingestBatch(entries);
await db.close();
Expected Behavior
With compression: 'scalar', the VEC_SEG should store int8 quantized vectors (384 bytes per vector instead of 1,536). File size for 100 vectors should be ~40 KB of vector data.
Actual Behavior
The VEC_SEG stores raw Float32 regardless of the compression setting:
- 100 vectors × 384 dims × 4 bytes = 153,600 bytes expected for raw f32
- Actual VEC_SEG payload: 154,406 bytes (essentially raw f32 + minor overhead)
- File size: 155,002 bytes — ratio of 1.01x vs raw data (no compression applied)
Root Cause (from source analysis)
In rvf-node/src/lib.rs, the RvfOptions struct (approx lines 48-63) contains fields for dimension, metric, profile, signing, m, and ef_construction — but no compression field. The js_options_to_rust() function (lines 169-184) maps JS options to RustRvfOptions using ..Default::default(), so the compression value from JavaScript is never read.
The TypeScript SDK pipeline works correctly up to the N-API boundary:
types.tsdefinesCompressionProfile✅backend.tsmapCompressionToNative()maps'scalar'→'Scalar'✅backend.tsmapOptionsToNative()includescompression: 'Scalar'in the native options object ✅rvf-nodeN-API binding drops the compression field because the Rust struct doesn't have it ❌
Environment
- macOS arm64 (Apple Silicon, M3 Max)
- Node.js v22.13.1
@ruvector/rvfv0.2.0@ruvector/rvf-nodev0.1.7
Suggested Fix
Add a compression field to the RvfOptions struct in rvf-node/src/lib.rs and wire it through js_options_to_rust() to the RustRvfOptions struct in rvf-runtime, which already has the quantization infrastructure via rvf-quant.
Workaround
The WASM module exports rvf_load_sq_params and rvf_dequant_i8 for runtime dequantization, so it's possible to implement JS-side scalar quantization at build time and use the WASM dequantization path at query time.