fix: correct buffer handling for base64 embedding decoding

- Fixed unsafe buffer handling that could cause dimension truncation
- Use DataView with proper byte order handling for Float32Array conversion
- This prevents reading beyond buffer boundaries and data corruption
- Affects all models using base64 encoding, not just Gemini

The previous implementation used buffer.buffer directly which could:
1. Read from wrong memory locations if buffer was a view
2. Cause dimension truncation for large embeddings (like 3072-dim)
3. Result in incorrect embedding values

Fixes #7348
This commit is contained in:
Roo Code 2025-08-23 19:20:08 +00:00
parent 32fc3d6498
commit 5d27110dea

View file

@ -278,8 +278,15 @@ export class OpenAICompatibleEmbedder implements IEmbedder {
if (typeof item.embedding === "string") {
const buffer = Buffer.from(item.embedding, "base64")
// Create Float32Array view over the buffer
const float32Array = new Float32Array(buffer.buffer, buffer.byteOffset, buffer.byteLength / 4)
// Safe approach: Create Float32Array from a properly aligned copy
// This avoids issues with Node.js Buffers that may be views into larger ArrayBuffers
const float32Array = new Float32Array(buffer.length / 4)
const dataView = new DataView(buffer.buffer, buffer.byteOffset, buffer.byteLength)
// Read floats with proper byte order handling (little-endian)
for (let i = 0; i < float32Array.length; i++) {
float32Array[i] = dataView.getFloat32(i * 4, true)
}
return {
...item,