Skip to content

Comments

Normalize historic benchmark data after adding OperationsPerInvocation annotation#8100

Merged
jack-berg merged 1 commit intoopen-telemetry:benchmarksfrom
jack-berg:fix-benchmark-history
Feb 18, 2026
Merged

Normalize historic benchmark data after adding OperationsPerInvocation annotation#8100
jack-berg merged 1 commit intoopen-telemetry:benchmarksfrom
jack-berg:fix-benchmark-history

Conversation

@jack-berg
Copy link
Member

@jack-berg jack-berg commented Feb 18, 2026

In #8031 we updated the benchmarks to output actual operation figures instead of aggregate operations (where each operation performs a loop of 1024*10 operations).

See comment1 and comment2.

This screws up the benchmark history because all the figures now look 1024*10x higher than previously.

This PR fixes the issue by multiplying all benchmark figures prior to #8031 by 1024*10.

Current benchmark:

Screenshot 2026-02-18 at 12 34 39 PM

Benchmark with this PR:

Screenshot 2026-02-18 at 12 34 32 PM
Expand to see untracked one-off script used to make the change
#!/usr/bin/env node
/**
 * Update benchmark data to multiply old values by 10240.
 * Data prior to commit bf3c4f3085d631b08d2cd210c1ddc3cd86182a53 needs to be multiplied.
 */

const fs = require('fs');
const path = require('path');

// Read the file
const filePath = path.join(__dirname, 'benchmarks', 'data.js');
const content = fs.readFileSync(filePath, 'utf8');

// Extract the JSON part (remove the JavaScript wrapper)
const match = content.match(/window\.BENCHMARK_DATA = ({.*});?\s*$/s);
if (!match) {
  throw new Error('Could not find BENCHMARK_DATA in file');
}

const data = JSON.parse(match[1]);

// Target commit where the change was made
const targetCommit = 'bf3c4f3085d631b08d2cd210c1ddc3cd86182a53';

// Process entries
const entries = data.entries || {};
let totalUpdated = 0;

for (const [benchmarkType, benchmarkEntries] of Object.entries(entries)) {
  let foundTarget = false;

  // Iterate through commits in order
  for (let i = 0; i < benchmarkEntries.length; i++) {
    const entry = benchmarkEntries[i];
    const commitId = entry.commit?.id || '';

    // Check if this is the target commit
    if (commitId === targetCommit) {
      foundTarget = true;
      console.log(`Found target commit at index ${i}: ${commitId}`);
      continue;
    }

    // If we haven't found the target commit yet, multiply the values
    if (!foundTarget) {
      const benches = entry.benches || [];
      for (const bench of benches) {
        if ('value' in bench) {
          const oldValue = bench.value;
          bench.value = oldValue * 10240;
          totalUpdated++;
          if (totalUpdated <= 10) {
            console.log(`  Multiplied ${oldValue.toFixed(2)} -> ${bench.value.toFixed(2)} for commit ${commitId.substring(0, 8)}`);
          }
        }
      }
    }
  }
}

// Write back to file
const outputContent = `window.BENCHMARK_DATA = ${JSON.stringify(data, null, 2)};\n`;
fs.writeFileSync(filePath, outputContent, 'utf8');

console.log(`\nUpdate complete! Updated ${totalUpdated} benchmark values.`);

@jack-berg jack-berg merged commit c375ff1 into open-telemetry:benchmarks Feb 18, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants