Skip to content

Conversation

@seanmcgary
Copy link
Member

@seanmcgary seanmcgary commented May 8, 2025

Description

Remove suffix of gold tables and add generated_rewards_snapshot_id column. This will prune gold tables and improve on table sizes.

Fixes #403

Type of change

Please delete options that are not relevant.

  • Performance improvement

How Has This Been Tested?

--- PASS: Test_RewardsV2_1 (1657.66s)
    --- PASS: Test_RewardsV2_1/Should_initialize_the_rewards_calculator (1640.35s)
PASS
ok  	github.com/Layr-Labs/sidecar/pkg/rewards	1657.949s

Checklist:

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • Any dependent changes have been merged and published in downstream modules
  • I have checked my code and corrected any misspellings

@serichoi65 serichoi65 force-pushed the sm-rewardsTablePerf branch 3 times, most recently from 83d6e1d to 3d9eb27 Compare May 30, 2025 19:30
@serichoi65 serichoi65 changed the title wip - combined rewards tables perf: combine rewards tables by removing snapshot date suffix Jun 2, 2025
@serichoi65 serichoi65 marked this pull request as ready for review June 4, 2025 13:54
@serichoi65 serichoi65 requested a review from a team as a code owner June 4, 2025 13:54
@serichoi65 serichoi65 requested review from 0xrajath and serichoi65 June 4, 2025 13:54
@serichoi65 serichoi65 force-pushed the sm-rewardsTablePerf branch 2 times, most recently from 9f42954 to 1335f78 Compare June 4, 2025 17:33
@serichoi65 serichoi65 force-pushed the sm-rewardsTablePerf branch from 1335f78 to 7527e61 Compare June 18, 2025 19:52
Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: SQL Template Syntax Error

The DeleteCorruptedRewardsFromBlockHeight function incorrectly uses Go template syntax {{.generatedRewardsSnapshotId}} within raw SQL query strings. These queries are then executed with sql.Named parameters, which expect SQL placeholder syntax (e.g., @generatedRewardsSnapshotId). This mismatch results in literal {{.generatedRewardsSnapshotId}} text in the executed SQL, causing syntax errors. The correct approach is to use @generatedRewardsSnapshotId for SQL named parameters or properly render the string as a template.

Additionally, if no previous snapshot is found (i.e., gorm.ErrRecordNotFound for previousSnapshot), the previousSnapshot variable remains nil, leading to a panic when previousSnapshot.Id is subsequently accessed.

pkg/rewards/rewards.go#L410-L431

for _, tableName := range rewardsUtils.RewardsTableBaseNames {
rc.logger.Sugar().Infow("Deleting rows from rewards table", "tableName", tableName)
dropQuery := fmt.Sprintf("delete from %s where generated_rewards_snapshot_id >= {{.generatedRewardsSnapshotId}}", tableName)
res := rc.grm.Exec(dropQuery, sql.Named("generatedRewardsSnapshotId", previousSnapshot.Id))
if res.Error != nil {
rc.logger.Sugar().Errorw("Failed to delete rows from rewards table", "error", res.Error, "tableName", tableName)
return res.Error
}
rc.logger.Sugar().Infow("Deleted rows from rewards table",
"tableName", tableName,
"recordsDeleted", res.RowsAffected)
}
// Also delete from generated_rewards_snapshots table
res = rc.grm.Exec("delete from generated_rewards_snapshots where id >= {{.generatedRewardsSnapshotId}}",
sql.Named("generatedRewardsSnapshotId", previousSnapshot.Id))
if res.Error != nil {
rc.logger.Sugar().Errorw("Failed to delete from generated_rewards_snapshots", "error", res.Error)
return res.Error
}
return nil

Fix in Cursor


Bug: Incorrect Table Pattern Causes Migration Failure

The ExistingTablePattern for the rewards_gold_10_avs_od_reward_amounts sub-migration is incorrectly set to gold_[0-9]+_staker_avs_od_reward_amounts_[0-9_]+$. The pattern includes an erroneous "staker_" prefix and should be gold_[0-9]+_avs_od_reward_amounts_[0-9_]+$. This prevents the migration from finding and processing the correct existing avs_od_reward_amounts tables.

pkg/postgres/migrations/202505301218_migrateRewardsTables/up.go#L354-L355

NewTableName: "rewards_gold_10_avs_od_reward_amounts",
ExistingTablePattern: "gold_[0-9]+_staker_avs_od_reward_amounts_[0-9_]+$",

Fix in Cursor


Was this report helpful? Give feedback by reacting with 👍 or 👎

@serichoi65 serichoi65 force-pushed the sm-rewardsTablePerf branch 4 times, most recently from 6e92d01 to d49be48 Compare July 2, 2025 17:41
@serichoi65 serichoi65 force-pushed the sm-rewardsTablePerf branch 2 times, most recently from 58ba633 to c6cb895 Compare July 8, 2025 15:29
@serichoi65 serichoi65 force-pushed the sm-rewardsTablePerf branch from 6ba2306 to 7e6ac5b Compare July 14, 2025 19:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

perf: prune rewards tables

3 participants