stagedsync, state: add prune backlog detection for CommitmentDomain and ChangeSets3#98
stagedsync, state: add prune backlog detection for CommitmentDomain and ChangeSets3#98HugoDutchie wants to merge 1 commit into0xPolygon:release/3.2from
Conversation
|
any updates on review? for me not a problem but it seems the initial beta release was not sufficient to solve the problem for other people. Its stable on my node for over a week now (with this pr code included) |
|
Hey @HugoDutchie - sorry for the delay, people from our team will review it shortly. |
|
I actually notice there is still a problem with mdbx growth and node slowing down. It now just takes 10+ days to show trough instead of the earlier 2-5 days. This seems like an issue in how erigon handles mdbx cleanup and potentially its internal structure. It also seems a problem seen by other clients using mdbx. I don't know where to report this or if there are other investigations going. Seems like something for upstream erigon team to look into, if not already. |
Complementary fix to #92. Extends prune backlog detection to additional components that can fall behind during extended sync periods.
Changes
db/state/aggregator.go
CommitmentBacklogInfo()to detect CommitmentDomain history pruning backlogexecution/stagedsync/exec3.go
GreedyPruneHistorywhen CommitmentDomain is >10M txNums behindexecution/stagedsync/stage_execute.go
>1M blocks behind→ aggressive (unlimited, 5min timeout)>100K blocks behind→ medium (100K limit, 5min timeout)Testing
Tested on Polygon mainnet node recovering from prune backlog.
MDBX.dat now seems to be stable and only growing small steps every once and a while. Not affecting node being able to stay on chaintip.