[TEMP] ingest perf

[nbiscaro]

Summary

Diff shows the method used for timing only the ingest path. We simply keep an event counter and print a timestamp when the desired number of events is reached. We skip the process_cdc_event call entirely.

Below are the results for both the ingest only path and the full end-to-end run (no code changes for end to end). For each we test and append only insert with 10M rows and 100M rows.

We build with cargo pgrx run --release.

Results for ingest only path:
pg = postres (shows both the insert time for postgres as well as the timestamp)
ml= moonlink
lag= time between postgres finishing and moonlink finishing

using the following knobs in pg:

max_wal_size = 10GB
wal_compression = pglz
checkpoint_completion_target = 0.9

Ingest only:

100M rows:

ingest only

pg:
Time: 116939.699 ms (01:56.940)            
-------------------------------
 2025-08-14 23:17:32.583118+00
 
 ml: "2025-08-14T23:17:57.363518011+00:00"

lag: 25

pg: 
Time: 115937.758 ms (01:55.938)         
-------------------------------
 2025-08-14 23:24:31.414931+00

ml:
"2025-08-14T23:24:59.526272804+00:00"

lag: 28

pg: 
Time: 114591.656 ms (01:54.592)          
------------------------------
 2025-08-14 23:28:24.98472+00

 ml: "2025-08-14T23:28:55.114134968+00:00"

lag: 27

End to End: Here we measure the time between the postgres insert and the SELECT * on moonlink

100M:

Time: 132470.657 ms (02:12.471)
 total_rows 
------------
  100000000
(1 row)

Time: 32215.291 ms (00:32.215)
pg_mooncake (pid: 4431) =# 

Time: 126627.748 ms (02:06.628)
 total_rows 
------------
  100000000
(1 row)

Time: 37435.062 ms (00:37.435)
pg_mooncake (pid: 2634) =# 

Time: 129537.795 ms (02:09.538)
 total_rows 
------------
  100000000
(1 row)

Time: 38021.843 ms (00:38.022)
pg_mooncake (pid: 2860) =# 

Flamegraphs:

Ingest only:

Related Issues

Closes # or links to related issues.

Changes

Checklist

• Code builds correctly
• Tests have been added or updated
• Documentation updated if necessary
• I have reviewed my own changes

[nbiscaro]

Done with testing.