CIP-104: Add temporal ACS store for doing asOf lookups by dfordivam · Pull Request #4126 · hyperledger-labs/splice

dfordivam · 2026-02-25T09:54:04Z

Fixes #4118

Note this does not implement caching, only the DB based queries

Pull Request Checklist

Cluster Testing

If a cluster test is required, comment /cluster_test on this PR to request it, and ping someone with access to the DA-internal system to approve it.
If a hard-migration test is required (from the latest release), comment /hdm_test on this PR to request it, and ping someone with access to the DA-internal system to approve it.

PR Guidelines

Include any change that might be observable by our partners or affect their deployment in the release notes.
Specify fixed issues with Fixes #n, and mention issues worked on using #n
Include a screenshot for frontend-related PRs - see README or use your favorite screenshot tool

Merge Guidelines

Make the git commit message look sensible when squash-merging on GitHub (most likely: just copy your PR description).

dfordivam · 2026-02-25T10:00:32Z

@meiersi-da This is the basic implementation of ScanTcsStore for a quick round of review.

meiersi-da

Nice work!

meiersi-da · 2026-02-25T16:25:53Z

...mmon/src/main/resources/db/migration/canton-network/postgres/stable/V057__scan_tcs_store.sql

+);
+
+create index acs_store_archived_template_temporal
+    on acs_store_archived_template (store_id, migration_id, template_id_qualified_name, created_at, archived_at);


Suggested change

on acs_store_archived_template (store_id, migration_id, template_id_qualified_name, created_at, archived_at);

on acs_store_archived_template (store_id, migration_id, template_id_qualified_name, created_at) include (archived_at);

probably the better option to have cheaper internal btree nodes and statistics

actually I just realized that we will typically be looking up recent timestamps, which means changing the index to efficiently skip over past events is valuable.

Concretely

Suggested change

on acs_store_archived_template (store_id, migration_id, template_id_qualified_name, created_at, archived_at);

on acs_store_archived_template (store_id, migration_id, template_id_qualified_name, archived_at) include (created_at);

meiersi-da · 2026-02-25T16:26:49Z

...mmon/src/main/resources/db/migration/canton-network/postgres/stable/V057__scan_tcs_store.sql

+
+-- lookup FeaturedAppRight
+create index scan_tcs_store_active_sid_mid_tid_farp
+    on scan_tcs_store_active (store_id, migration_id, template_id_qualified_name, featured_app_right_provider)


Suggested change

on scan_tcs_store_active (store_id, migration_id, template_id_qualified_name, featured_app_right_provider)

on scan_tcs_store_active (store_id, migration_id, featured_app_right_provider)

seems enough or not?

I will investigate

also make the index partial to exclude NULLs. That ensures a small index size.

I got rid of this as discussed yesterday we won't be doing lookup on party IDs

meiersi-da · 2026-02-25T16:28:12Z

...resources/db/migration/canton-network/postgres/stable/V057__scan_rewards_reference_store.sql

+
+    -- record_time of the transaction that archived the contract, in micros since epoch.
+    archived_at bigint not null
+);


Do we need this given that we use like <actual_store_table_active> below?

This is used only for the unit test of DbTemporalAcsStoreTest

Consider renaming it to test_ instead of template_ then.

meiersi-da · 2026-02-25T16:28:42Z

...mmon/src/main/resources/db/migration/canton-network/postgres/stable/V057__scan_tcs_store.sql

+
+-- temporal query support: created_at + archived_at filtering for point-in-time lookups on the archive table
+create index scan_tcs_store_archived_temporal
+    on scan_tcs_store_archived (store_id, migration_id, template_id_qualified_name, created_at, archived_at);


Suggested change

on scan_tcs_store_archived (store_id, migration_id, template_id_qualified_name, created_at, archived_at);

on scan_tcs_store_archived (store_id, migration_id, template_id_qualified_name, created_at) include (archived_at);

Done, ie using archived_at in index

meiersi-da · 2026-02-25T16:31:16Z

apps/common/src/main/scala/org/lfdecentralizedtrust/splice/store/db/DbMultiDomainAcsStore.scala

+            )
+            .asRuntimeException()
+        }
+        .flatMap(_ => waitUntilRecordTimeReached(asOf))


@nicu-da : I remember you fixing some memory leak problems wrt this kind of recursive loop on futures. Is this construction here safe?

After the fix in https://github.com/DACH-NY/canton-network-node/pull/9212 it should be safe. Modern futures are also pretty safe in chaining this way as they will flatten the recursion loop but it can still cause issues. Main issue if the waitUntilRecordTimeReached is called from a "hotspot" in the ingestion pipeline with a asOf which is relatively far in the future, thus we will have large number of waitUntilRecordTimeReached running in loops for each ingestion from the looks of it.
Ideally we should reverse and just add a promise that gets fulfilled when the ingestion reaches the asOf timestamp, removing the constant looping.

At the moment the temporal store APIs have no external endpoints, and a single automation doing lookups, hence I went with the simpler approach. I doubt we will ever be exposing this to an external endpoint. The lookup record_time also would typically not be far in future. So I will let this be as it is for now.

The lookup record_time also would typically not be far in future. So I will let this be as it is for now.

This is likely true. It is though one of these things that's going to be expensive to rediscover as part of scalability work. Let's create at least a tech-debt item the Scalability milestone to switch to a promise-based implementation.

The situation where it may not be true are catch-up situations. The annoying thing is that these are exceptional, but operationally critical circumstances. So keeping them in mind during development is valuable.

Ok, I fixed this to work similar to how offset based wakeup of Futures is currently being done.

meiersi-da · 2026-02-25T16:43:48Z

apps/common/src/main/scala/org/lfdecentralizedtrust/splice/store/db/TemporalAcsQueries.scala

+import slick.jdbc.canton.SQLActionBuilder
+import slick.sql.SqlStreamingAction
+
+trait TemporalAcsQueries extends AcsQueries {


Side note: @adetokunbo -- renaming DbAppActiviyRecordStore to AppActivityRecordQueries would seem to be more clear, as it really is mostly a helper class to build the queries akin to the TemporalAcsQueries here.

Food for thought. Not key to change, but might help with code clarity.

meiersi-da · 2026-02-25T16:45:25Z

.../common/src/test/scala/org/lfdecentralizedtrust/splice/store/db/DbTemporalAcsStoreTest.scala

+  protected def c(i: Int): Contract[AppRewardCoupon.ContractId, AppRewardCoupon] =
+    appRewardCoupon(i, dsoParty, contractId = validContractId(i))
+
+  "DbTemporalAcsStore" should {


missing test for waitUntilRecordTimeReached

meiersi-da · 2026-02-25T16:46:29Z

...an/src/main/scala/org/lfdecentralizedtrust/splice/scan/store/db/DbScanTemporalAcsStore.scala

+
+import scala.concurrent.{ExecutionContext, Future}
+
+class DbScanTemporalAcsStore(


Consider using the Tcs as an abbreviation for TemporalContractStore analogous to Acs for ActiveContractStore.

meiersi-da · 2026-02-25T16:47:35Z

...c/main/scala/org/lfdecentralizedtrust/splice/scan/store/db/DbScanRewardsReferenceStore.scala

+  ): Future[Seq[ContractWithState[TCid, T]]] =
+    multiDomainAcsStore.listContractsAsOf(companion, asOf, limit)
+
+  def lookupFeaturedAppRightsAsOf(


we're missing the one for looking them up by party. Ideally as a batch call.

and in a later PR with caching.

Hmmm. did we need [Party] based lookup in our flow?

We'll need the featured app status for all confirmers in the verdict of a confirmation request. My intuition was that organizing the code such that we pre-resolve all of them in a single call is not too onerous and has better performance properties.

Happy to postpone that though as a performance optimization, so we get to a working implementation more quickly.

As discussed yesterday on call, we plan on doing a single AsOf lookup of all parties per round.

meiersi-da · 2026-02-25T16:49:10Z

apps/scan/src/main/scala/org/lfdecentralizedtrust/splice/scan/store/ScanTemporalAcsStore.scala

+import org.lfdecentralizedtrust.splice.store.{AppStore, MultiDomainAcsStore}
+import org.lfdecentralizedtrust.splice.store.db.AcsInterfaceViewRowData
+
+trait ScanTemporalAcsStore extends AppStore {


Consider repeating the comment wrt purpose here from design doc. Goal is to inform future readers as to the tight scoping of this. Perhaps a name that relates to that would help ScanRewardsReferenceDataStore might be an option.

Signed-off-by: Divam <dfordivam@gmail.com>

dfordivam

@meiersi-da I have added the fixes and pushed the branch over latest main. PTAL

dfordivam · 2026-02-27T08:19:57Z

...resources/db/migration/canton-network/postgres/stable/V057__scan_rewards_reference_store.sql

+
+    -- record_time of the transaction that archived the contract, in micros since epoch.
+    archived_at bigint not null
+);


dfordivam · 2026-02-27T08:20:55Z

...mmon/src/main/resources/db/migration/canton-network/postgres/stable/V057__scan_tcs_store.sql

+);
+
+create index acs_store_archived_template_temporal
+    on acs_store_archived_template (store_id, migration_id, template_id_qualified_name, created_at, archived_at);


dfordivam · 2026-02-27T08:22:00Z

...mmon/src/main/resources/db/migration/canton-network/postgres/stable/V057__scan_tcs_store.sql

+
+-- lookup FeaturedAppRight
+create index scan_tcs_store_active_sid_mid_tid_farp
+    on scan_tcs_store_active (store_id, migration_id, template_id_qualified_name, featured_app_right_provider)


I got rid of this as discussed yesterday we won't be doing lookup on party IDs

dfordivam · 2026-02-27T08:22:30Z

...mmon/src/main/resources/db/migration/canton-network/postgres/stable/V057__scan_tcs_store.sql

+
+-- temporal query support: created_at + archived_at filtering for point-in-time lookups on the archive table
+create index scan_tcs_store_archived_temporal
+    on scan_tcs_store_archived (store_id, migration_id, template_id_qualified_name, created_at, archived_at);


Done, ie using archived_at in index

dfordivam · 2026-02-27T08:23:37Z

apps/common/src/main/scala/org/lfdecentralizedtrust/splice/store/db/DbMultiDomainAcsStore.scala

+            )
+            .asRuntimeException()
+        }
+        .flatMap(_ => waitUntilRecordTimeReached(asOf))


Ok, I fixed this to work similar to how offset based wakeup of Futures is currently being done.

dfordivam · 2026-02-27T08:24:21Z

apps/common/src/main/scala/org/lfdecentralizedtrust/splice/store/db/DbMultiDomainAcsStore.scala

      }
    }

    override def ingestUpdateBatch(batch: NonEmptyList[TreeUpdateOrOffsetCheckpoint])(implicit


Actually I missed it. Added it now, along with a test.

dfordivam · 2026-02-27T08:24:41Z

.../common/src/test/scala/org/lfdecentralizedtrust/splice/store/db/DbTemporalAcsStoreTest.scala

+  protected def c(i: Int): Contract[AppRewardCoupon.ContractId, AppRewardCoupon] =
+    appRewardCoupon(i, dsoParty, contractId = validContractId(i))
+
+  "DbTemporalAcsStore" should {


dfordivam · 2026-02-27T08:24:52Z

...an/src/main/scala/org/lfdecentralizedtrust/splice/scan/store/db/DbScanTemporalAcsStore.scala

+
+import scala.concurrent.{ExecutionContext, Future}
+
+class DbScanTemporalAcsStore(


dfordivam · 2026-02-27T08:26:27Z

...c/main/scala/org/lfdecentralizedtrust/splice/scan/store/db/DbScanRewardsReferenceStore.scala

+  ): Future[Seq[ContractWithState[TCid, T]]] =
+    multiDomainAcsStore.listContractsAsOf(companion, asOf, limit)
+
+  def lookupFeaturedAppRightsAsOf(


As discussed yesterday on call, we plan on doing a single AsOf lookup of all parties per round.

dfordivam · 2026-02-27T08:26:36Z

apps/scan/src/main/scala/org/lfdecentralizedtrust/splice/scan/store/ScanTemporalAcsStore.scala

+import org.lfdecentralizedtrust.splice.store.{AppStore, MultiDomainAcsStore}
+import org.lfdecentralizedtrust.splice.store.db.AcsInterfaceViewRowData
+
+trait ScanTemporalAcsStore extends AppStore {


dfordivam self-assigned this Feb 25, 2026

dfordivam requested a review from meiersi-da February 25, 2026 09:59

meiersi-da reviewed Feb 25, 2026

View reviewed changes

dfordivam added 26 commits February 27, 2026 08:15

Add migration for acs_store_archived_template table

70b3022

Signed-off-by: Divam <dfordivam@gmail.com>

Add optional onDelete add to archive in DbMultiDomainAcsStore

bc482ea

Signed-off-by: Divam <dfordivam@gmail.com>

Add lastIngestedRecordTime tracking to DbMultiDomainAcsStore

da5a6bd

Signed-off-by: Divam <dfordivam@gmail.com>

Add TemporalAcsQueries + TemporalAcsStore

4ec50c2

Signed-off-by: Divam <dfordivam@gmail.com>

Add waitUntilRecordTimeReached and use in queries

86b73ba

Signed-off-by: Divam <dfordivam@gmail.com>

Add migration for scan_tcs_store_active and archived tables

c8f9d10

Signed-off-by: Divam <dfordivam@gmail.com>

Add DbScanTemporalAcsStore

8417325

Signed-off-by: Divam <dfordivam@gmail.com>

Add DbScanTemporalAcsStoreTest

a4fa25c

Signed-off-by: Divam <dfordivam@gmail.com>

Fix comment

82c29c1

Signed-off-by: Divam <dfordivam@gmail.com>

rename migration file

7312077

Signed-off-by: Divam <dfordivam@gmail.com>

move archived_at to index include

5d35f0b

Signed-off-by: Divam <dfordivam@gmail.com>

Track lastIngestedRecordTime per SynchronizerId instead of globally

d6fef45

Signed-off-by: Divam <dfordivam@gmail.com>

Also update lastIngestedRecordTimes from offset checkpoint

04c78e0

Signed-off-by: Divam <dfordivam@gmail.com>

Add waitUntilRecordTimeReached tests with multi-domain verification

ea3b4bf

Signed-off-by: Divam <dfordivam@gmail.com>

Remove index on featured_app_right_provider, as we don't require it ATM

0dd2f3a

Signed-off-by: Divam <dfordivam@gmail.com>

Rename acs_store_archived_template to acs_store_archived_test

b703474

Signed-off-by: Divam <dfordivam@gmail.com>

waitUntilRecordTimeReached uses signals for wakeup

d3712d0

Signed-off-by: Divam <dfordivam@gmail.com>

test minor

4dcece8

Signed-off-by: Divam <dfordivam@gmail.com>

Update index

94ab6f7

Signed-off-by: Divam <dfordivam@gmail.com>

Rename TemporalAcsStore to TcsStore

f511435

Signed-off-by: Divam <dfordivam@gmail.com>

Rename sql scan_tcs_store_ -> scan_rewards_reference_store_

638ae09

Signed-off-by: Divam <dfordivam@gmail.com>

Rename to ScanRewardsReference*

f0ac07c

Signed-off-by: Divam <dfordivam@gmail.com>

Rename to TcsQueries

01c83b2

Signed-off-by: Divam <dfordivam@gmail.com>

More Acs -> Tcs rename

f4decd3

Signed-off-by: Divam <dfordivam@gmail.com>

Add comment about ScanRewardsReferenceStore

0f345d2

Signed-off-by: Divam <dfordivam@gmail.com>

Format

e0e833f

Signed-off-by: Divam <dfordivam@gmail.com>

dfordivam force-pushed the dfordivam/cip-104-scan-tcs-store branch from cb68841 to e0e833f Compare February 27, 2026 08:18

dfordivam commented Feb 27, 2026

View reviewed changes

dfordivam changed the base branch from main to dfordivam/feat-cip-104-activity-records February 27, 2026 08:28

dfordivam deleted the branch dfordivam/feat-cip-104-activity-records March 5, 2026 06:01

dfordivam closed this Mar 5, 2026

	on acs_store_archived_template (store_id, migration_id, template_id_qualified_name, created_at, archived_at);
	on acs_store_archived_template (store_id, migration_id, template_id_qualified_name, created_at) include (archived_at);

	on scan_tcs_store_active (store_id, migration_id, template_id_qualified_name, featured_app_right_provider)
	on scan_tcs_store_active (store_id, migration_id, featured_app_right_provider)

	on scan_tcs_store_archived (store_id, migration_id, template_id_qualified_name, created_at, archived_at);
	on scan_tcs_store_archived (store_id, migration_id, template_id_qualified_name, created_at) include (archived_at);


		import scala.concurrent.{ExecutionContext, Future}

		class DbScanTemporalAcsStore(

Conversation

dfordivam commented Feb 25, 2026

Pull Request Checklist

Cluster Testing

PR Guidelines

Merge Guidelines

Uh oh!

dfordivam commented Feb 25, 2026

Uh oh!

meiersi-da left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

meiersi-da Feb 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dfordivam left a comment

Choose a reason for hiding this comment

meiersi-da Feb 26, 2026 •

edited

Loading