H-5742: Dynamic Workflow Compiler + Demo

[lunelson]

WIP

This PR builds out the decomposition workflow as a Mastra workflow with "revision loop" control-flow

• All 4 fixtures pass on first attempt (no revisions needed)
• Revision loop infrastructure is ready if/when needed
• 4/4 valid plans, ~42s total

What's working well:

• Deterministic validation catches structural issues
• Planner agent generates valid plans reliably
• Revision feedback is ready for edge cases

Demo

[github-actions]

Benchmark results

@rust/hash-graph-benches – Integrations

policy_resolution_large

Function	Value	Mean	Flame graphs
resolve_policies_for_actor	user: empty, selectivity: high, policies: 2002	$$28.3 \mathrm{ms} \pm 186 \mathrm{μs}\left({\color{gray}1.66 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: empty, selectivity: low, policies: 1	$$3.43 \mathrm{ms} \pm 18.9 \mathrm{μs}\left({\color{gray}2.16 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: empty, selectivity: medium, policies: 1001	$$14.4 \mathrm{ms} \pm 137 \mathrm{μs}\left({\color{red}12.7 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: seeded, selectivity: high, policies: 3314	$$43.2 \mathrm{ms} \pm 311 \mathrm{μs}\left({\color{gray}-4.887 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: seeded, selectivity: low, policies: 1	$$14.5 \mathrm{ms} \pm 124 \mathrm{μs}\left({\color{gray}-0.971 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: seeded, selectivity: medium, policies: 1526	$$25.7 \mathrm{ms} \pm 201 \mathrm{μs}\left({\color{red}8.09 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: system, selectivity: high, policies: 2078	$$33.0 \mathrm{ms} \pm 219 \mathrm{μs}\left({\color{lightgreen}-22.644 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: system, selectivity: low, policies: 1	$$3.98 \mathrm{ms} \pm 33.7 \mathrm{μs}\left({\color{lightgreen}-80.203 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: system, selectivity: medium, policies: 1033	$$15.1 \mathrm{ms} \pm 131 \mathrm{μs}\left({\color{lightgreen}-45.860 \mathrm{\%}}\right) $$	Flame Graph

policy_resolution_medium

Function	Value	Mean	Flame graphs
resolve_policies_for_actor	user: empty, selectivity: high, policies: 102	$$4.32 \mathrm{ms} \pm 37.9 \mathrm{μs}\left({\color{red}17.7 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: empty, selectivity: low, policies: 1	$$3.10 \mathrm{ms} \pm 23.5 \mathrm{μs}\left({\color{red}7.43 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: empty, selectivity: medium, policies: 51	$$3.77 \mathrm{ms} \pm 37.5 \mathrm{μs}\left({\color{red}16.9 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: seeded, selectivity: high, policies: 269	$$5.46 \mathrm{ms} \pm 47.5 \mathrm{μs}\left({\color{red}9.46 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: seeded, selectivity: low, policies: 1	$$3.77 \mathrm{ms} \pm 30.6 \mathrm{μs}\left({\color{red}10.1 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: seeded, selectivity: medium, policies: 107	$$4.28 \mathrm{ms} \pm 33.4 \mathrm{μs}\left({\color{red}8.56 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: system, selectivity: high, policies: 133	$$4.53 \mathrm{ms} \pm 27.0 \mathrm{μs}\left({\color{red}5.74 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: system, selectivity: low, policies: 1	$$3.50 \mathrm{ms} \pm 23.3 \mathrm{μs}\left({\color{red}6.52 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: system, selectivity: medium, policies: 63	$$4.13 \mathrm{ms} \pm 30.3 \mathrm{μs}\left({\color{gray}4.15 \mathrm{\%}}\right) $$	Flame Graph

policy_resolution_none

Function	Value	Mean	Flame graphs
resolve_policies_for_actor	user: empty, selectivity: high, policies: 2	$$2.77 \mathrm{ms} \pm 21.3 \mathrm{μs}\left({\color{red}15.2 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: empty, selectivity: low, policies: 1	$$2.66 \mathrm{ms} \pm 14.5 \mathrm{μs}\left({\color{red}12.9 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: empty, selectivity: medium, policies: 1	$$2.97 \mathrm{ms} \pm 22.8 \mathrm{μs}\left({\color{red}18.8 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: system, selectivity: high, policies: 8	$$3.22 \mathrm{ms} \pm 25.6 \mathrm{μs}\left({\color{red}19.3 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: system, selectivity: low, policies: 1	$$2.97 \mathrm{ms} \pm 20.1 \mathrm{μs}\left({\color{red}11.9 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: system, selectivity: medium, policies: 3	$$3.31 \mathrm{ms} \pm 28.6 \mathrm{μs}\left({\color{red}17.1 \mathrm{\%}}\right) $$	Flame Graph

policy_resolution_small

Function	Value	Mean	Flame graphs
resolve_policies_for_actor	user: empty, selectivity: high, policies: 52	$$3.31 \mathrm{ms} \pm 29.6 \mathrm{μs}\left({\color{red}17.5 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: empty, selectivity: low, policies: 1	$$3.05 \mathrm{ms} \pm 21.6 \mathrm{μs}\left({\color{red}25.1 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: empty, selectivity: medium, policies: 25	$$3.13 \mathrm{ms} \pm 25.8 \mathrm{μs}\left({\color{red}20.8 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: seeded, selectivity: high, policies: 94	$$3.90 \mathrm{ms} \pm 37.4 \mathrm{μs}\left({\color{red}22.4 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: seeded, selectivity: low, policies: 1	$$3.08 \mathrm{ms} \pm 18.9 \mathrm{μs}\left({\color{red}15.5 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: seeded, selectivity: medium, policies: 26	$$3.52 \mathrm{ms} \pm 30.7 \mathrm{μs}\left({\color{red}15.9 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: system, selectivity: high, policies: 66	$$3.77 \mathrm{ms} \pm 32.6 \mathrm{μs}\left({\color{red}24.5 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: system, selectivity: low, policies: 1	$$3.38 \mathrm{ms} \pm 30.6 \mathrm{μs}\left({\color{red}28.4 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: system, selectivity: medium, policies: 29	$$3.52 \mathrm{ms} \pm 31.2 \mathrm{μs}\left({\color{red}21.6 \mathrm{\%}}\right) $$	Flame Graph

read_scaling_complete

Function	Value	Mean	Flame graphs
entity_by_id;one_depth	1 entities	$$39.4 \mathrm{ms} \pm 245 \mathrm{μs}\left({\color{gray}1.66 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id;one_depth	10 entities	$$76.2 \mathrm{ms} \pm 360 \mathrm{μs}\left({\color{lightgreen}-5.612 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id;one_depth	25 entities	$$42.7 \mathrm{ms} \pm 167 \mathrm{μs}\left({\color{gray}-3.667 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id;one_depth	5 entities	$$46.5 \mathrm{ms} \pm 232 \mathrm{μs}\left({\color{gray}1.26 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id;one_depth	50 entities	$$55.1 \mathrm{ms} \pm 338 \mathrm{μs}\left({\color{gray}0.961 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id;two_depth	1 entities	$$40.3 \mathrm{ms} \pm 163 \mathrm{μs}\left({\color{gray}-1.236 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id;two_depth	10 entities	$$417 \mathrm{ms} \pm 816 \mathrm{μs}\left({\color{gray}-0.447 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id;two_depth	25 entities	$$97.9 \mathrm{ms} \pm 409 \mathrm{μs}\left({\color{gray}2.57 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id;two_depth	5 entities	$$86.8 \mathrm{ms} \pm 386 \mathrm{μs}\left({\color{gray}1.38 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id;two_depth	50 entities	$$315 \mathrm{ms} \pm 639 \mathrm{μs}\left({\color{gray}-0.433 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id;zero_depth	1 entities	$$16.2 \mathrm{ms} \pm 98.7 \mathrm{μs}\left({\color{gray}4.20 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id;zero_depth	10 entities	$$15.4 \mathrm{ms} \pm 110 \mathrm{μs}\left({\color{lightgreen}-10.724 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id;zero_depth	25 entities	$$15.7 \mathrm{ms} \pm 83.3 \mathrm{μs}\left({\color{lightgreen}-9.479 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id;zero_depth	5 entities	$$15.2 \mathrm{ms} \pm 106 \mathrm{μs}\left({\color{gray}-2.127 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id;zero_depth	50 entities	$$17.7 \mathrm{ms} \pm 117 \mathrm{μs}\left({\color{lightgreen}-5.645 \mathrm{\%}}\right) $$	Flame Graph

read_scaling_linkless

Function	Value	Mean	Flame graphs
entity_by_id	1 entities	$$15.6 \mathrm{ms} \pm 99.4 \mathrm{μs}\left({\color{lightgreen}-11.822 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id	10 entities	$$15.1 \mathrm{ms} \pm 75.1 \mathrm{μs}\left({\color{lightgreen}-16.840 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id	100 entities	$$15.2 \mathrm{ms} \pm 84.8 \mathrm{μs}\left({\color{gray}-4.145 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id	1000 entities	$$15.8 \mathrm{ms} \pm 105 \mathrm{μs}\left({\color{gray}-2.171 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id	10000 entities	$$23.1 \mathrm{ms} \pm 171 \mathrm{μs}\left({\color{gray}2.42 \mathrm{\%}}\right) $$	Flame Graph

representative_read_entity

Function	Value	Mean	Flame graphs
entity_by_id	entity type ID: https://blockprotocol.org/@alice/types/entity-type/block/v/1	$$30.3 \mathrm{ms} \pm 353 \mathrm{μs}\left({\color{gray}-2.655 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id	entity type ID: https://blockprotocol.org/@alice/types/entity-type/book/v/1	$$29.8 \mathrm{ms} \pm 296 \mathrm{μs}\left({\color{gray}-2.268 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id	entity type ID: https://blockprotocol.org/@alice/types/entity-type/building/v/1	$$30.1 \mathrm{ms} \pm 315 \mathrm{μs}\left({\color{gray}3.18 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id	entity type ID: https://blockprotocol.org/@alice/types/entity-type/organization/v/1	$$30.4 \mathrm{ms} \pm 273 \mathrm{μs}\left({\color{gray}2.68 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id	entity type ID: https://blockprotocol.org/@alice/types/entity-type/page/v/2	$$29.3 \mathrm{ms} \pm 379 \mathrm{μs}\left({\color{gray}-0.910 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id	entity type ID: https://blockprotocol.org/@alice/types/entity-type/person/v/1	$$30.0 \mathrm{ms} \pm 287 \mathrm{μs}\left({\color{gray}1.18 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id	entity type ID: https://blockprotocol.org/@alice/types/entity-type/playlist/v/1	$$30.0 \mathrm{ms} \pm 359 \mathrm{μs}\left({\color{gray}2.84 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id	entity type ID: https://blockprotocol.org/@alice/types/entity-type/song/v/1	$$29.9 \mathrm{ms} \pm 297 \mathrm{μs}\left({\color{lightgreen}-5.305 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id	entity type ID: https://blockprotocol.org/@alice/types/entity-type/uk-address/v/1	$$30.5 \mathrm{ms} \pm 368 \mathrm{μs}\left({\color{gray}4.41 \mathrm{\%}}\right) $$	Flame Graph

representative_read_entity_type

Function	Value	Mean	Flame graphs
get_entity_type_by_id	Account ID: bf5a9ef5-dc3b-43cf-a291-6210c0321eba	$$8.17 \mathrm{ms} \pm 32.2 \mathrm{μs}\left({\color{gray}0.618 \mathrm{\%}}\right) $$	Flame Graph

representative_read_multiple_entities

Function	Value	Mean	Flame graphs
entity_by_property	traversal_paths=0	0	$$45.7 \mathrm{ms} \pm 216 \mathrm{μs}\left({\color{gray}-1.020 \mathrm{\%}}\right) $$
entity_by_property	traversal_paths=255	1,resolve_depths=inherit:1;values:255;properties:255;links:127;link_dests:126;type:true	$$97.7 \mathrm{ms} \pm 595 \mathrm{μs}\left({\color{gray}1.94 \mathrm{\%}}\right) $$
entity_by_property	traversal_paths=2	1,resolve_depths=inherit:0;values:0;properties:0;links:0;link_dests:0;type:false	$$51.9 \mathrm{ms} \pm 388 \mathrm{μs}\left({\color{gray}1.28 \mathrm{\%}}\right) $$
entity_by_property	traversal_paths=2	1,resolve_depths=inherit:0;values:0;properties:0;links:1;link_dests:0;type:true	$$60.6 \mathrm{ms} \pm 334 \mathrm{μs}\left({\color{gray}0.087 \mathrm{\%}}\right) $$
entity_by_property	traversal_paths=2	1,resolve_depths=inherit:0;values:0;properties:2;links:1;link_dests:0;type:true	$$69.0 \mathrm{ms} \pm 357 \mathrm{μs}\left({\color{gray}-3.311 \mathrm{\%}}\right) $$
entity_by_property	traversal_paths=2	1,resolve_depths=inherit:0;values:2;properties:2;links:1;link_dests:0;type:true	$$75.6 \mathrm{ms} \pm 404 \mathrm{μs}\left({\color{gray}2.29 \mathrm{\%}}\right) $$
link_by_source_by_property	traversal_paths=0	0	$$51.6 \mathrm{ms} \pm 338 \mathrm{μs}\left({\color{gray}-0.560 \mathrm{\%}}\right) $$
link_by_source_by_property	traversal_paths=255	1,resolve_depths=inherit:1;values:255;properties:255;links:127;link_dests:126;type:true	$$79.6 \mathrm{ms} \pm 417 \mathrm{μs}\left({\color{lightgreen}-5.037 \mathrm{\%}}\right) $$
link_by_source_by_property	traversal_paths=2	1,resolve_depths=inherit:0;values:0;properties:0;links:0;link_dests:0;type:false	$$58.6 \mathrm{ms} \pm 322 \mathrm{μs}\left({\color{gray}-0.486 \mathrm{\%}}\right) $$
link_by_source_by_property	traversal_paths=2	1,resolve_depths=inherit:0;values:0;properties:0;links:1;link_dests:0;type:true	$$67.8 \mathrm{ms} \pm 339 \mathrm{μs}\left({\color{gray}-1.501 \mathrm{\%}}\right) $$
link_by_source_by_property	traversal_paths=2	1,resolve_depths=inherit:0;values:0;properties:2;links:1;link_dests:0;type:true	$$67.4 \mathrm{ms} \pm 318 \mathrm{μs}\left({\color{lightgreen}-7.766 \mathrm{\%}}\right) $$
link_by_source_by_property	traversal_paths=2	1,resolve_depths=inherit:0;values:2;properties:2;links:1;link_dests:0;type:true	$$68.0 \mathrm{ms} \pm 360 \mathrm{μs}\left({\color{gray}-1.411 \mathrm{\%}}\right) $$

scenarios

Function	Value	Mean	Flame graphs
full_test	query-limited	$$142 \mathrm{ms} \pm 616 \mathrm{μs}\left({\color{red}7.48 \mathrm{\%}}\right) $$	Flame Graph
full_test	query-unlimited	$$140 \mathrm{ms} \pm 725 \mathrm{μs}\left({\color{red}7.78 \mathrm{\%}}\right) $$	Flame Graph
linked_queries	query-limited	$$44.6 \mathrm{ms} \pm 237 \mathrm{μs}\left({\color{lightgreen}-21.088 \mathrm{\%}}\right) $$	Flame Graph
linked_queries	query-unlimited	$$578 \mathrm{ms} \pm 1.30 \mathrm{ms}\left({\color{gray}-0.764 \mathrm{\%}}\right) $$	Flame Graph