Conversation
There was a problem hiding this comment.
Pull request overview
This PR adds a new @microsoft/durabletask-js-export-history package that enables exporting orchestration instance history to Azure Blob Storage. This is a JavaScript/TypeScript port of the Export History feature from durabletask-dotnet, providing parity with the .NET SDK.
Changes:
- New package with entity-based job management (ExportJob entity tracks lifecycle: Pending → Active → Completed/Failed)
- Export orchestrators with batch processing, retry logic (exponential backoff), and checkpoint support
- Activities for listing terminal instances and exporting history to Azure Blob Storage (JSONL.GZ or JSON formats)
- Client API (ExportHistoryClient, ExportHistoryJobClient) for creating, querying, and managing export jobs
- Comprehensive test coverage: 54 unit tests and 18 E2E tests (gated behind EXPORT_HISTORY_E2E=1)
Reviewed changes
Copilot reviewed 37 out of 39 changed files in this pull request and generated 9 comments.
Show a summary per file
| File | Description |
|---|---|
| packages/durabletask-js-export-history/package.json | Package configuration with dependencies, version 0.1.0-beta.1, Node.js >=22.0.0 |
| packages/durabletask-js-export-history/tsconfig.json | TypeScript configuration extending base config |
| packages/durabletask-js-export-history/tsconfig.build.json | Build-specific TypeScript configuration with path mapping |
| packages/durabletask-js-export-history/jest.config.js | Jest configuration with module name mapping |
| packages/durabletask-js-export-history/src/index.ts | Main package exports for models, entity, orchestrators, activities, client, builders, constants, and errors |
| packages/durabletask-js-export-history/src/constants.ts | Named constants for orchestrators, activities, entities, and instance ID helpers |
| packages/durabletask-js-export-history/src/errors.ts | Custom error classes for validation, state transitions, and not found scenarios |
| packages/durabletask-js-export-history/src/models/*.ts | Type definitions for export configuration, job state, formats, filters, checkpoints, and validation logic |
| packages/durabletask-js-export-history/src/entity/export-job.ts | Durable entity managing job lifecycle, state transitions, and orchestrator scheduling |
| packages/durabletask-js-export-history/src/orchestrators/export-job-orchestrator.ts | Main orchestrator with batch processing, retry logic, checkpoint commits, and continue-as-new support |
| packages/durabletask-js-export-history/src/orchestrators/execute-export-job-operation-orchestrator.ts | Wrapper orchestrator for transactional entity operations |
| packages/durabletask-js-export-history/src/activities/list-terminal-instances-activity.ts | Activity for listing terminal orchestration instances with pagination |
| packages/durabletask-js-export-history/src/activities/export-instance-history-activity.ts | Activity for fetching, serializing, compressing, and uploading instance history to blob storage |
| packages/durabletask-js-export-history/src/client/export-history-client.ts | Client API for creating, querying, and managing export jobs via entity interactions |
| packages/durabletask-js-export-history/src/builders/use-export-history.ts | Helper functions to register export history components on workers and create clients |
| packages/durabletask-js-export-history/test/*.spec.ts | Unit tests for models, entity, errors, constants, builders, and activities |
| test/e2e-azuremanaged/export-history.spec.ts | E2E tests validating full export workflow against real DTS scheduler with blob verification |
| jest.config.js | Root Jest config updated with module name mapping for new package |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
packages/durabletask-js-export-history/src/client/export-history-client.ts
Outdated
Show resolved
Hide resolved
packages/durabletask-js-export-history/src/models/export-job-creation-options.ts
Outdated
Show resolved
Hide resolved
packages/durabletask-js-export-history/src/client/export-history-client.ts
Outdated
Show resolved
Hide resolved
packages/durabletask-js-export-history/src/entity/export-job.ts
Outdated
Show resolved
Hide resolved
packages/durabletask-js-export-history/src/entity/export-job.ts
Outdated
Show resolved
Hide resolved
packages/durabletask-js-export-history/src/models/export-job-creation-options.ts
Outdated
Show resolved
Hide resolved
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
…etask-js into wangbill/export
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
…reation-options.ts Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
…ry-client.ts Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
| * JS toISOString: "2026-02-20T15:30:00.000Z" | ||
| * This ensures blob filename hashes match between JS and .NET implementations. | ||
| */ | ||
| function formatDateRoundTrip(date: Date): string { |
There was a problem hiding this comment.
The function looks fine, but I'm curious if this is the only way to do this? I would assume .NET complies to a specific standard and I would assume that that is also built into JS. If .NET is weird and using a datestring that JS doesn't agree with, I'm fine with it, but if they have a built in, we should use it.
| if (!input) { | ||
| throw new Error("input is required"); | ||
| } | ||
| if (!input.instanceId) { | ||
| throw new Error("instanceId is required"); | ||
| } | ||
| if (!input.destination) { | ||
| throw new Error("destination is required"); | ||
| } | ||
| if (!input.format) { | ||
| throw new Error("format is required"); | ||
| } |
There was a problem hiding this comment.
Just my own curiosity, do we not do tighter typed errors in JS? Like an IllegalArgumentException in .NET?
| * @param client The TaskHubGrpcClient to use for listing instances. | ||
| * @returns An activity function that lists terminal instance IDs. | ||
| */ | ||
| export function createListTerminalInstancesActivity(client: TaskHubGrpcClient) { |
There was a problem hiding this comment.
Since this is named explicitly to terminal instances, should we validate that the runtime state in the request is actually terminal? If we're doing that in a different part of the orchestration, that's fine, but we may want to rename this to just be about listing instances.
| // The entity's Create operation signals Run, which schedules the export orchestrator. | ||
| // No need to schedule the orchestrator from the client. |
There was a problem hiding this comment.
Why the comments at the end here?
| /** | ||
| * The export job is actively running. | ||
| */ | ||
| Active = "Active", |
| * an EntityInstanceId object to ensure it survives JSON round-tripping through | ||
| * orchestration input serialization. | ||
| */ | ||
| export interface ExportJobOperationRequest { |
There was a problem hiding this comment.
Nit: Since we have a model directory, any reason why we didn't include this (and a few other request objects)?
| jobEntityId: jobEntityId.toString(), | ||
| processedCycles: 0, | ||
| }; | ||
| ctx.continueAsNew(newRequest, false); |
There was a problem hiding this comment.
Can you explain how continue as new picks up where the previous one left off? It's because the info is stored in the JobState right?
| @@ -0,0 +1,50 @@ | |||
| // Copyright (c) Microsoft Corporation. All rights reserved. | |||
There was a problem hiding this comment.
I would love an illegal argument style error. We have a lot of input validation that just throws a generic error.
| "files": [ | ||
| "dist", | ||
| "LICENSE" | ||
| ], |
There was a problem hiding this comment.
Do these represent real files? Cause I don't see any of that name, or does it exist elsewhere in the repo (outside of this PR)?
Summary
What changed?
Add a new
@microsoft/durabletask-js-export-historypackage that enables exporting orchestration instance history to Azure Blob Storage. This is a JavaScript/TypeScript port of the Export History feature fromdurabletask-dotnet.New package:
packages/durabletask-js-export-history/export-job.ts): Durable entity that tracks export job lifecycle (Pending → Active → Completed/Failed) with checkpoint support and state transitionsexport-job-orchestrator.ts: Main orchestrator that lists terminal instances, exports their history in batches with retry logic (exponential backoff), commits checkpoints, and marks completion/failureexecute-export-job-operation-orchestrator.ts: Wrapper orchestrator that creates the entity and starts the export jobexport-instance-history-activity.ts: Fetches instance history via gRPC, serializes to JSONL/JSON format, compresses with gzip, and uploads to Azure Blob Storagelist-terminal-instances-activity.ts: Lists orchestration instances in terminal states within a time windowexport-history-client.ts):ExportHistoryClientandExportHistoryJobClientclasses for creating, querying, deleting, and describing export jobsuse-export-history.ts):useExportHistoryWorker()helper to register all export history orchestrators, activities, and entities on a workerExportJobClientValidationError,ExportJobInvalidTransitionError)Test coverage:
test/e2e-azuremanaged/export-history.spec.ts) covering:EXPORT_HISTORY_E2E=1env var (skipped in CI — DTS emulator lacks entity support)Other changes:
jest.config.js: Added module name mapping for@microsoft/durabletask-js-export-historyCHANGELOG.md: Removed duplicate v0.2.0 entries (branch was based on pre-v0.2.0)durabletask-jsanddurabletask-js-azuremanagedpackage versions set to0.1.0-beta.1Why is this change needed?
Enables users to export durable orchestration instance history to Azure Blob Storage for auditing, analytics, and long-term retention. This fills a gap in the JS SDK where the equivalent feature already exists in the .NET SDK (
durabletask-dotnet).Issues / work items
durabletask-dotnetProject checklist
CHANGELOG.mdAI-assisted code disclosure (required)
Was an AI tool used? (select one)
If AI was used:
packages/durabletask-js-export-history/,test/e2e-azuremanaged/export-history.spec.ts,jest.config.jsAI verification (required if AI was used):
Testing
Automated tests
EXPORT_HISTORY_E2E=1+ real DTS scheduler + Azurite)Manual validation (only if runtime/behavior changed)
Notes for reviewers
EXPORT_HISTORY_E2E=1and will auto-skip in CI.createTimercalls in the JS orchestrator passnew Date()objects (not raw numbers) because the JS SDK interprets numeric arguments as seconds-delay and can produce invalid timestamps.RetryPolicyoncallActivitydue to a DTS server-side Timestamp validation bug, so retry is handled at the batch level in the orchestrator.0.1.0-beta.1to match the branch base. The CHANGELOG removal of v0.2.0 entries should be reverted if this PR targets post-v0.2.0.