Temporal: Braintrust Temporal for ESM projects and fix merging of WorkflowClientInterceptors#1348
Temporal: Braintrust Temporal for ESM projects and fix merging of WorkflowClientInterceptors#1348
Conversation
| // 2. wrapAISDKProvider adds LLM tracing (prompts, tokens, completions) | ||
| // 3. BraintrustTemporalPlugin traces Temporal workflows and activities | ||
| plugins: [ | ||
| new AiSdkPlugin({ |
There was a problem hiding this comment.
@ibolmo (and @clutchski since you wrote the python implementation and my know more about temporal in general)
I plan to split this example into two different examples to illustrate the issues I have been BraintrustTemporalPlugin and AISDKPlugin working together.
The two different workflows I have found are:
- The BraintrustTemporalPlugin works with just wrapAISDK (no AISDKPlugin) if you create activities in temporal for llm calls.
- I found one way of making the AISDKPlugin work with the BraintrustTemporalPlugin (make sure our plugin is registered first) and only wrap the underlying model provider.
// this will wrap just the doGenerate call and map it with out temporal flows
const tracedProvider = wrapAISDKProvider(openai as any);
const worker = await Worker.create({
connection,
namespace: "default",
taskQueue: TASK_QUEUE,
// Load the workflows from this example's `src` so the exported
// workflow functions (`haikuAgent`, `haikuAgentTraced`, etc.) are available.
workflowsPath: new URL("./workflows.ts", import.meta.url).pathname,
activities: { getWeather },
plugins: [
new AiSdkPlugin({ modelProvider: tracedProvider }),
new BraintrustTemporalPlugin(),
],
The advantage of the temporal AISDKPlugin is that the plugin can work without creating explicit activities for the llm calls but it does that by a low level wrapping which means the trace looks less detailed if I just wrap the model.
What you get with creating activities and using wrapAISDK (which does not support the temporal aisdk plugin type)

What you get with wrapping just the Model Provider

What the user was getting when trying to currently use our workflow integration with just the AISDKPlugin

I am still looking into what might work to wrap the entire plugin but haven't figured it out yet and wondering if I should push out something that sort of works with the model provider wrapping or if that is not that useful.
| import { createBraintrustSinks } from "./sinks"; | ||
|
|
||
| // Add the workflow interceptor package specifier so the Temporal bundler can include it | ||
| const WORKFLOW_INTERCEPTORS_SPEC = "@braintrust/temporal/workflow-interceptors"; |
There was a problem hiding this comment.
This is the important change for ESM to work, Temporal is able to bundle the package by name.
| workflow = { | ||
| ...existing, | ||
| ...braintrustInterceptor, | ||
| calls: [...(existing.calls ?? []), () => braintrustInterceptor], |
There was a problem hiding this comment.
This was leading to our plugin being dropped when registered with another plugin (ie. AISDKPlugin).
| * Uses a generic type constraint instead of specific LanguageModel version | ||
| * to ensure compatibility with both current and future AI SDK versions. | ||
| */ | ||
| function wrapModel< |
There was a problem hiding this comment.
This is temp until I decide what I wish to do for the temporal integration
- Drop attempting to wrap the temporal plugin and just merge the fix for making our plugin work
- Add this simple wrapping on the model
- Continue the quest to figure out how to integrate with the plugin
This PR adds an improvement to our temporal integration to work in esm environments and fixes an issue with the use of the older WorkflowClientInterceptors.
Additional examples have been added: