- // assemble request config
- ctx = callbacks.OnStart(ctx, &fmodel.CallbackInput{ Messages: in, Tools: append(cm.rawTools), ToolChoice: nil, Config: reqConf })
+### Component Implementation Internal Triggering (Component Callback)
- // invoke provider API and read response
+In the component implementation code, call `OnStart(), OnEnd(), OnError(), OnStartWithStreamInput(), OnEndWithStreamOutput()` from the callbacks package. Taking Ark's ChatModel implementation as an example, in the Generate method:
+
+```go
+func (cm *ChatModel) Generate(ctx context.Context, in []*schema.Message, opts ...fmodel.Option) (
+ outMsg *schema.Message, err error) {
+
+ defer func() {
+ if err != nil {
+ _ = callbacks.OnError(ctx, err)
+ }
+ }()
+
+ // omit multiple lines... instantiate req conf
+
+ ctx = callbacks.OnStart(ctx, &fmodel.CallbackInput{
+ Messages: in,
+ Tools: append(cm.rawTools), // join tool info from call options
+ ToolChoice: nil, // not support in api
+ Config: reqConf,
+ })
+
+ // omit multiple lines... invoke Ark chat API and get the response
+
+ _ = callbacks.OnEnd(ctx, &fmodel.CallbackOutput{
+ Message: outMsg,
+ Config: reqConf,
+ TokenUsage: toModelCallbackUsage(outMsg.ResponseMeta),
+ })
- _ = callbacks.OnEnd(ctx, &fmodel.CallbackOutput{ Message: outMsg, Config: reqConf, TokenUsage: toModelCallbackUsage(outMsg.ResponseMeta) })
return outMsg, nil
}
+```
-func (cm *ChatModel) Stream(ctx context.Context, in []*schema.Message, opts ...fmodel.Option) (outStream *schema.StreamReader[*schema.Message], err error) {
- defer func() { if err != nil { _ = callbacks.OnError(ctx, err) } }()
+In the Stream method:
- // assemble request config
- ctx = callbacks.OnStart(ctx, &fmodel.CallbackInput{ Messages: in, Tools: append(cm.rawTools), ToolChoice: nil, Config: reqConf })
+```go
+func (cm *ChatModel) Stream(ctx context.Context, in []*schema.Message, opts ...fmodel.Option) ( // byted_s_too_many_lines_in_func
+ outStream *schema.StreamReader[*schema.Message], err error) {
+
+ defer func() {
+ if err != nil {
+ _ = callbacks.OnError(ctx, err)
+ }
+ }()
+
+ // omit multiple lines... instantiate req conf
+
+ ctx = callbacks.OnStart(ctx, &fmodel.CallbackInput{
+ Messages: in,
+ Tools: append(cm.rawTools), // join tool info from call options
+ ToolChoice: nil, // not support in api
+ Config: reqConf,
+ })
+
+ // omit multiple lines... make request to Ark API and convert response stream to StreamReader[model.*CallbackOutput]
- // invoke provider API and convert response to StreamReader[model.CallbackOutput]
_, sr = callbacks.OnEndWithStreamOutput(ctx, sr)
- return schema.StreamReaderWithConvert(sr, func(src *fmodel.CallbackOutput) (*schema.Message, error) {
- if src.Message == nil { return nil, schema.ErrNoValue }
- return src.Message, nil
- }), nil
+ return schema.StreamReaderWithConvert(sr,
+ func(src *fmodel.CallbackOutput) (*schema.Message, error) {
+ if src.Message == nil {
+ return nil, schema.ErrNoValue
+ }
+
+ return src.Message, nil
+ },
+ ), nil
}
```
-### Graph/Node-level Callbacks
+You can see that Generate call triggers OnEnd, while Stream call triggers OnEndWithStreamOutput:
-When a Component is orchestrated into a Graph Node, and the Component does not implement callbacks, the Node injects callback trigger points matching the Component’s streaming paradigm. For example, a ChatModelNode triggers OnStart/OnEnd around `Generate`, and OnStart/OnEndWithStreamOutput around `Stream`. Which timing is triggered depends on both Graph’s execution mode (Invoke/Stream/Collect/Transform) and the Component’s streaming support.
+When triggering Callbacks inside component implementations:
-See [Streaming Essentials](/docs/eino/core_modules/chain_and_graph_orchestration/stream_programming_essentials).
+- **When component input is StreamReader, trigger OnStartWithStreamInput, otherwise trigger OnStart**
+- **When component output is StreamReader, trigger OnEndWithStreamOutput, otherwise trigger OnEnd**
-### Graph-level Callbacks
+Components that implement callback triggering internally should implement the Checker interface, with IsCallbacksEnabled returning true, to communicate "I have implemented callback triggering internally" to the outside:
-Graph triggers callbacks at its own start/end/error timings. If Graph is called via `Invoke`, it triggers `OnStart/OnEnd/OnError`. If called via `Stream/Collect/Transform`, it triggers `OnStartWithStreamInput/OnEndWithStreamOutput/OnError` because Graph internally always executes as `Invoke` or `Transform`. See [Streaming Essentials](/docs/eino/core_modules/chain_and_graph_orchestration/stream_programming_essentials).
+```go
+// Checker tells callback aspect status of component's implementation
+// When the Checker interface is implemented and returns true, the framework will not start the default aspect.
+// Instead, the component will decide the callback execution location and the information to be injected.
+type Checker interface {
+ IsCallbacksEnabled() bool
+}
+```
+
+If a component implementation doesn't implement the Checker interface, or IsCallbacksEnabled returns false, it can be assumed that the component doesn't trigger callbacks internally, and Graph Node needs to be responsible for injection and triggering (when used within a Graph).
+
+### Graph Node Triggering (Node Callback)
+
+When a Component is orchestrated into a Graph, it becomes a Node. At this point, if the Component itself triggers callbacks, the Node reuses the Component's callback handling. Otherwise, the Node wraps callback handler trigger points around the Component. These points correspond to the Component's streaming paradigm. For example, a ChatModelNode wraps OnStart/OnEnd/OnError around the Generate method, and OnStart/OnEndWithStreamOutput/OnError around the Stream method.
+
+At Graph runtime, components run in Invoke or Transform paradigm, and based on the component's specific business streaming paradigm, call the corresponding component methods. For example, when Graph runs with Invoke, Chat Model Node runs with Invoke, calling the Generate method. When Graph runs with Stream, Chat Model Node runs with Transform, but since Chat Model's business streaming paradigm doesn't have Transform, it automatically falls back to calling the Stream method. Therefore:
+
+**Which timing point (OnStart vs OnStartWithStreamInput) a Graph Node specifically triggers depends on two factors: the component implementation's business streaming paradigm and the Graph's execution mode.**
+
+For detailed introduction to Eino streaming programming, see [Eino Streaming Programming Essentials](/docs/eino/core_modules/chain_and_graph_orchestration/stream_programming_essentials)
+
+### Graph Self-Triggering (Graph Callback)
-Note: Graph is also a component. Therefore, a graph callback is a special form of component callback. Per Node Callback semantics, when a node’s internal component (including a nested graph added via `AddGraphNode`) implements callback timings itself, the node reuses the component’s behavior and does not add duplicate node-level callbacks.
+Graph triggers Callback Handlers at its own start, end, and error timings. If Graph is called in Invoke form, it triggers OnStart/OnEnd/OnError. If called in Stream/Collect/Transform form, it triggers OnStartWithStreamInput/OnEndWithStreamOutput/OnError. This is because **Graph internally always executes as Invoke or Transform**. See [Eino Streaming Programming Essentials](/docs/eino/core_modules/chain_and_graph_orchestration/stream_programming_essentials)
+
+It's worth noting: graph is also a type of component, so graph callback is also a special form of component callback. According to the Node Callback definition, when the component inside a Node implements awareness and handling of triggering timings, the Node directly reuses the Component's implementation and won't implement Node Callback. This means when a graph is added to another Graph as a Node via AddGraphNode, this Node reuses the internal graph's graph callback.
## Parsing Callback Input & Output
-Underlying types are `any`, while specific components may pass their own typed inputs/outputs. Handler method parameters are `any` as well, so convert as needed.
+From the above, we know that Callback Input & Output's underlying type is Any, but different component types may pass their own specific types when actually triggering callbacks. And in the Callback Handler interface definition, the parameters of each method are also Any-typed Callback Input & Output.
+
+Therefore, specific Handler implementations need to do two things:
+
+1. Determine which component type is currently triggering the callback based on RunInfo, such as RunInfo.Component == "ChatModel", or RunInfo.Type == "xxx Chat Model".
+2. Convert the any-typed Callback Input & Output to the corresponding specific type, taking RunInfo.Component == "ChatModel" as an example:
```go
// ConvCallbackInput converts the callback input to the model callback input.
func ConvCallbackInput(src callbacks.CallbackInput) *CallbackInput {
switch t := src.(type) {
- case *CallbackInput: // component implementation already passed typed *model.CallbackInput
+ case *CallbackInput: // when callback is triggered within component implementation, the input is usually already a typed *model.CallbackInput
return t
- case []*schema.Message: // graph node injected callback passes ChatModel interface input: []*schema.Message
- return &CallbackInput{ Messages: t }
+ case []*schema.Message: // when callback is injected by graph node, not the component implementation itself, the input is the input of Chat Model interface, which is []*schema.Message
+ return &CallbackInput{
+ Messages: t,
+ }
default:
return nil
}
@@ -217,31 +322,39 @@ func ConvCallbackInput(src callbacks.CallbackInput) *CallbackInput {
// ConvCallbackOutput converts the callback output to the model callback output.
func ConvCallbackOutput(src callbacks.CallbackOutput) *CallbackOutput {
switch t := src.(type) {
- case *CallbackOutput: // component implementation already passed typed *model.CallbackOutput
+ case *CallbackOutput: // when callback is triggered within component implementation, the output is usually already a typed *model.CallbackOutput
return t
- case *schema.Message: // graph node injected callback passes ChatModel interface output: *schema.Message
- return &CallbackOutput{ Message: t }
+ case *schema.Message: // when callback is injected by graph node, not the component implementation itself, the output is the output of Chat Model interface, which is *schema.Message
+ return &CallbackOutput{
+ Message: t,
+ }
default:
return nil
}
}
```
-To reduce boilerplate, prefer helpers/builders when focusing on specific components or timings.
+If the Handler needs to add switch cases to determine RunInfo.Component, and for each case, call the corresponding conversion function to convert Any to a specific type, it's indeed somewhat complex. To reduce the repetitive work of writing glue code, we provide two convenient tool functions for implementing Handlers.
-## Handler Implementation
+## Handler Implementation Methods
+
+Besides directly implementing the Handler interface, Eino provides two convenient Handler implementation tools.
### HandlerHelper
-When a handler only targets specific component types (e.g., in ReAct scenarios focusing on ChatModel and Tool), use `HandlerHelper` to quickly create typed handlers:
+If the user's Handler only focuses on specific component types, such as in ReActAgent scenarios only focusing on ChatModel and Tool, it's recommended to use HandlerHelper to quickly create typed Callback Handlers:
```go
-handler := NewHandlerHelper().ChatModel(modelHandler).Tool(toolHandler).Handler()
+import ucb "github.com/cloudwego/eino/utils/callbacks"
+
+handler := ucb.NewHandlerHelper().ChatModel(modelHandler).Tool(toolHandler).Handler()
```
-The `modelHandler` can use a typed helper for ChatModel callbacks:
+Where modelHandler is Chat Model component's further encapsulation of callback handler:
```go
+// from package utils/callbacks
+
// ModelCallbackHandler is the handler for the model callback.
type ModelCallbackHandler struct {
OnStart func(ctx context.Context, runInfo *callbacks.RunInfo, input *model.CallbackInput) context.Context
@@ -251,37 +364,39 @@ type ModelCallbackHandler struct {
}
```
-This encapsulation provides:
+The above ModelCallbackHandler encapsulates three operations:
+
+1. No longer need to determine RunInfo.Component to select callbacks triggered by ChatModel, as automatic filtering is already done.
+2. Only requires implementing the triggering timings supported by Chat Model component, here removing the unsupported OnStartWithStreamInput. Also, if the user only cares about some of the four timings supported by Chat Model, such as only OnStart, they can implement only OnStart.
+3. Input / Output are no longer Any types, but already converted model.CallbackInput, model.CallbackOutput.
-- Automatic filtering by component type (no need to switch on `RunInfo.Component`)
-- Only the timings supported by ChatModel (drops `OnStartWithStreamInput`); implement any subset
-- Typed `Input/Output` (`model.CallbackInput`, `model.CallbackOutput`) instead of `any`
+HandlerHelper supports all official components, the current list is: ChatModel, ChatTemplate, Retriever, Indexer, Embedding, Document.Loader, Document.Transformer, Tool, ToolsNode.
-`HandlerHelper` supports official components: ChatModel, ChatTemplate, Retriever, Indexer, Embedding, Document.Loader, Document.Transformer, Tool, ToolsNode. For Lambda/Graph/Chain, it filters by type but you still implement generic `callbacks.Handler` for timings and conversions:
+For Lambda, Graph, Chain and other "components" with uncertain input/output types, HandlerHelper can also be used, but can only achieve point 1 above, i.e., automatic filtering by component type, points 2 and 3 still need to be implemented by the user:
```go
-handler := NewHandlerHelper().Lambda(callbacks.Handler).Graph(callbacks.Handler).Handler()
+import ucb "github.com/cloudwego/eino/utils/callbacks"
+
+handler := ucb.NewHandlerHelper().Lambda(callbacks.Handler).Graph(callbacks.Handler)...Handler()
```
+At this point, NewHandlerHelper().Lambda() needs to pass in callbacks.Handler which can be implemented using the HandlerBuilder below.
+
### HandlerBuilder
-If a handler needs to target multiple component types but only a subset of timings, use `HandlerBuilder`:
+If the user's Handler needs to focus on multiple component types, but only needs to focus on some triggering timings, HandlerBuilder can be used:
```go
-handler := NewHandlerBuilder().OnStartFn(fn).Build()
-```
-
-## Usage Notes
+import "github.com/cloudwego/eino/callbacks"
-- Prefer typed inputs/outputs for provider-specific handlers
-- Use global handlers for common concerns; node-specific handlers for fine-grained control
-- Remember handler order is unspecified; design idempotent handlers
+handler := callbacks.NewHandlerBuilder().OnStartFn(fn)...Build()
+```
-### Best Practices
+## Best Practices
-#### In Graph
+### Using in Graph
-- Actively use Global Handlers for always-on concerns.
+- Actively use Global Handlers to register always-effective Handlers.
```go
package main
@@ -322,7 +437,7 @@ func main() {
}
```
-- Inject per-run handlers with `WithCallbacks` and target nodes via `DesignateNode` or by path.
+- Inject Handlers at runtime through WithHandlers option, and specify effective Nodes / nested internal Graphs / internal Graph Nodes through DesignateNode or DesignateNodeByPath.
```go
package main
@@ -365,13 +480,13 @@ func main() {
}
```
-
+### Using Outside Graph
-#### Outside Graph
+This scenario is: not using Graph/Chain/Workflow orchestration capabilities, but directly calling ChatModel/Tool/Lambda and other components with code, and hoping these components can successfully trigger Callback Handlers.
-This scenario: you do not use Graph/Chain/Workflow orchestration, but you directly call components like ChatModel/Tool/Lambda and still want callbacks to trigger.
+The problem users need to solve in this scenario is: manually setting correct RunInfo and Handlers, because there's no Graph to help users automatically set RunInfo and Handlers.
-You must manually set correct `RunInfo` and Handlers because there is no Graph to do it for you.
+Complete example:
```go
package main
@@ -384,7 +499,7 @@ import (
)
func innerLambda(ctx context.Context, input string) (string, error) {
- // As provider of ComponentB: ensure default RunInfo when entering the component (Name cannot default)
+ // As ComponentB's implementer: add default RunInfo when entering the component (Name cannot be given a default value)
ctx = callbacks.EnsureRunInfo(ctx, "Lambda", compose.ComponentOfLambda)
ctx = callbacks.OnStart(ctx, input)
out := "inner:" + input
@@ -393,17 +508,17 @@ func innerLambda(ctx context.Context, input string) (string, error) {
}
func outerLambda(ctx context.Context, input string) (string, error) {
- // As provider of ComponentA: ensure default RunInfo when entering
+ // As ComponentA's implementer: add default RunInfo when entering the component
ctx = callbacks.EnsureRunInfo(ctx, "Lambda", compose.ComponentOfLambda)
ctx = callbacks.OnStart(ctx, input)
- // Recommended: replace RunInfo before calling inner component, ensuring correct name/type/component
+ // Recommended: replace RunInfo before calling, ensuring inner component gets correct name/type/component
ctxInner := callbacks.ReuseHandlers(ctx,
&callbacks.RunInfo{Name: "ComponentB", Type: "Lambda", Component: compose.ComponentOfLambda},
)
out1, _ := innerLambda(ctxInner, input) // inner RunInfo.Name = "ComponentB"
- // Without replacement: framework clears RunInfo after a complete callback cycle; EnsureRunInfo adds defaults (Name empty)
+ // Without replacement: framework clears RunInfo, can only rely on EnsureRunInfo to add default values (Name is empty)
out2, _ := innerLambda(ctx, input) // inner RunInfo.Name == ""
final := out1 + "|" + out2
@@ -412,7 +527,7 @@ func outerLambda(ctx context.Context, input string) (string, error) {
}
func main() {
- // Standalone components outside graph: initialize RunInfo and Handlers
+ // Using components standalone outside graph: initialize RunInfo and Handlers
h := callbacks.NewHandlerBuilder().Build()
ctx := callbacks.InitCallbacks(context.Background(),
&callbacks.RunInfo{Name: "ComponentA", Type: "Lambda", Component: compose.ComponentOfLambda},
@@ -423,19 +538,21 @@ func main() {
}
```
-Notes:
+Explanation of the above sample code:
+
+- Initialization: When using components outside graph/chain, use InitCallbacks to set the first RunInfo and Handlers, so subsequent component execution can get the complete callback context.
+- Internal calls: Before component A calls component B internally, use ReuseHandlers to replace RunInfo (keeping original handlers), ensuring B's callbacks get correct Type/Component/Name.
+- Consequences of not replacing: After a complete set of Callbacks is triggered, Eino clears the RunInfo in the current ctx. At this point, because RunInfo is empty, Eino won't trigger Callbacks anymore; Component B's developer can only use EnsureRunInfo in their own implementation to add default values for Type/Component, to ensure RunInfo is non-empty and roughly correct, thus successfully triggering Callbacks. But cannot give a reasonable Name, so RunInfo.Name will be an empty string.
-- Initialization: use `InitCallbacks` to set the first `RunInfo` and Handlers when using components outside graph/chain so subsequent components receive the full callback context.
-- Internal calls: before Component A calls Component B, use `ReuseHandlers` to replace `RunInfo` (keeping existing handlers) so B receives correct `Type/Component/Name`.
-- Without replacement: after a complete set of callbacks, Eino clears `RunInfo` from the current context; providers can call `EnsureRunInfo` to supply default `Type/Component` to keep callbacks working, but `Name` cannot be inferred and will be empty.
+### Component Nesting Usage
-#### Component Nesting
+Scenario: Inside one component, such as Lambda, manually call another component, such as ChatModel.
-Scenario: inside a component (e.g., a Lambda), manually call another component (e.g., ChatModel).
+At this point, if the outer component's ctx has callback handlers, because this ctx is also passed to the inner component, the inner component will also receive the same callback handlers.
-If the outer component’s context has handlers, the inner component receives the same handlers. To control whether the inner component triggers callbacks:
+Distinguishing by "whether you want the inner component to trigger callbacks":
-1) Want callbacks triggered: set `RunInfo` for the inner component using `ReuseHandlers`.
+1. Want to trigger: Basically equivalent to the situation in the previous section, it's recommended to manually set `RunInfo` for the inner component through `ReuseHandlers`.
```go
package main
@@ -450,20 +567,20 @@ import (
"github.com/cloudwego/eino/schema"
)
-// Outer lambda calls ChatModel inside
+// Outer Lambda, manually calls ChatModel inside
func OuterLambdaCallsChatModel(cm model.BaseChatModel) *compose.Lambda {
return compose.InvokableLambda(func(ctx context.Context, input string) (string, error) {
- // 1) Reuse outer handlers and set RunInfo explicitly for the inner component
+ // 1) Reuse outer handlers and explicitly set RunInfo for inner component
innerCtx := callbacks.ReuseHandlers(ctx, &callbacks.RunInfo{
- Type: "InnerCM",
- Component: components.ComponentOfChatModel,
- Name: "inner-chat-model",
+ Type: "InnerCM", // Customizable
+ Component: components.ComponentOfChatModel, // Mark component type
+ Name: "inner-chat-model", // Customizable
})
// 2) Build input messages
msgs := []*schema.Message{{Role: schema.User, Content: input}}
- // 3) Call ChatModel (inner implementation triggers its callbacks)
+ // 3) Call ChatModel (internally triggers corresponding callbacks)
out, err := cm.Generate(innerCtx, msgs)
if err != nil {
return "", err
@@ -473,12 +590,12 @@ func OuterLambdaCallsChatModel(cm model.BaseChatModel) *compose.Lambda {
}
```
-If the inner ChatModel’s `Generate` does not trigger callbacks, the outer component should trigger them around the inner call:
+The above code assumes "the inner ChatModel's Generate method internally has already called OnStart, OnEnd, OnError methods". If not, you need to call these methods "on behalf of the inner component" inside the outer component:
```go
func OuterLambdaCallsChatModel(cm model.BaseChatModel) *compose.Lambda {
return compose.InvokableLambda(func(ctx context.Context, input string) (string, error) {
- // Reuse outer handlers and set RunInfo explicitly for inner component
+ // Reuse outer handlers and explicitly set RunInfo for inner component
ctx = callbacks.ReuseHandlers(ctx, &callbacks.RunInfo{
Type: "InnerCM",
Component: components.ComponentOfChatModel,
@@ -507,35 +624,35 @@ func OuterLambdaCallsChatModel(cm model.BaseChatModel) *compose.Lambda {
}
```
-2) Do not want inner callbacks: assume the inner component implements `IsCallbacksEnabled()` returning true and calls `EnsureRunInfo`. By default, inner callbacks will trigger. To disable, pass a new context without handlers to the inner component:
+1. Don't want to trigger: This assumes the inner component implements `IsCallbacksEnabled()` and returns true, and internally calls `EnsureRunInfo`. At this point, inner callbacks will trigger by default. If you don't want them to trigger, the simplest way is to remove handlers from ctx, such as passing a new ctx to the inner component:
-```go
-package main
+ ```go
+ package main
-import (
- "context"
+ import (
+ "context"
- "github.com/cloudwego/eino/components/model"
- "github.com/cloudwego/eino/compose"
- "github.com/cloudwego/eino/schema"
-)
+ "github.com/cloudwego/eino/components/model"
+ "github.com/cloudwego/eino/compose"
+ "github.com/cloudwego/eino/schema"
+ )
-func OuterLambdaNoCallbacks(cm model.BaseChatModel) *compose.Lambda {
- return compose.InvokableLambda(func(ctx context.Context, input string) (string, error) {
- // Use a brand-new context; do not reuse outer handlers
- innerCtx := context.Background()
+ func OuterLambdaNoCallbacks(cm model.BaseChatModel) *compose.Lambda {
+ return compose.InvokableLambda(func(ctx context.Context, input string) (string, error) {
+ // Use a brand new ctx, don't reuse outer handlers
+ innerCtx := context.Background()
- msgs := []*schema.Message{{Role: schema.User, Content: input}}
- out, err := cm.Generate(innerCtx, msgs)
- if err != nil {
- return "", err
- }
- return out.Content, nil
- })
-}
-```
+ msgs := []*schema.Message{{Role: schema.User, Content: input}}
+ out, err := cm.Generate(innerCtx, msgs)
+ if err != nil {
+ return "", err
+ }
+ return out.Content, nil
+ })
+ }
+ ```
-Sometimes you may want to disable a specific handler for inner components but keep others. Implement filtering by `RunInfo` inside that handler:
+ 1. But sometimes users may want to "only not trigger a specific callback handler, but still trigger other callback handlers". The recommended approach is to add code in this callback handler to filter out inner components by RunInfo:
```go
package main
@@ -549,13 +666,14 @@ import (
"github.com/cloudwego/eino/compose"
)
-// A selective handler: no-ops for the inner ChatModel (Type=InnerCM, Name=inner-chat-model)
+// A handler that filters by RunInfo: does nothing for inner ChatModel (Type=InnerCM, Name=inner-chat-model)
func newSelectiveHandler() callbacks.Handler {
return callbacks.
NewHandlerBuilder().
OnStartFn(func(ctx context.Context, info *callbacks.RunInfo, input callbacks.CallbackInput) context.Context {
if info != nil && info.Component == components.ComponentOfChatModel &&
info.Type == "InnerCM" && info.Name == "inner-chat-model" {
+ // Filter target: inner ChatModel, return directly without processing
return ctx
}
log.Printf("[OnStart] %s/%s (%s)", info.Type, info.Name, info.Component)
@@ -564,6 +682,7 @@ func newSelectiveHandler() callbacks.Handler {
OnEndFn(func(ctx context.Context, info *callbacks.RunInfo, output callbacks.CallbackOutput) context.Context {
if info != nil && info.Component == components.ComponentOfChatModel &&
info.Type == "InnerCM" && info.Name == "inner-chat-model" {
+ // Filter target: inner ChatModel, return directly without processing
return ctx
}
log.Printf("[OnEnd] %s/%s (%s)", info.Type, info.Name, info.Component)
@@ -572,32 +691,48 @@ func newSelectiveHandler() callbacks.Handler {
Build()
}
-// Composition example: outer call triggers; selective handler filters out inner ChatModel
+// Composition example: outer call wants to trigger, specific handler filters out inner ChatModel through RunInfo
func Example(cm model.BaseChatModel) (compose.Runnable[string, string], error) {
handler := newSelectiveHandler()
chain := compose.NewChain[string, string]().
- AppendLambda(OuterLambdaCallsChatModel(cm))
+ AppendLambda(OuterLambdaCallsChatModel(cm)) // Internally will ReuseHandlers + RunInfo
return chain.Compile(
context.Background(),
+ // Mount handler (can also combine with global handlers)
compose.WithCallbacks(handler),
)
}
```
-### Read/Write Input & Output Carefully
+### Reading and Writing Input & Output in Handler
-Inputs/outputs flow by direct assignment; pointers/maps refer to the same data across nodes and handlers. Avoid mutations inside nodes and handlers to prevent race conditions.
+When input & output flow through the graph, they are direct variable assignments. As shown in the figure below, NodeA.Output, NodeB.Input, NodeC.Input, and the input & output obtained in each Handler, if they are reference types like struct pointers or Maps, they are all the same piece of data. Therefore, whether in Node or Handler, it's not recommended to modify Input & Output, as it will cause concurrency issues: even in synchronous situations, Node B and Node C are concurrent, causing internal handler1 and handler2 to be concurrent. When there's asynchronous processing logic, there are more possible concurrency scenarios.
-### Stream Closing
+In stream passing scenarios, all downstream nodes and handlers' input streams are streams obtained from StreamReader.Copy(n), which can be read independently. However, each chunk in the stream is direct variable assignment, if the chunk is a reference type like struct pointer or Map, each copied stream reads the same piece of data. Therefore, in Node and Handler, it's also not recommended to modify stream chunks, there are concurrency issues.
-With true streaming components (e.g., ChatModel streams), callback consumers and downstream nodes both consume the stream. Streams are copied per consumer; ensure callback readers close their streams to avoid blocking resource release.
-
-
+
### Passing Information Between Handlers
-Use the returned `context.Context` to pass information across timings within the same handler (e.g., set with `context.WithValue` in `OnStart`, read in `OnEnd`). Do not rely on ordering between different handlers; if sharing data is required, store request-scoped shared state on the outermost context and ensure concurrency safety.
+Between different timings of the same Handler, information can be passed through ctx, such as returning a new context through context.WithValue in OnStart, and then retrieving this value from context in OnEnd.
+
+Between different Handlers, there's no guarantee of execution order, so it's not recommended to pass information between different Handlers through the above mechanism. Essentially, there's no guarantee that the context returned by one Handler will definitely enter the function execution of the next Handler.
+
+If you need to pass information between different Handlers, the recommended approach is to set a global, request-scoped variable in the outermost context (such as the context passed in when graph executes) as a shared space for storing and retrieving common information, and read and update this shared variable as needed in each Handler. Users need to ensure the concurrency safety of this shared variable themselves.
+
+### Remember to Close Streams
+
+Taking the existence of ChatModel, a node with true streaming output, as an example, when there are Callback aspects, ChatModel's output stream:
+
+- Needs to be consumed by downstream nodes as input, and also consumed by Callback aspects
+- A frame (Chunk) in a stream can only be consumed by one consumer, i.e., streams are not broadcast models
+
+So at this point, the stream needs to be copied, the copy relationship is as follows:
+
+
+
+- If one of the Callback n doesn't Close the corresponding stream, it may cause the original Stream to be unable to Close and release resources.
diff --git a/content/en/docs/eino/core_modules/chain_and_graph_orchestration/chain_graph_introduction.md b/content/en/docs/eino/core_modules/chain_and_graph_orchestration/chain_graph_introduction.md
index d11042c20a1..26212627646 100644
--- a/content/en/docs/eino/core_modules/chain_and_graph_orchestration/chain_graph_introduction.md
+++ b/content/en/docs/eino/core_modules/chain_and_graph_orchestration/chain_graph_introduction.md
@@ -1,6 +1,6 @@
---
Description: ""
-date: "2025-12-09"
+date: "2026-01-20"
lastmod: ""
tags: []
title: 'Eino: Chain/Graph Orchestration Introduction'
@@ -86,8 +86,8 @@ func (m *mockChatModel) Stream(ctx context.Context, input []*schema.Message, opt
sr, sw := schema.Pipe[*schema.Message](0)
go func() {
defer sw.Close()
- sw.Send(schema.AssistantMessage("the weather is", nil), nil)
- sw.Send(schema.AssistantMessage("good", nil), nil)
+ sw.Send(schema.AssistantMessage("the weather is", nil), nil)
+ sw.Send(schema.AssistantMessage("good", nil), nil)
}()
return sr, nil
}
@@ -134,7 +134,7 @@ func main() {
callbacks.AppendGlobalHandlers(&loggerCallbacks{})
// 1. create an instance of ChatTemplate as 1st Graph Node
- systemTpl := `你是一名房产经纪人，结合用户的薪酬和工作，使用 user_info API，为其提供相关的房产信息。邮箱是必须的`
+ systemTpl := `You are a real estate agent. Based on the user's salary and job, use the user_info API to provide relevant property information. Email is required.`
chatTpl := prompt.FromMessages(schema.FString,
schema.SystemMessage(systemTpl),
schema.MessagesPlaceholder("message_histories", true),
@@ -161,15 +161,15 @@ func main() {
userInfoTool := utils.NewTool(
&schema.ToolInfo{
Name: "user_info",
- Desc: "根据用户的姓名和邮箱，查询用户的公司、职位、薪酬信息",
+ Desc: "Query user's company, position, and salary information based on user's name and email",
ParamsOneOf: schema.NewParamsOneOfByParams(map[string]*schema.ParameterInfo{
"name": {
Type: "string",
- Desc: "用户的姓名",
+ Desc: "User's name",
},
"email": {
Type: "string",
- Desc: "用户的邮箱",
+ Desc: "User's email",
},
}),
},
@@ -212,8 +212,8 @@ func main() {
)
// 6. create an instance of Graph
- // input type is 1st Graph Node's input type: map[string]any
- // output type is last Graph Node's output type: []*schema.Message
+ // input type is 1st Graph Node's input type, that is ChatTemplate's input type: map[string]any
+ // output type is last Graph Node's output type, that is ToolsNode's output type: []*schema.Message
g := compose.NewGraph[map[string]any, []*schema.Message]()
// 7. add ChatTemplate into graph
@@ -225,13 +225,16 @@ func main() {
// 9. add ToolsNode into graph
_ = g.AddToolsNode(nodeKeyOfTools, toolsNode)
- // 10. connect nodes
+ // 10. add connection between nodes
_ = g.AddEdge(compose.START, nodeKeyOfTemplate)
+
_ = g.AddEdge(nodeKeyOfTemplate, nodeKeyOfChatModel)
+
_ = g.AddEdge(nodeKeyOfChatModel, nodeKeyOfTools)
+
_ = g.AddEdge(nodeKeyOfTools, compose.END)
- // compile Graph[I, O] to Runnable[I, O]
+ // 9. compile Graph[I, O] to Runnable[I, O]
r, err := g.Compile(ctx)
if err != nil {
logs.Errorf("Compile failed, err=%v", err)
@@ -240,14 +243,16 @@ func main() {
out, err := r.Invoke(ctx, map[string]any{
"message_histories": []*schema.Message{},
- "user_query": "我叫 zhangsan, 邮箱是 zhangsan@bytedance.com, 帮我推荐一处房产",
+ "user_query": "My name is zhangsan, email is zhangsan@bytedance.com, please recommend a property for me",
})
if err != nil {
logs.Errorf("Invoke failed, err=%v", err)
return
}
logs.Infof("Generation: %v Messages", len(out))
- for _, msg := range out { logs.Infof(" %v", msg) }
+ for _, msg := range out {
+ logs.Infof(" %v", msg)
+ }
}
type userInfoRequest struct {
@@ -291,14 +296,16 @@ func (l *loggerCallbacks) OnEndWithStreamOutput(ctx context.Context, info *callb
### Graph with state
-Graph can have a graph-level "global" state. Enable it via `WithGenLocalState` when creating the Graph:
+Graph can have a graph-level "global" state. Enable it via `WithGenLocalState` option when creating the Graph:
```go
// compose/generic_graph.go
// type GenLocalState[S any] func(ctx context.Context) (state S)
-func WithGenLocalState[S any](gls GenLocalState[S]) NewGraphOption { /* ... */ }
+func WithGenLocalState[S any](gls GenLocalState[S]) NewGraphOption {
+ // --snip--
+}
```
Add nodes with Pre/Post Handlers to process state:
@@ -309,8 +316,13 @@ Add nodes with Pre/Post Handlers to process state:
// type StatePreHandler[I, S any] func(ctx context.Context, in I, state S) (I, error)
// type StatePostHandler[O, S any] func(ctx context.Context, out O, state S) (O, error)
-func WithStatePreHandler[I, S any](pre StatePreHandler[I, S]) GraphAddNodeOpt { /* ... */ }
-func WithStatePostHandler[O, S any](post StatePostHandler[O, S]) GraphAddNodeOpt { /* ... */ }
+func WithStatePreHandler[I, S any](pre StatePreHandler[I, S]) GraphAddNodeOpt {
+ // --snip--
+}
+
+func WithStatePostHandler[O, S any](post StatePostHandler[O, S]) GraphAddNodeOpt {
+ // --snip--
+}
```
Inside a node, use `ProcessState` to read/write state:
@@ -355,70 +367,152 @@ func main() {
const (
nodeOfL1 = "invokable"
- nodeOfL2 = "streamable"
- nodeOfL3 = "transformable"
+ nodeOfL2 = "streamable"
+ nodeOfL3 = "transformable"
)
- type testState struct { ms []string }
- gen := func(ctx context.Context) *testState { return &testState{} }
+ type testState struct {
+ ms []string
+ }
+
+ gen := func(ctx context.Context) *testState {
+ return &testState{}
+ }
sg := compose.NewGraph[string, string](compose.WithGenLocalState(gen))
- l1 := compose.InvokableLambda(func(ctx context.Context, in string) (out string, err error) { return "InvokableLambda: " + in, nil })
- l1StateToInput := func(ctx context.Context, in string, state *testState) (string, error) { state.ms = append(state.ms, in); return in, nil }
- l1StateToOutput := func(ctx context.Context, out string, state *testState) (string, error) { state.ms = append(state.ms, out); return out, nil }
- _ = sg.AddLambdaNode(nodeOfL1, l1, compose.WithStatePreHandler(l1StateToInput), compose.WithStatePostHandler(l1StateToOutput))
-
- l2 := compose.StreamableLambda(func(ctx context.Context, input string) (*schema.StreamReader[string], error) {
- outStr := "StreamableLambda: " + input
- sr, sw := schema.Pipe[string](utf8.RuneCountInString(outStr))
- go func() {
- for _, field := range strings.Fields(outStr) { sw.Send(field+" ", nil) }
- sw.Close()
- }()
- return sr, nil
+ l1 := compose.InvokableLambda(func(ctx context.Context, in string) (out string, err error) {
+ return "InvokableLambda: " + in, nil
+ })
+
+ l1StateToInput := func(ctx context.Context, in string, state *testState) (string, error) {
+ state.ms = append(state.ms, in)
+ return in, nil
+ }
+
+ l1StateToOutput := func(ctx context.Context, out string, state *testState) (string, error) {
+ state.ms = append(state.ms, out)
+ return out, nil
+ }
+
+ _ = sg.AddLambdaNode(nodeOfL1, l1,
+ compose.WithStatePreHandler(l1StateToInput), compose.WithStatePostHandler(l1StateToOutput))
+
+ l2 := compose.StreamableLambda(func(ctx context.Context, input string) (output *schema.StreamReader[string], err error) {
+ outStr := "StreamableLambda: " + input
+
+ sr, sw := schema.Pipe[string](utf8.RuneCountInString(outStr))
+
+ // nolint: byted_goroutine_recover
+ go func() {
+ for _, field := range strings.Fields(outStr) {
+ sw.Send(field+" ", nil)
+ }
+ sw.Close()
+ }()
+
+ return sr, nil
})
- l2StateToOutput := func(ctx context.Context, out string, state *testState) (string, error) { state.ms = append(state.ms, out); return out, nil }
+
+ l2StateToOutput := func(ctx context.Context, out string, state *testState) (string, error) {
+ state.ms = append(state.ms, out)
+ return out, nil
+ }
+
_ = sg.AddLambdaNode(nodeOfL2, l2, compose.WithStatePostHandler(l2StateToOutput))
- l3 := compose.TransformableLambda(func(ctx context.Context, input *schema.StreamReader[string]) (*schema.StreamReader[string], error) {
- prefix := "TransformableLambda: "
- sr, sw := schema.Pipe[string](20)
- go func() {
- defer func() { if panicErr := recover(); panicErr != nil { err := safe.NewPanicErr(panicErr, debug.Stack()); logs.Errorf("panic occurs: %v\n", err) } }()
- for _, field := range strings.Fields(prefix) { sw.Send(field+" ", nil) }
- for {
- chunk, err := input.Recv()
- if err != nil { if err == io.EOF { break } ; sw.Send(chunk, err); break }
- sw.Send(chunk, nil)
- }
- sw.Close()
- }()
- return sr, nil
+ l3 := compose.TransformableLambda(func(ctx context.Context, input *schema.StreamReader[string]) (
+ output *schema.StreamReader[string], err error) {
+
+ prefix := "TransformableLambda: "
+ sr, sw := schema.Pipe[string](20)
+
+ go func() {
+
+ defer func() {
+ panicErr := recover()
+ if panicErr != nil {
+ err := safe.NewPanicErr(panicErr, debug.Stack())
+ logs.Errorf("panic occurs: %v\n", err)
+ }
+
+ }()
+
+ for _, field := range strings.Fields(prefix) {
+ sw.Send(field+" ", nil)
+ }
+
+ for {
+ chunk, err := input.Recv()
+ if err != nil {
+ if err == io.EOF {
+ break
+ }
+ // TODO: how to trace this kind of error in the goroutine of processing sw
+ sw.Send(chunk, err)
+ break
+ }
+
+ sw.Send(chunk, nil)
+
+ }
+ sw.Close()
+ }()
+
+ return sr, nil
})
+
l3StateToOutput := func(ctx context.Context, out string, state *testState) (string, error) {
- state.ms = append(state.ms, out)
- logs.Infof("state result: ")
- for idx, m := range state.ms { logs.Infof(" %vth: %v", idx, m) }
- return out, nil
+ state.ms = append(state.ms, out)
+ logs.Infof("state result: ")
+ for idx, m := range state.ms {
+ logs.Infof(" %vth: %v", idx, m)
+ }
+ return out, nil
}
+
_ = sg.AddLambdaNode(nodeOfL3, l3, compose.WithStatePostHandler(l3StateToOutput))
_ = sg.AddEdge(compose.START, nodeOfL1)
+
_ = sg.AddEdge(nodeOfL1, nodeOfL2)
+
_ = sg.AddEdge(nodeOfL2, nodeOfL3)
+
_ = sg.AddEdge(nodeOfL3, compose.END)
run, err := sg.Compile(ctx)
- if err != nil { logs.Errorf("sg.Compile failed, err=%v", err); return }
+ if err != nil {
+ logs.Errorf("sg.Compile failed, err=%v", err)
+ return
+ }
out, err := run.Invoke(ctx, "how are you")
- if err != nil { logs.Errorf("run.Invoke failed, err=%v", err); return }
+ if err != nil {
+ logs.Errorf("run.Invoke failed, err=%v", err)
+ return
+ }
logs.Infof("invoke result: %v", out)
stream, err := run.Stream(ctx, "how are you")
- if err != nil { logs.Errorf("run.Stream failed, err=%v", err); return }
- for { chunk, err := stream.Recv(); if err != nil { if errors.Is(err, io.EOF) { break } ; logs.Infof("stream.Recv() failed, err=%v", err); break } ; logs.Tokenf("%v", chunk) }
+ if err != nil {
+ logs.Errorf("run.Stream failed, err=%v", err)
+ return
+ }
+
+ for {
+
+ chunk, err := stream.Recv()
+ if err != nil {
+ if errors.Is(err, io.EOF) {
+ break
+ }
+ logs.Infof("stream.Recv() failed, err=%v", err)
+ break
+ }
+
+ logs.Tokenf("%v", chunk)
+ }
stream.Close()
sr, sw := schema.Pipe[string](1)
@@ -426,8 +520,24 @@ func main() {
sw.Close()
stream, err = run.Transform(ctx, sr)
- if err != nil { logs.Infof("run.Transform failed, err=%v", err); return }
- for { chunk, err := stream.Recv(); if err != nil { if errors.Is(err, io.EOF) { break } ; logs.Infof("stream.Recv() failed, err=%v", err); break } ; logs.Infof("%v", chunk) }
+ if err != nil {
+ logs.Infof("run.Transform failed, err=%v", err)
+ return
+ }
+
+ for {
+
+ chunk, err := stream.Recv()
+ if err != nil {
+ if errors.Is(err, io.EOF) {
+ break
+ }
+ logs.Infof("stream.Recv() failed, err=%v", err)
+ break
+ }
+
+ logs.Infof("%v", chunk)
+ }
stream.Close()
}
```
@@ -461,54 +571,105 @@ func main() {
modelName := os.Getenv("MODEL_NAME")
ctx := context.Background()
+ // build branch func
const randLimit = 2
- branchCond := func(ctx context.Context, input map[string]any) (string, error) {
- if rand.Intn(randLimit) == 1 { return "b1", nil }
+ branchCond := func(ctx context.Context, input map[string]any) (string, error) { // nolint: byted_all_nil_return
+ if rand.Intn(randLimit) == 1 {
+ return "b1", nil
+ }
+
return "b2", nil
}
b1 := compose.InvokableLambda(func(ctx context.Context, kvs map[string]any) (map[string]any, error) {
logs.Infof("hello in branch lambda 01")
- if kvs == nil { return nil, fmt.Errorf("nil map") }
+ if kvs == nil {
+ return nil, fmt.Errorf("nil map")
+ }
+
kvs["role"] = "cat"
return kvs, nil
})
+
b2 := compose.InvokableLambda(func(ctx context.Context, kvs map[string]any) (map[string]any, error) {
logs.Infof("hello in branch lambda 02")
- if kvs == nil { return nil, fmt.Errorf("nil map") }
- kvs["role"] = "dog"
- return kvs, nil
+ if kvs == nil {
+ return nil, fmt.Errorf("nil map")
+ }
+
+ kvs["role"] = "dog"
+ return kvs, nil
})
+ // build parallel node
parallel := compose.NewParallel()
- parallel.AddLambda("role", compose.InvokableLambda(func(ctx context.Context, kvs map[string]any) (string, error) {
- role, ok := kvs["role"].(string)
- if !ok || role == "" { role = "bird" }
- return role, nil
- })).AddLambda("input", compose.InvokableLambda(func(ctx context.Context, kvs map[string]any) (string, error) {
- return "你的叫声是怎样的？", nil
- }))
-
- modelConf := &openai.ChatModelConfig{ BaseURL: openAPIBaseURL, APIKey: openAPIAK, ByAzure: true, Model: modelName, Temperature: gptr.Of(float32(0.7)), APIVersion: "2024-06-01" }
+ parallel.
+ AddLambda("role", compose.InvokableLambda(func(ctx context.Context, kvs map[string]any) (string, error) {
+ // may be change role to others by input kvs, for example (dentist/doctor...)
+ role, ok := kvs["role"].(string)
+ if !ok || role == "" {
+ role = "bird"
+ }
+
+ return role, nil
+ })).
+ AddLambda("input", compose.InvokableLambda(func(ctx context.Context, kvs map[string]any) (string, error) {
+ return "What does your call sound like?", nil
+ }))
+
+ modelConf := &openai.ChatModelConfig{
+ BaseURL: openAPIBaseURL,
+ APIKey: openAPIAK,
+ ByAzure: true,
+ Model: modelName,
+ Temperature: gptr.Of(float32(0.7)),
+ APIVersion: "2024-06-01",
+ }
+
+ // create chat model node
cm, err := openai.NewChatModel(context.Background(), modelConf)
- if err != nil { log.Panic(err); return }
+ if err != nil {
+ log.Panic(err)
+ return
+ }
rolePlayerChain := compose.NewChain[map[string]any, *schema.Message]()
- rolePlayerChain.AppendChatTemplate(prompt.FromMessages(schema.FString, schema.SystemMessage(`You are a {role}.`), schema.UserMessage(`{input}`))).AppendChatModel(cm)
+ rolePlayerChain.
+ AppendChatTemplate(prompt.FromMessages(schema.FString, schema.SystemMessage(`You are a {role}.`), schema.UserMessage(`{input}`))).
+ AppendChatModel(cm)
+ // =========== build chain ===========
chain := compose.NewChain[map[string]any, string]()
- chain.AppendLambda(compose.InvokableLambda(func(ctx context.Context, kvs map[string]any) (map[string]any, error) {
- logs.Infof("in view lambda: %v", kvs)
- return kvs, nil
- })).AppendBranch(compose.NewChainBranch(branchCond).AddLambda("b1", b1).AddLambda("b2", b2)).AppendPassthrough().AppendParallel(parallel).AppendGraph(rolePlayerChain).AppendLambda(compose.InvokableLambda(func(ctx context.Context, m *schema.Message) (string, error) {
- logs.Infof("in view of messages: %v", m.Content)
- return m.Content, nil
- }))
-
+ chain.
+ AppendLambda(compose.InvokableLambda(func(ctx context.Context, kvs map[string]any) (map[string]any, error) {
+ // do some logic to prepare kv as input val for next node
+ // just pass through
+ logs.Infof("in view lambda: %v", kvs)
+ return kvs, nil
+ })).
+ AppendBranch(compose.NewChainBranch(branchCond).AddLambda("b1", b1).AddLambda("b2", b2)). // nolint: byted_use_receiver_without_nilcheck
+ AppendPassthrough().
+ AppendParallel(parallel).
+ AppendGraph(rolePlayerChain).
+ AppendLambda(compose.InvokableLambda(func(ctx context.Context, m *schema.Message) (string, error) {
+ // do some logic to check the output or something
+ logs.Infof("in view of messages: %v", m.Content)
+ return m.Content, nil
+ }))
+
+ // compile
r, err := chain.Compile(ctx)
- if err != nil { log.Panic(err); return }
+ if err != nil {
+ log.Panic(err)
+ return
+ }
+
output, err := r.Invoke(context.Background(), map[string]any{})
- if err != nil { log.Panic(err); return }
+ if err != nil {
+ log.Panic(err)
+ return
+ }
+
logs.Infof("output is : %v", output)
}
```
diff --git a/content/en/docs/eino/core_modules/chain_and_graph_orchestration/checkpoint_interrupt.md b/content/en/docs/eino/core_modules/chain_and_graph_orchestration/checkpoint_interrupt.md
index f226fa7281d..d8ff8c97319 100644
--- a/content/en/docs/eino/core_modules/chain_and_graph_orchestration/checkpoint_interrupt.md
+++ b/content/en/docs/eino/core_modules/chain_and_graph_orchestration/checkpoint_interrupt.md
@@ -1,12 +1,13 @@
---
Description: ""
-date: "2025-12-09"
+date: "2026-01-20"
lastmod: ""
tags: []
title: 'Eino: Interrupt & CheckPoint Manual'
weight: 7
---
+> 💡
> Note: A bug in v0.3.26 broke CheckPoint serialization. For new CheckPoint usage, use v0.3.26+ (preferably latest).
>
> Eino provides a compatibility branch for users with pre-v0.3.26 checkpoints to migrate; once old data is invalidated, upgrade to mainline. The branch incurs overhead and is not merged due to typical short checkpoint lifetimes.
@@ -15,6 +16,7 @@ weight: 7
`Interrupt & CheckPoint` lets you pause a Graph at specified locations and resume later. For `StateGraph`, you can modify `State` before resuming.
+> 💡
> Resuming restores inputs and per-node runtime data. Ensure the Graph orchestration is identical and pass the same CallOptions again (unless you explicitly rely on CallOptions to carry resume-time data).
## Using Static Interrupt
@@ -28,9 +30,9 @@ import (
func main() {
g := NewGraph[string, string]()
- err := g.AddLambdaNode("node1", compose.InvokableLambda(func(ctx **context**._Context_, input string) (output string, err error) {/*invokable func*/})
+ err := g.AddLambdaNode("node1", compose.InvokableLambda(func(ctx context.Context, input string) (output string, err error) {/*invokable func*/})
if err != nil {/* error handle */}
- err = g.AddLambdaNode("node2", compose.InvokableLambda(func(ctx **context**._Context_, input string) (output string, err error) {/*invokable func*/})
+ err = g.AddLambdaNode("node2", compose.InvokableLambda(func(ctx context.Context, input string) (output string, err error) {/*invokable func*/})
if err != nil {/* error handle */}
/** other graph composed code
@@ -42,11 +44,14 @@ func main() {
}
```
+> 💡
> Tip: Currently only compile-time static breakpoints are supported. If you need request-time configuration, please open an issue.
Extract interrupt info from the run error:
```go
+// compose/checkpoint.go
+
type InterruptInfo struct {
State any
BeforeNodes []string
@@ -57,7 +62,7 @@ type InterruptInfo struct {
InterruptContexts []*InterruptCtx
}
-func ExtractInterruptInfo(err error) (info *InterruptInfo, existed bool)
+func ExtractInterruptInfo(err error) (info *InterruptInfo, existed bool) {}
```
Example:
@@ -80,6 +85,7 @@ if err != nil {
}
```
+> 💡
> During interrupt, the output is empty and should be ignored.
## Using CheckPoint
@@ -94,8 +100,8 @@ CheckPoint records Graph runtime state to support resuming.
// compose/checkpoint.go
type CheckpointStore interface {
- Get(ctx **context**._Context_, key string) (value []byte, existed bool,err error)
- Set(ctx **context**._Context_, key string, value []byte) (err error)
+ Get(ctx context.Context, key string) (value []byte, existed bool,err error)
+ Set(ctx context.Context, key string, value []byte) (err error)
}
```
@@ -116,13 +122,15 @@ type MyState struct {
func init() {
// Register the type with a stable name for serialization/persistence.
// Use the pointer form if you persist pointers to this type.
- // Recommended to register within init() in the same file where the type is declared.
+ // It's recommended to register types within the `init()` function
+ // within the same file your type is declared.
schema.RegisterName[*MyState]("my_state_v1")
}
```
After registration, type metadata is included during serialization. On deserialization, Eino can restore the correct type even when the destination is `interface{}`. The key uniquely identifies the type; once chosen, do not change it, otherwise persisted checkpoints cannot be restored.
+> 💡
> Struct unexported fields are inaccessible and thus not stored/restored.
By default, Eino uses its built-in serializer. If a registered type implements `json.Marshaler` and `json.Unmarshaler`, those custom methods are used.
@@ -144,7 +152,7 @@ Eino also provides an option to use `gob` serialization:
```go
r, err := compose.NewChain[*AgentInput, Message]().
AppendLambda(compose.InvokableLambda(func(ctx context.Context, input *AgentInput) ([]Message, error) {
- return a.genModelInput(ctx, instruction, input
+ return a.genModelInput(ctx, instruction, input)
})).
AppendChatModel(a.model).
Compile(ctx, compose.WithGraphName(a.name),
@@ -175,32 +183,13 @@ func main() {
At request time, provide a checkpoint ID:
-```go
-func WithCheckPointID(checkPointID string, sm StateModifier) Option
```
+// compose/checkpoint.go
-The checkpoint ID is used as the `CheckpointStore` key. During execution, if the ID exists, the graph resumes from it; on interrupt, the graph stores its state under that ID.
-
-```go
-/* graph compose and compile
-xxx
-*/
-
-// first run interrupt
-id := GenUUID()
-_, err := runner.Invoke(ctx, input, WithCheckPointID(id))
-
-// resume from id
-_, err = runner.Invoke(ctx, input/*unused*/,
- WithCheckPointID(id),
- WithStateModifier(func(ctx context.Context, path NodePath, state any) error{
- state.(*testState).Field1 = "hello"
- return nil
- }),
-)
+func WithCheckPointID(checkPointID string) Option
```
-> During resume, input is ignored; pass a zero value.
+The checkpoint ID is used as the `CheckpointStore` key. During execution, if the ID exists, the graph resumes from it; on interrupt, the graph stores its state under that ID.
## Dynamic Interrupt
@@ -218,7 +207,7 @@ var InterruptAndRerun = errors.New("interrupt and rerun")
func NewInterruptAndRerunErr(extra any) error
```
-When the graph receives such an error, it interrupts. On resume, the node runs again; before rerun, `StateModifier` is applied if configured. The rerun’s input is replaced with a zero value rather than the original; if the original input is needed, save it into `State` beforehand.
+When the graph receives such an error, it interrupts. On resume, the node runs again; before rerun, `StateModifier` is applied if configured. The rerun's input is replaced with a zero value rather than the original; if the original input is needed, save it into `State` beforehand.
### From Eino v0.7.0 onward
@@ -234,19 +223,47 @@ func Interrupt(ctx context.Context, info any) error
// persistent LOCALLY-DEFINED state
func StatefulInterrupt(ctx context.Context, info any, state any) error
-// emit an interrupt signal WRAPPING other interrupt signals
-// emitted from inner processes,
+// emit an interrupt signal WRAPPING other interrupt signals
+// emitted from inner processes,
// such as ToolsNode wrapping Tools.
func CompositeInterrupt(ctx context.Context, info any, state any, errs ...error)
```
See design details: [Eino human-in-the-loop framework: architecture guide](/docs/eino/core_modules/eino_adk/agent_hitl)
+## External Active Interrupt
+
+Sometimes, we want to actively trigger an interrupt from outside the Graph, save the state, and resume later. These scenarios may include graceful instance shutdown, etc. In such cases, you can call `WithGraphInterrupt` to get a ctx and an interrupt function. The ctx is passed to `graph.Invoke()` and other run methods, while the interrupt function is called when you want to actively interrupt:
+
+```go
+// from compose/graph_call_options.go
+
+// WithGraphInterrupt creates a context with graph cancellation support.
+// When the returned context is used to invoke a graph or workflow, calling the interrupt function will trigger an interrupt.
+// The graph will wait for current tasks to complete by default.
+func WithGraphInterrupt(parent context.Context) (ctx context.Context, interrupt func(opts ...GraphInterruptOption)) {}
+```
+
+When actively calling the interrupt function, you can pass parameters such as timeout:
+
+```go
+// from compose/graph_call_options.go
+
+// WithGraphInterruptTimeout specifies the max waiting time before generating an interrupt.
+// After the max waiting time, the graph will force an interrupt. Any unfinished tasks will be re-run when the graph is resumed.
+func WithGraphInterruptTimeout(timeout time.Duration) GraphInterruptOption {
+ return func(o *graphInterruptOptions) {
+ o.timeout = &timeout
+ }
+}
+```
+
## CheckPoint in Streaming
Streaming checkpoints require concatenation of chunks. Register a concat function:
```go
+// compose/stream_concat.go
func RegisterStreamChunkConcatFunc[T any](fn func([]T) (T, error))
// example
@@ -271,6 +288,9 @@ Eino provides defaults for `*schema.Message`, `[]*schema.Message`, and `string`.
When the parent sets a `CheckpointStore`, use `WithGraphCompileOptions` during `AddGraphNode` to configure child interrupts:
```go
+/* graph compose code
+xxx
+*/
g.AddGraphNode("node1", subGraph, WithGraphCompileOptions(
WithInterruptAfterNodes([]string{"node2"}),
))
@@ -278,7 +298,7 @@ g.AddGraphNode("node1", subGraph, WithGraphCompileOptions(
g.Compile(ctx, WithCheckPointStore(cp))
```
-If a child interrupts, resuming modifies the child’s state. TODO: clarify Path usage in `StateModifier`.
+If a child interrupts, resuming modifies the child's state. TODO: clarify Path usage in `StateModifier`.
## Recovery
@@ -307,7 +327,7 @@ id := GenUUID()
_, err := runner.Invoke(ctx, input, WithCheckPointID(id))
// resume from id
-_, err = runner.Invoke(ctx, input/*unused*/,
+_, err = runner.Invoke(ctx, input/*unused*/,
WithCheckPointID(id),
WithStateModifier(func(ctx context.Context, path NodePath, state any) error{
state.(*testState).Field1 = "hello"
@@ -316,6 +336,7 @@ _, err = runner.Invoke(ctx, input/*unused*/,
)
```
+> 💡
> During resume, input is ignored; pass a zero value.
### From Eino v0.7.0 onward
@@ -323,7 +344,7 @@ _, err = runner.Invoke(ctx, input/*unused*/,
In addition to `StateModifier`, you can selectively resume particular interrupt points and provide resume data:
```go
-// specifically resume particular interrupt point(s),
+// specifically resume particular interrupt point(s),
// without specifying resume data
func Resume(ctx context.Context, interruptIDs ...string) context.Context
@@ -388,14 +409,26 @@ func (i InvokableApprovableTool) InvokableRun(ctx context.Context, argumentsInJS
}
```
-## Examples
+# Examples
+
+### Prior to Eino v0.7.0
+
+[https://github.com/cloudwego/eino-examples/tree/main/compose/graph/react_with_interrupt](https://github.com/cloudwego/eino-examples/tree/main/compose/graph/react_with_interrupt)
+
+### From Eino v0.7.0 onward
+
+[https://github.com/cloudwego/eino/blob/main/compose/resume_test.go](https://github.com/cloudwego/eino/blob/main/compose/resume_test.go)
+
+Including:
+
+`TestInterruptStateAndResumeForRootGraph`: simple dynamic interrupt
+
+`TestInterruptStateAndResumeForSubGraph`: subgraph interrupt
+
+`TestInterruptStateAndResumeForToolInNestedSubGraph`: nested subgraph tool interrupt
+
+`TestMultipleInterruptsAndResumes`: parallel interrupts
-- https://github.com/cloudwego/eino-examples/tree/main/compose/graph/react_with_interrupt
-- https://github.com/cloudwego/eino/blob/main/compose/resume_test.go
+`TestReentryForResumedTools`: tool interrupt in ReAct Agent, multiple re-entries after resume
-- `TestInterruptStateAndResumeForRootGraph`: simple dynamic interrupt
-- `TestInterruptStateAndResumeForSubGraph`: subgraph interrupt
-- `TestInterruptStateAndResumeForToolInNestedSubGraph`: nested subgraph tool interrupt
-- `TestMultipleInterruptsAndResumes`: parallel interrupts
-- `TestReentryForResumedTools`: tool interrupt in ReAct Agent, multiple re-entries after resume
-- `TestGraphInterruptWithinLambda`: Lambda node contains a standalone Graph and interrupts internally
+`TestGraphInterruptWithinLambda`: Lambda node contains a standalone Graph and interrupts internally
diff --git a/content/en/docs/eino/core_modules/chain_and_graph_orchestration/orchestration_design_principles.md b/content/en/docs/eino/core_modules/chain_and_graph_orchestration/orchestration_design_principles.md
index 64561088024..e7124274fd0 100644
--- a/content/en/docs/eino/core_modules/chain_and_graph_orchestration/orchestration_design_principles.md
+++ b/content/en/docs/eino/core_modules/chain_and_graph_orchestration/orchestration_design_principles.md
@@ -1,103 +1,181 @@
---
Description: ""
-date: "2025-12-09"
+date: "2026-01-20"
lastmod: ""
tags: []
title: 'Eino: Orchestration Design Principles'
weight: 2
---
-`langchain`/`langgraph` are popular orchestration solutions in Python/TS — both highly flexible languages. Flexibility accelerates SDK development but often burdens users with ambiguity. Go’s simplicity and static typing help reduce cognitive load. Eino embraces this with “deterministic types” plus “compile-time type checking”.
+The mainstream language for large model application orchestration frameworks is Python, a language known for its flexibility. While flexibility facilitates SDK development, it also places a cognitive burden on SDK users.
-## Upstream–Downstream Type Alignment as a First Principle
+Eino, based on Golang, is `statically typed`, performing type checking at compile time, avoiding the runtime type issues of dynamic languages like Python.
-Eino’s orchestration centers on Graph (and simplified Chain). Fundamentally, it’s “logic nodes” plus “upstream/downstream relations”. At runtime, outputs of one node become inputs of the next.
+## Upstream-Downstream `Type Alignment` as the Fundamental Principle
-We assume: the upstream output can be fed to the downstream input.
+Eino's most basic orchestration method is graph, along with the simplified wrapper chain. Regardless of the orchestration method, the essence is `logic nodes` + `upstream/downstream relationships`. When the orchestration product runs, it executes from one logic node, then proceeds to run the next node connected to it.
-In Go, two approaches:
+This implies a fundamental assumption: **The output value of the previous running node can be used as the input value of the next node.**
-1) Use generalized types (e.g., `any`, `map[string]any`) everywhere.
- - With `any`, developers must assert types repeatedly; high cognitive load.
- - With `map[string]any`, nodes extract values by keys. Still requires type assertions, not ideal.
+In Golang, there are two basic approaches to implement this assumption:
-2) Preserve each node’s expected types, and enforce upstream–downstream compatibility at compile time.
+1. Convert the inputs and outputs of different nodes to a more generalized type, such as `any`, `map[string]any`, etc.
+ 1. Adopting the approach of generalizing to any, but the corresponding cost is: developers need to explicitly convert to specific types when writing code. This greatly increases the cognitive burden on developers, so this approach was ultimately abandoned.
+ 2. LangChain's approach can be seen as passing `map[string]any` throughout, where each logic node uses the corresponding key to get the corresponding value according to its needs. In the langchaingo implementation, this is exactly how it's done, but similarly, any in Golang still requires `type assertion` to be used. This approach still has a significant cognitive burden for developers.
+2. Keep the input and output types of each node as expected by developers, and ensure upstream and downstream types are consistent during the Compile phase.
-Eino chooses (2). Orchestration becomes like “LEGO”: only matching studs/sockets connect.
+Approach 2 is the final solution chosen by Eino. This approach is the easiest to understand when orchestrating - the whole process is like `building blocks`, where each block's protruding and recessed parts have their own specifications, and only matching specifications can form upstream/downstream relationships.
+
+As shown below:
-Only downstream nodes that understand upstream outputs can run. Eino makes this explicit so developers can build with confidence instead of guessing with `any`.
+For any orchestration, only when the downstream can recognize and process the upstream's output can the orchestration run normally. This fundamental assumption is clearly expressed in Eino, allowing developers to have full confidence in understanding how the orchestration logic runs and flows, rather than guessing whether the values passed from a series of any are correct.
### Type Alignment in Graph
-#### Edges
+#### Edge
+
+In a graph, a node's output flows to the next node along an `edge`, therefore, nodes connected by edges must be type-aligned.
+
+As shown below:
+
+> This simulates a scenario of ① direct conversation with a large model ② using RAG mode, with results that can be used to compare the effects of both modes
-Edges require assignable types:
+The green parts in the diagram are normal Edge connections, which require that the upstream output must be `assignable` to the downstream. The acceptable types are:
+
+① Same upstream and downstream types: e.g., upstream outputs *schema.Message and downstream input is also *schema.Message
+
+② Downstream receives an interface, upstream implements that interface: e.g., upstream struct implements the Format() interface, downstream receives an interface{ Format() }. A special case is when downstream is any (empty interface), upstream always implements any, so it can always connect.
+
+③ Upstream is an interface, downstream is a concrete type: When the downstream concrete type implements the upstream interface type, it may or may not work - this cannot be determined at compile time, only at runtime when the upstream's concrete type is determined. For detailed description, see: [Eino: Orchestration Design Principles](/docs/eino/core_modules/chain_and_graph_orchestration/orchestration_design_principles)
+
+The yellow parts in the diagram show another type conversion mechanism provided by Eino: if the downstream receives type `map[string]any`, but the upstream output type is not map[string]any, you can use `graph.AddXXXNode(node_key, xxx, compose.WithOutputKey("outkey")` to convert the upstream output type to map[string]any, where the map's key is the OutputKey specified in the option. This mechanism is convenient when multiple edges converge to a single node.
+
+Similarly, if the upstream is `map[string]any`, but the downstream input type is not map[string]any, you can use `graph.AddXXXNode(node_key, xxx, compose.WithInputKey("inkey")` to get one key's value from the upstream output as the downstream's input.
-1) Same types: e.g., upstream `*schema.Message` → downstream `*schema.Message`.
-2) Downstream expects an interface that upstream implements. Special case: downstream `any` — everything assigns.
-3) Upstream is an interface, downstream is a concrete type: depends on runtime concrete type; compile-time cannot guarantee. Only when the upstream concrete type implements the downstream expectation will it work.
+#### Branch
-Yellow paths show Eino’s map conversion: when downstream needs `map[string]any` but upstream doesn’t produce it, use `compose.WithOutputKey("outkey")` to wrap upstream output into a map with the given key. Similarly, `compose.WithInputKey("inkey")` lets downstream pick a specific key from upstream’s map output.
+If a node is followed by multiple edges, each edge's downstream node will run once. Branch is another mechanism: a branch is followed by n nodes, but only the node corresponding to the node key returned by the condition will run. Nodes after the same branch must be type-aligned.
-#### Branches
+As shown below:
-A node with multiple edges runs all downstream nodes. A `Branch` chooses exactly one downstream based on a condition function. All branch targets must be type-compatible with upstream outputs.
+> This simulates the running logic of a react agent
+As you can see, a branch itself has a `condition`, and this function's input must be type-aligned with the upstream. At the same time, each node connected after a branch must also, like the condition, be able to receive the upstream's output.
+
### Type Alignment in Chain
#### Chain
+From an abstract perspective, a chain is a `chain`, as shown below:
+
-All node pairs must align. Example:
+Logic node types can be divided into 3 categories:
+
+- Orchestrable components (e.g., chat model, chat template, retriever, lambda, graph, etc.)
+- Branch nodes
+- Parallel nodes
+
+As you can see, from the chain's perspective, whether it's a simple node (e.g., chat model) or a complex node (e.g., graph, branch, parallel), they are all the same - during execution, one step is one node's execution.
+
+Therefore, between upstream and downstream nodes in a chain, types must be aligned, as follows:
```go
-chain := compose.NewChain[map[string]interface{}, string]()
-chain.
- AppendChatTemplate(&fakeChatTemplate{}).
- AppendLambda(&fakeLambda{}).
- AppendChatModel(&fakeChatModel{}).
- AppendLambda(&fakeLambda{})
+func TestChain() {
+ chain := compose.NewChain[map[string]interface,string]()
+
+ nodeTemplate := &fakeChatTemplate{} // input: map[string]any, output: []*schema.Message
+
+ nodeHistoryLambda := &fakeLambda{} // input: []*schema.Message, output: []*schema.Message
+
+ nodeChatModel := &fakeChatModel{} // input: []*schema.Message, output: *schema.Message
+
+ nodeConvertResLambda := &fakeLambda{} // input: *schema.Message, output: string
+
+ chain.
+ AppendChatTemplate(nodeTemplate).
+ AppendLambda(nodeHistoryLambda).
+ AppendChatModel(nodeChatModel).
+ AppendLambda(nodeConvertResLambda)
+}
```
+The above logic represented as a diagram:
+
-Misalignment causes compile errors: Chain errors at `Compile()`, Graph errors at `AddXXXNode()`.
+If upstream and downstream types are not aligned, chain will return an error at chain.Compile(). Graph will report an error at graph.AddXXXNode().
#### Parallel
+Parallel is a special type of node in chain. From the chain's perspective, parallel is no different from other nodes. Inside parallel, its basic topology structure is as follows:
+
-Parallel assumes exactly one node per branch (that node can itself be a Graph). All parallel nodes must accept the upstream’s output type. Parallel outputs a `map[string]any`, with keys from `AddXXX(outKey, ...)` and values as node outputs.
+One of the structures formed by multiple edges in a graph is this. The basic assumption here is: each edge in a parallel has exactly one node. Of course, this one node can also be a graph. Note that currently the framework does not directly provide the ability to nest branch or parallel within parallel.
+
+For each node in parallel, since their upstream node is the same, they all need to be type-aligned with the upstream node's output type. For example, in the diagram, the upstream node outputs `*schema.Message`, so each node must be able to receive this type. The receiving methods are the same as in graph, typically using `same type`, `interface definition`, `any`, or `input key option`.
+
+The output of a parallel node is always a `map[string]any`, where the key is the output_key specified in `parallel.AddXXX(output_key, xxx, opts...)`, and the value is the actual output of the internal node.
+
+An example of building a parallel:
+
+```go
+func TestParallel() {
+ chain := compose.NewChain[map[string]any, map[string]*schema.Message]()
+
+ parallel := compose.NewParallel()
+ model01 := &fakeChatModel{} // input: []*schema.Message, output: *schema.Message
+ model02 := &fakeChatModel{} // input: []*schema.Message, output: *schema.Message
+ model03 := &fakeChatModel{} // input: []*schema.Message, output: *schema.Message
+
+ parallel.
+ AddChatModel("outkey_01", model01).
+ AddChatModel("outkey_02", model02).
+ AddChatModel("outkey_03", model03)
+
+ lambdaNode := &fakeLambdaNode{} // input: map[string]any, output: map[string]*schema.Message
+
+ chain.
+ AppendParallel(parallel).
+ AppendLambda(lambdaNode)
+}
+```
+
+A parallel's view in a chain is as follows:
+
+> The diagram simulates the same question being answered by different large models, with results that can be used to compare effects
-#### Branch in Chain
+> Note that this structure is only a logical view. Since chain itself is implemented using graph, parallel will be flattened into the underlying graph.
-Similar to Graph; all branch targets must align. Chain branches typically converge to the same downstream node or END.
+#### Branch
+
+Chain's branch is similar to graph's branch - all nodes in the branch must be type-aligned with the upstream node, so we won't elaborate here. The special thing about chain branch is that all possible branch nodes will connect to the same node in the chain, or all will connect to END.
### Type Alignment in Workflow
-Field-level mapping replaces whole-object alignment:
+The dimension of type alignment in Workflow changes from overall Input & Output to field level. Specifically:
-- Whole output → specific field
-- Specific field → whole input
-- Specific field → specific field
+- The overall upstream output is type-aligned to a specific field of the downstream.
+- A specific field of the upstream output is type-aligned to the overall downstream.
+- A specific field of the upstream output is type-aligned to a specific field of the downstream input.
-Same principles apply as whole-object alignment.
+The principles and rules are the same as overall type alignment.
-### StateHandlers
+### Type Alignment of StateHandler
-StatePreHandler: input type must align with the node’s non‑streaming input type.
+StatePreHandler: The input type needs to align with the corresponding node's non-streaming input type.
```go
-// input type: []*schema.Message, aligns to ChatModel non‑streaming input
+// input type is []*schema.Message, aligns with ChatModel's non-streaming input type
preHandler := func(ctx context.Context, input []*schema.Message, state *state) ([]*schema.Message, error) {
// your handler logic
}
@@ -105,10 +183,10 @@ preHandler := func(ctx context.Context, input []*schema.Message, state *state) (
AddChatModelNode("xxx", model, WithStatePreHandler(preHandler))
```
-StatePostHandler: input type must align with the node’s non‑streaming output type.
+StatePostHandler: The input type needs to align with the corresponding node's non-streaming output type.
```go
-// input type: *schema.Message, aligns to ChatModel non‑streaming output
+// input type is *schema.Message, aligns with ChatModel's non-streaming output type
postHandler := func(ctx context.Context, input *schema.Message, state *state) (*schema.Message, error) {
// your handler logic
}
@@ -116,10 +194,10 @@ postHandler := func(ctx context.Context, input *schema.Message, state *state) (*
AddChatModelNode("xxx", model, WithStatePostHandler(postHandler))
```
-StreamStatePreHandler: input type must align with the node’s streaming input type.
+StreamStatePreHandler: The input type needs to align with the corresponding node's streaming input type.
```go
-// input type: *schema.StreamReader[[]*schema.Message], aligns to ChatModel streaming input
+// input type is *schema.StreamReader[[]*schema.Message], aligns with ChatModel's streaming input type
preHandler := func(ctx context.Context, input *schema.StreamReader[[]*schema.Message], state *state) (*schema.StreamReader[[]*schema.Message], error) {
// your handler logic
}
@@ -127,10 +205,10 @@ preHandler := func(ctx context.Context, input *schema.StreamReader[[]*schema.Mes
AddChatModelNode("xxx", model, WithStreamStatePreHandler(preHandler))
```
-StreamStatePostHandler: input type must align with the node’s streaming output type.
+StreamStatePostHandler: The input type needs to align with the corresponding node's streaming output type.
```go
-// input type: *schema.StreamReader[*schema.Message], aligns to ChatModel streaming output
+// input type is *schema.StreamReader[*schema.Message], aligns with ChatModel's streaming output type
postHandler := func(ctx context.Context, input *schema.StreamReader[*schema.Message], state *state) (*schema.StreamReader[*schema.Message], error) {
// your handler logic
}
@@ -138,145 +216,262 @@ postHandler := func(ctx context.Context, input *schema.StreamReader[*schema.Mess
AddChatModelNode("xxx", model, WithStreamStatePostHandler(postHandler))
```
-### Invoke vs Stream Alignment
+### Type Alignment in Invoke and Stream Modes
+
+In Eino, the result of orchestration is a graph or chain. To run it, you need to use `Compile()` to generate a `Runnable` interface.
-`Runnable` offers `Invoke/Stream/Collect/Transform`. See [Streaming Essentials](/docs/eino/core_modules/chain_and_graph_orchestration/stream_programming_essentials).
+An important function of Runnable is to provide four calling methods: "Invoke", "Stream", "Collect", and "Transform".
-Assume a `Graph[[]*schema.Message, []*schema.Message]` with a ChatModel node and a Lambda node, compiled to `Runnable[[]*schema.Message, []*schema.Message]`:
+> For an introduction to the above calling methods and detailed Runnable introduction, see: [Eino Stream Programming Essentials](/docs/eino/core_modules/chain_and_graph_orchestration/stream_programming_essentials)
+
+Suppose we have a `Graph[[]*schema.Message, []*schema.Message]` with a ChatModel node and a Lambda node. After Compile, it becomes a `Runnable[[]*schema.Message, []*schema.Message]`.
```go
-g1 := compose.NewGraph[[]*schema.Message, string]()
-_ = g1.AddChatModelNode("model", &mockChatModel{})
-_ = g1.AddLambdaNode("lambda", compose.InvokableLambda(func(_ context.Context, msg *schema.Message) (string, error) {
- return msg.Content, nil
-}))
-_ = g1.AddEdge(compose.START, "model")
-_ = g1.AddEdge("model", "lambda")
-_ = g1.AddEdge("lambda", compose.END)
-
-runner, _ := g1.Compile(ctx)
-c, _ := runner.Invoke(ctx, []*schema.Message{ schema.UserMessage("what's the weather in beijing?") })
-s, _ := runner.Stream(ctx, []*schema.Message{ schema.UserMessage("what's the weather in beijing?") })
-var fullStr string
-for {
- chunk, err := s.Recv()
- if err != nil {
- if err == io.EOF { break }
- panic(err)
- }
- fullStr += chunk
+package main
+
+import (
+ "context"
+ "io"
+ "testing"
+
+ "github.com/cloudwego/eino/compose"
+ "github.com/cloudwego/eino/schema"
+ "github.com/stretchr/testify/assert"
+)
+
+func TestTypeMatch(t *testing.T) {
+ ctx := context.Background()
+
+ g1 := compose.NewGraph[[]*schema.Message, string]()
+ _ = g1.AddChatModelNode("model", &mockChatModel{})
+ _ = g1.AddLambdaNode("lambda", compose.InvokableLambda(func(_ context.Context, msg *schema.Message) (string, error) {
+ return msg.Content, nil
+ }))
+ _ = g1.AddEdge(compose.START, "model")
+ _ = g1.AddEdge("model", "lambda")
+ _ = g1.AddEdge("lambda", compose.END)
+
+ runner, err := g1.Compile(ctx)
+ assert.NoError(t, err)
+
+ c, err := runner.Invoke(ctx, []*schema.Message{
+ schema.UserMessage("what's the weather in beijing?"),
+ })
+ assert.NoError(t, err)
+ assert.Equal(t, "the weather is good", c)
+
+ s, err := runner.Stream(ctx, []*schema.Message{
+ schema.UserMessage("what's the weather in beijing?"),
+ })
+ assert.NoError(t, err)
+
+ var fullStr string
+ for {
+ chunk, err := s.Recv()
+ if err != nil {
+ if err == io.EOF {
+ break
+ }
+ panic(err)
+ }
+
+ fullStr += chunk
+ }
+ assert.Equal(t, c, fullStr)
}
```
-In Stream mode, ChatModel outputs `*schema.StreamReader[*schema.Message]`, while the downstream InvokableLambda expects non‑stream `*schema.Message`. Eino auto‑concatenates streamed frames into a full message, satisfying type alignment.
+When we call the compiled Runnable above in Stream mode, the model node will output `*schema.StreamReader[*Message]`, but the lambda node is an InvokableLambda that only accepts non-streaming `*schema.Message` as input. This also conforms to the type alignment rules because the Eino framework will automatically concatenate the streamed Message into a complete Message.
+
+In stream mode, concatenating frames is a very common operation. During concatenation, all elements from `*StreamReader[T]` are first extracted and converted to `[]T`, then an attempt is made to concatenate `[]T` into a complete `T`. The framework has built-in support for concatenating the following types:
+
+- `*schema.Message`: See `schema.ConcatMessages()`
+- `string`: Implementation logic is equivalent to `+=`
+- `[]*schema.Message`: See `compose.concatMessageArray()`
+- `Map`: Merge values with the same key, with the same merge logic as above. If there are types that cannot be merged, it fails (note: not overwrite)
+- Other slices: Can only be merged when the slice has exactly one non-zero element.
+
+For other scenarios, or when users want to override the default behavior above with custom logic, developers can implement their own concat method and register it to the global concatenation function using `compose.RegisterStreamChunkConcatFunc()`.
+
+Example:
+
+```go
+// Assume our own struct is as follows
+type tStreamConcatItemForTest struct {
+ s string
+}
-Concatenation behavior:
-- `*schema.Message`: see `schema.ConcatMessages()`
-- `string`: equivalent to `+=`
-- `[]*schema.Message`: concatenated via framework helper
-- `Map`: merge values by key with type‑appropriate concatenation; fails if types cannot be merged
-- Other slices: only concatenated when exactly one element is non‑zero
+// Implement a concatenation method
+func concatTStreamForTest(items []*tStreamConcatItemForTest) (*tStreamConcatItemForTest, error) {
+ var s string
+ for _, item := range items {
+ s += item.s
+ }
-You can override defaults by registering custom concat functions via `compose.RegisterStreamChunkConcatFunc`.
+ return &tStreamConcatItemForTest{s: s}, nil
+}
-### Runtime Type Checks
+func Init() {
+ // Register to the global concatenation method
+ compose.RegisterStreamChunkConcatFunc(concatTStreamForTest)
+}
+```
-Graph verifies type alignment during `AddEdge("node1", "node2")` and at `Compile()` for rules above. When upstream outputs an interface and the downstream expects a concrete type, the final assignability is only known at runtime once the upstream concrete type is available; the framework performs runtime checks for that scenario.
+### Scenarios Where Type Alignment is Checked at Runtime
+
+Eino's Graph type alignment check is performed at `err = graph.AddEdge("node1", "node2")` to check whether the two node types match. This allows type mismatch errors to be discovered during `the graph building process` or `the Compile process`. This applies to rules ① ② ③ listed in [Eino: Orchestration Design Principles](/docs/eino/core_modules/chain_and_graph_orchestration/orchestration_design_principles).
+
+When the upstream node's output is an `interface`, if the downstream node type implements that `interface`, the upstream may be able to convert to the downstream type (type assertion), but this can only be known during `runtime`. The type check for this scenario is moved to runtime.
+
+The structure is shown below:
-## Opinionated Design Choices
+This scenario is suitable for cases where developers can handle upstream and downstream type alignment themselves, and can choose downstream execution nodes based on different types.
+
+## Design Choices with Clear Preferences
-### External Variables Are Read-Only
+### External Variables Read-Only Principle
-Data flows in Graph across Nodes, Branches, and Handlers via direct assignment, not deep copies. When inputs are reference types (struct pointers, maps, slices), mutating them inside Nodes/Branches/Handlers causes side effects and potential races. Treat external inputs as read-only; if mutation is needed, copy first. The same applies to streamed chunks in `StreamReader`.
+When data flows between Nodes, Branches, and Handlers in Eino's Graph, it is always variable assignment, not Copy. When Input is a reference type, such as Struct pointer, map, or slice, modifications to Input inside Node, Branch, or Handler will have side effects externally and may cause concurrency issues. Therefore, Eino follows the external variables read-only principle: Node, Branch, Handler should not modify Input internally. If modification is needed, copy it first.
-### Fan‑in and Merge
+This principle also applies to Chunks in StreamReader.
-Multiple upstreams can feed into one downstream. Define how to merge outputs:
+### Fan-in and Merge
-- Without custom merge functions, upstream actual types must be identical and be a `map`; keys must be disjoint. Non‑stream: merged into one map; stream: merged into one `StreamReader` with fair reading.
-- Use `WithOutputKey` to convert a node’s output into a map:
+**Fan-in**: Data from multiple upstreams flows into the downstream, together serving as the downstream's input. It is necessary to clearly define how multiple upstream outputs are **merged**.
+
+By default, first, the **actual types** of multiple upstream outputs must be the same and be a Map, and keys must not overlap. Second:
+
+- In non-streaming scenarios, after merging, it becomes one Map containing all key-value pairs from all upstreams.
+- In streaming scenarios, multiple upstream StreamReaders of the same type are merged into one StreamReader. The actual Recv effect is fair reading from multiple upstream StreamReaders.
+
+When AddNode, you can add the WithOutputKey Option to convert the node's output to a Map:
```go
+// This node's output will change from string to map[string]any,
+// and the map has only one element, key is your_output_key, value is the actual string output by the node
graph.AddLambdaNode("your_node_key", compose.InvokableLambda(func(ctx context.Context, input []*schema.Message) (str string, err error) {
// your logic
return
}), compose.WithOutputKey("your_output_key"))
```
-- Register custom merge:
+You can also register a Merge method to implement merge of any type:
```go
// eino/compose/values_merge.go
func RegisterValuesMergeFunc[T any](fn func([]T) (T, error))
```
-Workflow maps fields across nodes; upstream structs are converted to maps, so the same merge rules apply.
+Workflow can map multiple output fields from multiple upstreams to different fields of the downstream node. This is not a merge scenario, but point-to-point field mapping. In fact, eino workflow currently does not support "multiple upstream fields mapping to the same downstream field simultaneously".
### Streaming Handling
-- Auto concatenate: prefer user‑registered concat functions, then framework defaults (Message, Message array, string, map, struct and pointers)
-- Auto boxing: convert non‑stream `T` to `StreamReader[T]`
-- Auto merge: see Fan‑in above
-- Auto copy: duplicate streams where needed (fan‑out to multiple downstreams, callbacks)
+Eino believes that components should only need to implement the streaming paradigms that are real in business scenarios. For example, ChatModel doesn't need to implement Collect. Therefore, in orchestration scenarios, Eino automatically helps all nodes **complete missing streaming paradigms**.
+
+Running Graph in Invoke mode, all internal nodes run in Invoke paradigm. Running Graph in Stream, Collect, or Transform mode, all internal nodes run in Transform paradigm.
+
+**Auto Concatenate**: For scenarios where Stream chunks are concatenated into complete content, user-registered custom concatenation functions are used first, followed by framework-provided default behaviors, including Message, Message array, String, Map, and Struct and Struct pointers.
+
+**Auto Box**: For scenarios where non-streaming T needs to become StreamReader[T], the framework executes automatically.
+
+**Auto Merge**: See the "Fan-in and Merge" section above.
+
+**Auto Copy**: Automatic stream copying in scenarios that require it, including a stream fanning out to multiple downstream nodes, and a stream entering one or more callback handlers.
-All orchestration elements can sense/handle streams (branch, state handler, callback handler, passthrough, lambda, etc.). See [Streaming Essentials](/docs/eino/core_modules/chain_and_graph_orchestration/stream_programming_essentials).
+Finally, Eino requires all orchestration elements to be able to sense and handle streams. This includes branch, state handler, callback handler, passthrough, lambda, etc.
+
+For Eino's stream handling capabilities, see [Eino Stream Programming Essentials](/docs/eino/core_modules/chain_and_graph_orchestration/stream_programming_essentials).
### Global State
-- Provide `State` via `compose.WithGenLocalState` when creating a Graph; request‑scoped and readable/writable across steps
-- Use `StatePreHandler` and `StatePostHandler` to read/write state and optionally replace node input/output; match input types to node non‑stream types (and the streaming variants for stream types)
-- External handlers modify inputs/outputs outside nodes, preserving node statelessness
-- Internal state access: `compose.ProcessState[S any](ctx context.Context, handler func(context.Context, S) error)`
-- All state access is synchronized by the framework
+**State**: Pass the State creation method through `compose.WithGenLocalState` when NewGraph. This request-scoped global state can be read and written in various stages of a request.
+
+Eino recommends using `StatePreHandler` and `StatePostHandler`, with the functional positioning of:
+
+- StatePreHandler: Read and write State before each node execution, and replace the node's Input as needed. Input needs to align with the node's non-streaming input type.
+- StatePostHandler: Read and write State after each node execution, and replace the node's Output as needed. Input needs to align with the node's non-streaming output type.
+
+For streaming scenarios, use the corresponding `StreamStatePreHandler` and `StreamStatePostHandler`, with input needing to align with the node's streaming input and streaming output types respectively.
+
+These state handlers are located outside the node, affecting the node through modifications to Input or Output, thus ensuring the node's "state-independent" characteristic.
+
+If you need to read and write State inside a node, Eino provides the `ProcessState[S any](ctx context.Context, handler func(context.Context, S) error) error` function.
+
+The Eino framework will lock at all positions where State is read or written.
### Callback Injection
-Components may or may not implement callback aspects. If a component implements `Checker` with `IsCallbacksEnabled()==true`, the framework uses the component’s internal callbacks; otherwise it wraps external callbacks reporting only input/output. Graph always injects callbacks with `RunInfo` for itself. See [Callback Manual](/docs/eino/core_modules/chain_and_graph_orchestration/callback_manual).
+The Eino orchestration framework believes that components entering orchestration may or may not have Callback aspects embedded internally. This information is determined by whether the component implements the `Checker` interface and the return value of the `IsCallbacksEnabled` method in the interface.
+
+- When `IsCallbacksEnabled` returns true, the Eino orchestration framework uses the Callback aspects inside the component implementation.
+- Otherwise, it automatically wraps Callback aspects outside the component implementation, (only) reporting input and output.
+
+In either case, RunInfo will be automatically inferred.
+
+At the same time, for the Graph as a whole, Callback aspects will always be injected, with RunInfo being the Graph itself.
+
+For the complete description of Eino's Callback capabilities, see [Eino: Callback User Manual](/docs/eino/core_modules/chain_and_graph_orchestration/callback_manual).
### Option Distribution
-- Global by default — applies to all nodes, including nested graphs
-- Component‑type options — e.g., `AddChatModelOption` applies to all ChatModel nodes; Lambda with its own option type can be targeted similarly
-- Specific nodes — `DesignateNode(key ...string)`
-- Nested graphs or their nodes — `DesignateNodeWithPath(path ...*NodePath)`
+Eino supports various dimensions of Call Option distribution:
-See [CallOption Capabilities](/docs/eino/core_modules/chain_and_graph_orchestration/call_option_capabilities).
+- Global by default, i.e., distributed to all nodes, including nested internal graphs.
+- Can add Options for a specific component type, which are then distributed to all nodes of that type by default, such as AddChatModelOption. Lambda that defines its own Option type can also specify Options to itself this way.
+- Can specify any specific nodes using `DesignateNode(key ...string)`.
+- Can specify nested graphs at any depth, or any specific nodes within them, using `DesignateNodeWithPath(path ...*NodePath)`.
+
+For the complete description of Eino's Call Option capabilities, see [Eino: CallOption Capabilities and Specifications](/docs/eino/core_modules/chain_and_graph_orchestration/call_option_capabilities).
### Graph Nesting
-Compiled graphs (`Runnable`) are Lambda‑like; you can wrap them as a Lambda and nest into other graphs, or add subgraphs pre‑compile via `AddGraph`.
+The graph orchestration product `Runnable` has a very similar interface form to Lambda. Therefore, a compiled graph can be simply wrapped as a Lambda and nested into other graphs as a Lambda node.
+
+Another way is to directly nest Graph, Chain, Workflow, etc. into other graphs through AddGraph before compilation. The differences between the two approaches are:
-- Lambda wrapping adds an extra Lambda level in traces/callbacks
-- Lambda wrapping carries options via Lambda options, not `DesignateNodeWithPath`
-- Lambda wrapping requires pre‑compilation; `AddGraph` compiles the inner graph with the parent
+- With the Lambda approach, there will be an extra Lambda node level in the trace. Other Callback handler perspectives will also see an extra layer.
+- With the Lambda approach, CallOption needs to be received through Lambda's Option, and cannot use DesignateNodeWithPath.
+- With the Lambda approach, the internal graph needs to be compiled beforehand. With direct AddGraph, the internal graph is compiled together with the upper-level graph.
-### Internal Mechanics
+## Internal Mechanisms
-#### Execution Sequence
+### Execution Sequence
-Full streaming execution sequence for an InvokableLambda (string→int) with State handlers, InputKey/OutputKey and external callbacks:
+Taking an InvokableLambda (input is string, output is int) with StatePreHandler, StatePostHandler, InputKey, OutputKey, and no internal Callback aspects as an example, the complete streaming execution sequence in the graph is as follows:
-Workflow performs field mapping after `StatePostHandler` and stream copy, and before `StatePreHandler` via merge.
+In workflow scenarios, field mapping occurs at two positions:
-#### Execution Engines
+- After the node execution's StatePostHandler and "stream copy" steps, the fields needed by each downstream are extracted separately.
+- After the "merge" step before node execution and before StatePreHandler, the extracted upstream field values are converted to the current node's input.
-- `NodeTriggerMode == AnyPredecessor` → pregel engine (directed cyclic graph)
- - After current nodes run, all successors form a SuperStep and run together
- - Supports Branch and cycles; may need passthrough nodes to shape SuperSteps
+### Execution Engine
+
+When `NodeTriggerMode == AnyPredecessor`, the graph executes with the pregel engine, corresponding to a directed cyclic graph topology. The characteristics are:
+
+- All successor nodes of the currently executing one or more nodes form a SuperStep and execute together. At this point, these new nodes become the "current" nodes.
+- Supports Branch, supports cycles in the graph, but may require manually adding passthrough nodes to ensure the nodes in the SuperStep meet expectations, as shown below:
-Refactor with passthrough to meet expectations:
+In the above diagram, Node 4 and Node 5 are placed together for execution according to the rules, which is probably not as expected. It needs to be changed to:
-- `NodeTriggerMode == AllPredecessor` → DAG engine (directed acyclic graph)
- - Each node runs only after all predecessors complete
- - Cycles not supported; Branch is supported (unselected branch nodes are marked skipped at runtime)
- - SuperStep semantics apply; use `compose.WithEagerExecution()` to run ready nodes immediately. In v0.4.0+, AllPredecessor defaults to eager execution.
+When `NodeTriggerMode == AllPredecessor`, the graph executes with the dag engine, corresponding to a directed acyclic graph topology. The characteristics are:
+
+- Each node has definite predecessor nodes, and this node only has the condition to run after all predecessor nodes are complete.
+- Does not support cycles in the graph, because it would break the assumption that "each node has definite predecessor nodes".
+- Supports Branch. At runtime, nodes not selected by the Branch are marked as skipped, not affecting the AllPredecessor semantics.
+
+> 💡
+> After setting NodeTriggerMode = AllPredecessor, nodes will execute after all predecessors are ready, but not immediately - they still follow SuperStep semantics, running new runnable nodes after a batch of nodes has completed execution.
+>
+> If you pass compose.WithEagerExecution() during Compile, ready nodes will run immediately.
+>
+> In Eino v0.4.0 and later versions, setting NodeTriggerMode = AllPredecessor will enable EagerExecution by default.
-Summary: pregel is flexible but cognitively heavy; DAG is clear but constrained. In Eino, Chain uses pregel, Workflow uses DAG, Graph supports both selectable by users.
+In summary, pregel mode is flexible and powerful but has additional cognitive burden, while dag mode is clear and simple but has limited scenarios. In the Eino framework, Chain uses pregel mode, Workflow uses dag mode, and Graph supports both, allowing users to choose between pregel and dag.
diff --git a/content/en/docs/eino/core_modules/chain_and_graph_orchestration/stream_programming_essentials.md b/content/en/docs/eino/core_modules/chain_and_graph_orchestration/stream_programming_essentials.md
index be7d6a00160..fb3d8e4d5c1 100644
--- a/content/en/docs/eino/core_modules/chain_and_graph_orchestration/stream_programming_essentials.md
+++ b/content/en/docs/eino/core_modules/chain_and_graph_orchestration/stream_programming_essentials.md
@@ -1,6 +1,6 @@
---
Description: ""
-date: "2025-12-11"
+date: "2026-01-20"
lastmod: ""
tags: []
title: Eino Streaming Essentials
@@ -39,11 +39,13 @@ Key factors when orchestrating streaming graphs:
## Streaming at the Single-Component Level
-Eino is a “component-first” framework; components can be used independently. When defining component interfaces, streaming is guided by real business semantics.
+Eino is a "component-first" framework; components can be used independently. Do you need to consider streaming when defining component interfaces? The simple answer is no. The complex answer is "follow real business scenarios".
### Business Semantics of Components
-- ChatModel: besides `Invoke` (non-streaming), it naturally supports `Stream` (streaming). It therefore implements `Generate` and `Stream`, but not `Collect` or `Transform`:
+A typical component, such as ChatModel or Retriever, should define its interface according to actual business semantics. If it actually supports a certain streaming paradigm, implement that paradigm. If a certain streaming paradigm has no real business scenario, there's no need to implement it. For example:
+
+- ChatModel: besides `Invoke` (non-streaming), it naturally supports `Stream` (streaming). It therefore implements `Generate` and `Stream`, but `Collect` and `Transform` have no corresponding real business scenarios, so they are not implemented:
```go
type ChatModel interface {
@@ -54,7 +56,7 @@ type ChatModel interface {
}
```
-- Retriever: only `Invoke` has real use; the other paradigms don’t fit typical scenarios, so it exposes just `Retrieve`:
+- Retriever: besides `Invoke` (non-streaming), the other three streaming paradigms have no real business scenarios, so it only implements `Retrieve`:
```go
type Retriever interface {
@@ -62,7 +64,7 @@ type Retriever interface {
}
```
-### Which Paradigms Components Implement
+### Paradigms Supported by Components
| Component | Invoke | Stream | Collect | Transform |
| Tool | yes | yes | no | no |
-If the Tool is `StreamableTool` (output is `StreamReader[Message]`), then Tool → ChatModel may be streaming. However, ChatModel does not accept streaming input. Eino automatically bridges this by concatenating streams into non-streaming input:
+In the diagram above, if Tool is a StreamableTool (i.e., output is `StreamReader[Message]`), then Tool → ChatModel could have streaming output. However, ChatModel has no business scenario for accepting streaming input, nor does it have a corresponding interface. At this point, Eino framework automatically provides ChatModel with the ability to accept streaming input:
-Eino’s automatic `StreamReader[T] → T` conversion applies whenever a component expects `T` but upstream produces `StreamReader[T]`. You may need to provide a custom concat function for `T`.
+The "Concat message stream" above is a capability automatically provided by the Eino framework. Even if it's not a message but any arbitrary `T`, as long as certain conditions are met, Eino framework will automatically perform this `StreamReader[T]` to `T` conversion. The condition is: **In orchestration, when a component's upstream output is `StreamReader[T]`, but the component only provides a business interface with `T` as input, the framework will automatically concat `StreamReader[T]` into `T` before inputting it to the component.**
> 💡
-> The `StreamReader[T] → T` conversion may require a user-provided concat function. See [Orchestration Design Principles](/docs/eino/core_modules/chain_and_graph_orchestration/orchestration_design_principles) under “merge frames”.
+> The process of the framework automatically concatenating `StreamReader[T]` into `T` may require the user to provide a concat function. See [Eino: Orchestration Design Principles](/docs/eino/core_modules/chain_and_graph_orchestration/orchestration_design_principles) for the section on "merge frames".
-Conversely, consider another ReAct diagram:
+On the other hand, consider an opposite example. Still the ReAct Agent, this time with a more complete orchestration diagram:
-Here, `branch` reads the ChatModel’s output and decides whether to end or call a tool. Since `branch` can decide from the first frame, define it with `Collect` (streaming in, non-streaming out):
+In the diagram above, the branch receives the message output from the ChatModel and decides whether to end the agent's current run and output the message directly, or call the Tool and pass the result back to the ChatModel for cyclic processing, based on whether the message contains a tool call. Since this Branch can complete its logic judgment from the first frame of the message stream, we define this Branch with the `Collect` interface, i.e., streaming input, non-streaming output:
```go
compose.NewStreamGraphBranch(func(ctx context.Context, sr *schema.StreamReader[*schema.Message]) (endNode string, err error) {
@@ -123,32 +125,37 @@ compose.NewStreamGraphBranch(func(ctx context.Context, sr *schema.StreamReader[*
})
```
-If the agent is invoked via `Stream`, ChatModel outputs `StreamReader[Message]`, matching the branch’s input.
+ReactAgent has two interfaces, `Generate` and `Stream`, which implement the `Invoke` and `Stream` streaming programming paradigms respectively. When a ReactAgent is called via `Stream`, the ChatModel's output is `StreamReader[Message]`, so the Branch's input is `StreamReader[Message]`, which matches the function signature definition of this Branch condition, and can run without any conversion.
+
+However, when this ReactAgent is called via `Generate`, the ChatModel's output is `Message`, so the Branch's input would also be `Message`, which does not match the `StreamReader[Message]` function signature definition of the Branch Condition. At this point, the Eino framework will automatically box `Message` into `StreamReader[Message]` and pass it to the Branch, and this StreamReader will only have one frame.
-If the agent is invoked via `Generate`, ChatModel outputs `Message`. Eino automatically wraps `Message` into a single-frame `StreamReader[Message]` (pseudo-stream) to match the branch.
+> 💡
+> This kind of stream with only one frame is colloquially called a "pseudo-stream", because it doesn't bring the actual benefit of streaming, which is "low first-packet latency", but is simply boxed to meet the streaming input/output interface signature requirements.
-Summary: **When upstream outputs `T` but downstream expects `StreamReader[T]`, Eino wraps `T` into a single-frame `StreamReader[T]`.**
+In summary: **In orchestration, when a component's upstream output is `T`, but the component only provides a business interface with `StreamReader[T]` as input, the framework will automatically box `T` into a single-frame `StreamReader[T]` before inputting it to the component.**
### Streaming Paradigms of Orchestration Aids
-`Branch` is an orchestration-only aid, not a standalone component. Others include:
+The Branch mentioned above is not a component that can be used standalone, but an "orchestration aid" that only makes sense in orchestration scenarios. Similar "components" that are only meaningful in orchestration scenarios include:
| Element | Use Case | Invoke | Stream | Collect | Transform |
| Branch | Select a downstream node dynamically based on upstream output | yes | no | yes | no |
| StatePreHandler | Modify State/Input before entering a node. Streaming-friendly. | yes | no | no | yes |
| StatePostHandler | Modify State/Output after a node. Streaming-friendly. | yes | no | no | yes |
| Passthrough | Balance node counts across parallel branches by inserting passthroughs. Input equals output. | yes | no | no | yes |
| Lambda | Encapsulate custom business logic; implement the paradigm that matches your logic. | yes | yes | yes | yes |
| Branch | Dynamically select one from a set of downstream Nodes based on upstream output | yes | no | yes | no |
| StatePreHandler | Modify State and/or Input before entering a Node in Graph. Supports streaming. | yes | no | no | yes |
| StatePostHandler | Modify State and/or Output after a Node completes in Graph. Supports streaming. | yes | no | no | yes |
| Passthrough | In parallel scenarios, to balance the number of Nodes in each parallel branch, Passthrough nodes can be added to branches with fewer Nodes. Passthrough node's input and output are the same, following the upstream node's output or downstream node's input (expected to be the same). | yes | no | no | yes |
| Lambda | Encapsulate business logic not defined by official components. Choose the corresponding streaming paradigm based on which paradigm the business logic is. | yes | yes | yes | yes |
-- Else if `Collect` exists, wrap `Collect` as `Invoke` by boxing non-streaming input into a single-frame stream.
+- Otherwise, if the component implements `Collect`, wrap `Collect` as `Invoke`, i.e., convert non-streaming input to single-frame stream.
-- Else use `Transform`, wrapping it as `Invoke` by boxing input into a single-frame stream and concatenating the output stream.
+- If neither is implemented, the component must implement `Transform`, wrap `Transform` as `Invoke`, i.e., convert input to single-frame stream and concat output.
-When you call via **Stream / Collect / Transform**, all internal components run in the `Transform` paradigm. If a component does not implement `Transform`, Eino wraps using the first available option:
+If the user calls the Graph via **Stream/Collect/Transform**, all internal components are called with the `Transform` paradigm. If a component does not implement the `Transform` paradigm, the Eino framework automatically wraps the `Transform` calling paradigm based on the streaming paradigms the component has implemented, with the following priority:
-- If `Stream` exists, wrap `Stream` as `Transform` by concatenating the input stream.
+- If the component implements `Stream`, wrap `Stream` as `Transform`, i.e., automatically concat the input stream.
-- Else if `Collect` exists, wrap `Collect` as `Transform` by boxing non-streaming output into a single-frame stream.
+- Otherwise, if the component implements `Collect`, wrap `Collect` as `Transform`, i.e., convert non-streaming output to single-frame stream.
-- Else wrap `Invoke` as `Transform` by concatenating input streams and boxing outputs into single-frame streams.
+- If neither is implemented, the component must implement `Invoke`, wrap `Invoke` as `Transform`, i.e., concat input stream and convert output to single-frame stream.
-In summary, Eino’s automatic conversions between `T` and `Stream[T]` are:
+Combining the various cases enumerated above, Eino framework's automatic conversion between `T` and `Stream[T]` can be summarized as:
-- **T → Stream[T]**: box `T` into a single-frame stream (pseudo-stream).
-- **Stream[T] → T**: concat the stream into a complete `T`. For non-single-frame streams, you may need to provide a concat function.
+- **T -> Stream[T]: Box the complete `T` into a single-frame `Stream[T]`. Non-streaming becomes pseudo-streaming.**
+- **Stream[T] -> T: Concat `Stream[T]` into a complete `T`. When `Stream[T]` is not a single-frame stream, a concat method for `T` may need to be provided.**
-You might wonder why a graph-level `Invoke` enforces `Invoke` internally, and `Stream/Collect/Transform` enforces `Transform` internally. Consider these counterexamples:
+After seeing the implementation principles above, you might have questions: why does calling `Invoke` on a graph require all internal components to be called with `Invoke`? And why does calling `Stream/Collect/Transform` on a graph require all internal components to be called with `Transform`? After all, some counterexamples can be given:
-- Two components A and B composed into a Chain, called via `Invoke`. Suppose A implements `Stream`, B implements `Collect`.
- - Choice 1: A runs as `Stream`, B runs as `Collect`. The overall Chain remains `Invoke` to the caller while preserving true streaming semantics inside (no concat; A’s output stream feeds B in real time).
- - Current Eino behavior: both A and B run as `Invoke`. A’s output stream is concatenated to a full value; B’s input is boxed into a pseudo-stream. True streaming semantics inside are lost.
-- Two components A and B composed into a Chain, called via `Collect`. Suppose A implements `Transform` and `Collect`, B implements `Invoke`.
- - Choice 1: A runs as `Collect`, B runs as `Invoke`. The overall remains `Collect` with no automatic conversions or boxing.
- - Current Eino behavior: both A and B run as `Transform`. Since B only implements `Invoke`, its input may require concat from a real stream, potentially needing a user-provided concat function — which could have been avoided.
+- Components A and B are orchestrated into a Chain, called with `Invoke`. A's business interface implements `Stream`, B's business interface implements `Collect`. At this point, there are two choices for the calling paradigm of internal components in the graph:
+ - A is called with `Stream`, B is called with `Collect`, the overall Chain still has `Invoke` semantics, while preserving true streaming internal semantics. That is, A's output stream doesn't need to be concatenated and can be input to B in real-time.
+ - Current Eino implementation: A and B are both called with `Invoke`, requiring A's output stream to be concatenated and B's input to be made into a pseudo-stream. True streaming internal semantics are lost.
+- Components A and B are orchestrated into a Chain, called with `Collect`. A implements `Transform` and `Collect`, B implements `Invoke`. Two choices:
+ - A is called with `Collect`, B is called with `Invoke`: the overall still has `Collect` semantics, no automatic conversion or boxing operations are needed by the framework.
+ - Current Eino implementation: A and B are both called with `Transform`. Since A's business interface implements `Transform`, both A's output and B's input could be true streaming, but B's business interface only implements `Invoke`. According to the analysis above, B's input will need to be concatenated from true streaming to non-streaming. At this point, the user needs to additionally provide a concat function for B's input, which could have been avoided.
-Generalizing across arbitrary graphs, it’s difficult to define a universal rule that is always better and remains clear. Influencing factors include which paradigms components implement, aiming to maximize true streaming, and whether concat functions exist. For complex graphs, the number of factors grows quickly. Even if a more optimal universal rule exists, it would be hard to explain and use without exceeding the benefit it provides. Eino’s design therefore favors clarity and predictability.
+The two examples above can both find a clear, different from Eino's convention, but better streaming call path. However, when generalized to arbitrary orchestration scenarios, it's difficult to find a clearly defined, different from Eino's convention, yet always better universal rule. For example, A->B->C, called with `Collect` semantics, should `Collect` happen at A->B or B->C? Potential factors include the business interfaces specifically implemented by A, B, C, possibly the judgment of "use true streaming as much as possible", and maybe which parameters implement `Concat` and which don't. If it's a more complex Graph, the factors to consider will increase rapidly. In this situation, even if the framework can define a clear, better universal rule, it would be hard to explain clearly, and the understanding and usage cost would be high, likely already exceeding the actual benefit brought by this new rule.
-By design:
+In summary, we can say that the runtime paradigms of components inside Eino orchestration artifacts are **By Design**, clearly as follows:
-- **Invoke outside → Invoke inside**: no streaming inside.
-- **Stream/Collect/Transform outside → Transform inside**: streaming inside; `Stream[T] → T` may require a concat function.
+- **Called with `Invoke` overall, all internal components are called with `Invoke`, there is no streaming process.**
+- **Called with `Stream/Collect/Transform` overall, all internal components are called with `Transform`. When `Stream[T] -> T` concat process occurs, a concat function for `T` may need to be additionally provided.**
diff --git a/content/en/docs/eino/core_modules/chain_and_graph_orchestration/workflow_orchestration_framework.md b/content/en/docs/eino/core_modules/chain_and_graph_orchestration/workflow_orchestration_framework.md
index 7565c2a9655..ea063b20fa0 100644
--- a/content/en/docs/eino/core_modules/chain_and_graph_orchestration/workflow_orchestration_framework.md
+++ b/content/en/docs/eino/core_modules/chain_and_graph_orchestration/workflow_orchestration_framework.md
@@ -1,6 +1,6 @@
---
Description: ""
-date: "2025-12-09"
+date: "2026-01-20"
lastmod: ""
tags: []
title: 'Eino: Workflow Orchestration Framework'
@@ -37,24 +37,44 @@ Key traits:
### Flexible Input/Output Types
-Easily orchestrate existing business functions with unique `struct` inputs/outputs and map fields directly between them, preserving original signatures without forcing common types or `map[string]any` everywhere.
+For example, you need to orchestrate two lambda nodes containing two "existing business functions f1, f2" with specific struct inputs/outputs suited to business scenarios, each different:
+When orchestrating with Workflow, map f1's output field F1 directly to f2's input field F3, while preserving the original function signatures of f1 and f2. The effect achieved is: **each node's input/output is "determined by the business scenario", without needing to consider "who provides my input and who uses my output"**.
+
+When orchestrating with Graph, due to the "type alignment" requirement, if f1 → f2, then f1's output type and f2's input type need to align. You must choose one of two options:
+
+- Define a new common struct, and change both f1's output type and f2's input type to this common struct. This has costs and may intrude on business logic.
+- Change both f1's output type and f2's input type to map. This loses the strong typing alignment characteristic.
+
### Separate Control and Data Flow
+Consider the following scenario:
+
-- Dashed lines denote data-only edges; no execution dependency (A’s completion doesn’t gate D’s start).
-- Bold arrows denote control-only edges; no data transfer (D’s completion gates E’s start but E doesn’t read D’s outputs).
-- Other edges combine control and data.
+Node D references certain output fields from A, B, and C simultaneously. The dashed line from A to D is purely "data flow", not carrying "control" information - meaning whether A completes execution does not determine whether D starts execution.
+
+The bold arrow from node D to E represents that node E does not reference any output from node D - it is purely "control flow", not carrying "data". Whether D completes execution determines whether E starts execution, but D's output does not affect E's input.
+
+Other lines in the diagram combine control flow and data flow.
-Data transfers require the existence of a control path; a node can only read from predecessors.
+Note that data flow can only be transmitted when a control flow path exists. For example, the A→D data flow depends on the existence of A→branch→B→D or A→branch→C→D control flow. Data flow can only reference outputs from predecessor nodes.
-Example: cross-node data passing in Workflow vs Graph/Chain. In Workflow, a ChatTemplate can take exactly `{"prompt": ..., "context": ...}` from START and a Retriever; in Graph/Chain this either requires heavy map wrapping or state usage.
+For example, this "cross-node" specific data passing scenario:
+In the diagram above, the chat template node's input can be very explicit:
+
+`map[string]any{"prompt": "prompt from START", "context": "retrieved context"}`
+
+In contrast, if using Graph or Chain API, you must choose one of two options:
+
+- Use OutputKey to convert node output types (can't add to START node, so need an extra passthrough node), and the ChatTemplate node's input will include the full output of START and retriever (rather than just the specific fields actually needed).
+- Put START node's prompt in state, and ChatTemplate reads from state. This introduces additional state.
+
## Using Workflow
### Simplest Workflow
@@ -69,31 +89,70 @@ START → node → END
// (by using AddInput without field mappings),
// this simple workflow is equivalent to a Graph: START -> lambda -> END.
func main() {
+ // create a Workflow, just like creating a Graph
wf := compose.NewWorkflow[int, string]()
- wf.AddLambdaNode("lambda", compose.InvokableLambda(func(ctx context.Context, in int) (string, error) {
- return strconv.Itoa(in), nil
- })).AddInput(compose.START)
- wf.End().AddInput("lambda")
+
+ // add a lambda node to the Workflow, just like adding the lambda to a Graph
+ wf.AddLambdaNode("lambda", compose.InvokableLambda(
+ func(ctx context.Context, in int) (string, error) {
+ return strconv.Itoa(in), nil
+ })).
+ // add an input to this lambda node from START.
+ // this means mapping all output of START to the input of the lambda.
+ // the effect of AddInput is to set both a control dependency
+ // and a data dependency.
+ AddInput(compose.START)
+
+ // obtain the compose.END of the workflow for method chaining
+ wf.End().
+ // add an input to compose.END,
+ // which means 'using ALL output of lambda node as output of END'.
+ AddInput("lambda")
+
+ // compile the Workflow, just like compiling a Graph
run, err := wf.Compile(context.Background())
- if err != nil { logs.Errorf("workflow compile error: %v", err); return }
+ if err != nil {
+ logs.Errorf("workflow compile error: %v", err)
+ return
+ }
+
+ // invoke the Workflow, just like invoking a Graph
result, err := run.Invoke(context.Background(), 1)
- if err != nil { logs.Errorf("workflow run err: %v", err); return }
+ if err != nil {
+ logs.Errorf("workflow run err: %v", err)
+ return
+ }
+
logs.Infof("%v", result)
}
```
+[Eino example link](https://github.com/cloudwego/eino-examples/blob/main/compose/workflow/1_simple/main.go)
+
Core APIs:
-- `NewWorkflow[I, O](...)` — same as `NewGraph`
-- `AddXXXNode(key, comp, ...) *WorkflowNode` — same node types as Graph; returns `WorkflowNode` for chaining
-- `(*WorkflowNode).AddInput(fromKey string, inputs ...*FieldMapping) *WorkflowNode` — add field mappings
-- `Compile(ctx, ...) (Runnable[I, O], error)` — same signature as Graph
+- `func NewWorkflow[I, O any](opts ...NewGraphOption) *Workflow[I, O]`
+ - Creates a new Workflow.
+ - Signature is identical to `NewGraph`.
+- `func (wf *Workflow[I, O]) AddChatModelNode(key string, chatModel model.BaseChatModel, opts ...GraphAddNodeOpt) *WorkflowNode`
+ - Adds a new node to the Workflow.
+ - Supported node types are identical to Graph.
+ - Unlike Graph's AddXXXNode which returns an error immediately, Workflow defers error handling to the final Compile step.
+ - AddXXXNode returns a WorkflowNode, allowing subsequent field mapping operations via method chaining.
+- `func (n *WorkflowNode) AddInput(fromNodeKey string, inputs ...*FieldMapping) *WorkflowNode`
+ - Adds input field mappings to a WorkflowNode.
+ - Returns WorkflowNode for continued method chaining.
+- `(wf *Workflow[I, O]) Compile(ctx context.Context, opts ...GraphCompileOption) (Runnable[I, O], error)`
+ - Compiles a Workflow.
+ - Signature is identical to compiling a Graph.
### Field Mapping
START (struct input) → [parallel lambda c1, c2] → END (map output).
-We demonstrate counting occurrences of a substring in two different fields. The workflow input is an Eino `Message` plus a `SubStr`; `c1` counts occurrences in `Content`, `c2` counts occurrences in `ReasoningContent`. The two lambdas run in parallel and map their results to END:
+We demonstrate counting occurrences of a substring in two different fields. The workflow input is an Eino `Message` plus a `SubStr`; `c1` counts occurrences in `Content`, `c2` counts occurrences in `ReasoningContent`. The two lambdas run in parallel and map their results to END.
+
+In the diagram below, the workflow's overall input is a message struct, both `c1` and `c2` lambdas have counter struct inputs, both output int, and the workflow's overall output is `map[string]any`:
@@ -165,7 +224,7 @@ func main() {
}
```
-Eino example link: https://github.com/cloudwego/eino-examples/blob/main/compose/workflow/2_field_mapping/main.go
+[Eino example link](https://github.com/cloudwego/eino-examples/blob/main/compose/workflow/2_field_mapping/main.go)
The `AddInput` method accepts 0–n field mappings and can be called multiple times. This means:
@@ -183,13 +242,13 @@ The `AddInput` method accepts 0–n field mappings and can be called multiple ti
## Advanced Features
-### Data-only Edges
+### Data-only Edges (No Control Flow)
-Imagine: START → adder → multiplier → END. The multiplier consumes one field from START and the adder’s result:
+Imagine a simple scenario: START → adder node → multiplier node → END. The "multiplier node" multiplies one field from START with the result from the adder node:
-Use `AddInputWithOptions(fromNode, fieldMappings, WithNoDirectDependency)` to declare pure data dependencies:
+In the diagram above, the multiplier node executes after the adder node, meaning the "multiplier node" is controlled by the "adder node". However, the START node does not directly control the "multiplier node"; it only passes data to it. In code, use `AddInputWithOptions(fromNode, fieldMappings, WithNoDirectDependency)` to specify a pure data flow:
```go
func main() {
@@ -248,9 +307,9 @@ func main() {
}
```
-Eino examples link: https://github.com/cloudwego/eino-examples/blob/main/compose/workflow/3_data_only/main.go
+[Eino examples link](https://github.com/cloudwego/eino-examples/blob/main/compose/workflow/3_data_only/main.go)
-New API:
+New API introduced in this example:
```go
func (n *WorkflowNode) AddInputWithOptions(fromNodeKey string, inputs []*FieldMapping, opts ...WorkflowAddInputOpt) *WorkflowNode {
@@ -258,13 +317,29 @@ func (n *WorkflowNode) AddInputWithOptions(fromNodeKey string, inputs []*FieldMa
}
```
-### Control-only Edges
+And the new Option:
-Consider a “sequential bidding, prices kept confidential” scenario: START → bidder1 → branch → bidder2 → END.
+```go
+func WithNoDirectDependency() WorkflowAddInputOpt {
+ return func(opt *workflowAddInputOpts) {
+ opt.noDirectDependency = true
+ }
+}
+```
+
+Combined, these can add pure "data dependency relationships" to nodes.
+
+### Control-only Edges (No Data Flow)
+
+Imagine a "sequential bidding with confidential prices" scenario: START → bidder1 → threshold check → bidder2 → END:
-Bold lines are control-only edges. After bidder1 bids, we announce completion without passing the bid amount. Use `AddDependency(fromNode)` to declare control without data:
+In the diagram above, regular lines are "control + data", dashed lines are "data only", and bold lines are "control only". The logic is: input an initial price, bidder1 makes bid1, a branch checks if it's high enough - if so, end directly; otherwise, pass the initial price to bidder2 for bid2, and finally aggregate both bids for output.
+
+After bidder1 bids, an announcement is made: "bidder completed bidding". Note that bidder1→announcer is a bold solid line, "control only", because the amount must be kept confidential when announcing!
+
+The two bold lines from the branch are both "control only" because neither bidder2 nor END depends on data from the branch. In code, use `AddDependency(fromNode)` to specify pure control flow:
```go
func main() {
@@ -327,38 +402,46 @@ func main() {
}
```
-Eino examples link: https://github.com/cloudwego/eino-examples/blob/main/compose/workflow/4_control_only_branch/main.go
+[Eino examples link](https://github.com/cloudwego/eino-examples/blob/main/compose/workflow/4_control_only_branch/main.go)
-New API:
+New API introduced in this example:
```go
func (n *WorkflowNode) AddDependency(fromNodeKey string) *WorkflowNode {
- return n.addDependencyRelation(fromNodeKey, nil, &workflowAddInputOpts{dependencyWithoutInput: true})
+ return n.addDependencyRelation(fromNodeKey, nil, &workflowAddInputOpts{dependencyWithoutInput: _true_})
}
```
+You can use `AddDependency` to specify pure "control dependency relationships" for nodes.
+
### Branch
-Add branches similarly to Graph, with Workflow branches carrying control only; data for branch targets is mapped via `AddInput*`:
+In the example above, we added a branch in almost the same way as with the Graph API:
```go
-wf.AddBranch("b1", compose.NewGraphBranch(func(ctx context.Context, in float64) (string, error) {
- if in > 5.0 { return compose.END, nil }
- return "b2", nil
-}, map[string]bool{compose.END: true, "b2": true}))
+// add a branch just like adding branch in Graph.
+ wf.AddBranch("b1", compose.NewGraphBranch(func(ctx context.Context, in float64) (string, error) {
+ if in > 5.0 {
+ return compose.END, nil
+ }
+ return "b2", nil
+ }, map[string]bool{compose.END: true, "b2": true}))
```
-Branch semantics in Workflow (AllPredecessor mode) mirror Graph:
+Branch semantics are the same as Graph's AllPredecessor mode:
+
+- There is exactly one 'fromNode', meaning a branch can only have one predecessor control node.
+- Can be single-select (`NewGraphBranch`) or multi-select (`NewGraphMultiBranch`).
+- Selected branches can execute. Unselected branches are marked as skip.
+- A node can only execute when all incoming edges are complete (success or skip), and at least one edge succeeded. (Like END in the example above)
+- If all incoming edges of a node are skip, all outgoing edges of that node are automatically marked as skip.
-- Exactly one `fromNode` per branch
-- Single-select (`NewGraphBranch`) or multi-select (`NewGraphMultiBranch`)
-- Selected targets execute; unselected targets are marked skipped
-- A node executes only when all incoming edges finish (success or skip) and at least one succeeded
-- If all incoming edges are skip, all outgoing edges are auto-marked skip
+Additionally, there is one key difference between workflow branch and graph branch:
-Workflow branches differ from Graph: Workflow branches are control-only; branch targets must declare their input mappings via `AddInput*`.
+- Graph branch is always "control and data combined"; the downstream node's input of a branch is always the output of the branch's fromNode.
+- Workflow branch is always "control only"; the downstream node's input is specified via `AddInputWithOptions`.
-API:
+Related API:
```go
func (wf *Workflow[I, O]) AddBranch(fromNodeKey string, branch *GraphBranch) *WorkflowBranch {
@@ -372,12 +455,16 @@ func (wf *Workflow[I, O]) AddBranch(fromNodeKey string, branch *GraphBranch) *Wo
}
```
+Signature is almost identical to `Graph.AddBranch`, allowing you to add a branch to the workflow.
+
### Static Values
-Inject constants into node inputs via `SetStaticValue(fieldPath, value)`.
+Let's modify the "bidding" example above by giving bidder1 and bidder2 each a static "budget" configuration:
+budget1 and budget2 will be injected into bidder1 and bidder2's inputs as "static values" respectively. Use the `SetStaticValue` method to configure static values for workflow nodes:
+
```go
func main() {
type bidInput struct {
@@ -435,9 +522,9 @@ func main() {
}
```
-Eino examples link: https://github.com/cloudwego/eino-examples/blob/main/compose/workflow/5_static_values/main.go
+[Eino examples link](https://github.com/cloudwego/eino-examples/blob/main/compose/workflow/5_static_values/main.go)
-API:
+Related API:
```go
func (n *WorkflowNode) SetStaticValue(path FieldPath, value any) *WorkflowNode {
@@ -446,18 +533,22 @@ func (n *WorkflowNode) SetStaticValue(path FieldPath, value any) *WorkflowNode {
}
```
+Use this method to set static values on specified fields of Workflow nodes.
+
### Streaming Effects
-Return to the “character counting” example, but now the workflow input is a stream of messages and the counting function returns a stream of counts:
+Returning to the previous "character counting" example, if our workflow's input is no longer a single message but a message stream, and our counting function can count each message chunk in the stream separately and return a "count stream":
-Changes:
+We make some modifications to the previous example:
+
+- Change `InvokableLambda` to `TransformableLambda`, so it can consume streams and produce streams.
+- Change the `SubStr` in the input to a static value, injected into c1 and c2.
+- Change the Workflow's overall input to `*schema.Message`.
+- Call the workflow using Transform, passing a stream containing 2 `*schema.Message`.
-- Use `TransformableLambda` to consume and produce streams
-- Make `SubStr` a static value injected into both `c1` and `c2`
-- Workflow input type becomes `*schema.Message`
-- Call the workflow via `Transform` with a stream of two messages
+The completed code:
```go
// demonstrates the stream field mapping ability of eino workflow.
@@ -555,10 +646,10 @@ func main() {
return
}
- logs.Infof("%v", chunk)
+ logs.Infof("%v", chunk)
- contentCount += chunk["content_count"]
- reasoningCount += chunk["reasoning_content_count"]
+ contentCount += chunk["content_count"]
+ reasoningCount += chunk["reasoning_content_count"]
}
logs.Infof("content count: %d", contentCount)
@@ -566,48 +657,50 @@ func main() {
}
```
-Eino examples link: https://github.com/cloudwego/eino-examples/blob/main/compose/workflow/6_stream_field_map/main.go
+[Eino examples link](https://github.com/cloudwego/eino-examples/blob/main/compose/workflow/6_stream_field_map/main.go)
-Characteristics of streaming in Workflow:
+Based on the example above, we summarize some characteristics of workflow streaming:
-- Still 100% Eino stream: invoke/stream/collect/transform auto converted, copied, concatenated and merged by the framework
-- Field mappings need no special stream handling; Eino performs mapping over streams transparently
-- Static values need no special stream handling; they are injected into the input stream and may arrive after the first chunk
+- Still 100% Eino stream: four paradigms (invoke, stream, collect, transform), automatically converted, copied, concatenated, and merged by the Eino framework.
+- Field mapping configuration doesn't require special handling for streams: regardless of whether the actual input/output is a stream, the `AddInput` syntax is the same. The Eino framework handles stream-based mapping.
+- Static values don't require special handling for streams: even if the actual input is a stream, you can use `SetStaticValue` the same way. The Eino framework will place static values in the input stream, but not necessarily in the first chunk read.
### Field Mapping Scenarios
#### Type Alignment
-- Identical types: passes compile and aligns at runtime
-- Different types but upstream assignable to downstream (e.g., upstream concrete type, downstream `any`): passes compile and aligns at runtime
-- Upstream not assignable to downstream (e.g., upstream `int`, downstream `string`): compile error
-- Upstream may be assignable only at runtime (e.g., upstream `any`, downstream `int`): compile defers; runtime checks actual upstream type and errors if not assignable
+Workflow follows the same type alignment rules as Graph, except the alignment granularity changes from complete input/output alignment to alignment between mapped field pairs. Specifically:
+
+- Identical types: passes compile-time validation and will definitely align.
+- Different types but upstream can be assigned to downstream (e.g., upstream is a concrete type, downstream is `any`): passes compile-time validation and will definitely align.
+- Upstream cannot be assigned to downstream (e.g., upstream is `int`, downstream is `string`): compile error.
+- Upstream may be assignable to downstream (e.g., upstream is `any`, downstream is `int`): cannot be determined at compile time, deferred to runtime. At runtime, the actual upstream type is extracted and checked. If upstream cannot be assigned to downstream, an error is thrown.
#### Merge Scenarios
-Merging applies when a node’s input is mapped from multiple `FieldMapping`s:
+Merge refers to situations where a node's input is mapped from multiple `FieldMapping`s:
-- Map to multiple different fields: supported
-- Map to the same single field: not supported
-- Map whole input along with field mappings: conflict, not supported
+- Mapping to multiple different fields: supported
+- Mapping to the same single field: not supported
+- Mapping to the whole input while also having mappings to specific fields: conflict, not supported
#### Nested `map[string]any`
-For mappings like `ToFieldPath([]string{"a","b"})` where the target input type is `map[string]any`, the framework ensures nested maps are created:
+For example, this mapping: `ToFieldPath([]string{"a","b"})`, where the target node's input type is `map[string]any`, the mapping order is:
-1. Level "a": `map[string]any{"a": nil}`
-2. Level "b": `map[string]any{"a": map[string]any{"b": x}}`
+1. First level "a": result is `map[string]any{"a": nil}`
+2. Second level "b": result is `map[string]any{"a": map[string]any{"b": x}}`
-At the second level, Eino replaces `any` with the actual `map[string]any` as needed.
+As you can see, at the second level, the Eino framework automatically replaces `any` with the actual `map[string]any`.
#### CustomExtractor
-When standard field mapping semantics cannot express the intent (e.g., source is `[]int` and you need the first element), use `WithCustomExtractor`:
+In some scenarios, standard field mapping semantics cannot support the requirement. For example, if upstream is `[]int` and you want to extract the first element to map to downstream, use `WithCustomExtractor`:
```go
t.Run("custom extract from array element", func(t *testing.T) {
- wf := compose.NewWorkflow[[]int, map[string]int]()
- wf.End().AddInput(compose.START, compose.ToField("a", compose.WithCustomExtractor(func(input any) (any, error) {
+ wf := NewWorkflow[[]int, map[string]int]()
+ wf.End().AddInput(_START_, ToField("a", WithCustomExtractor(func(input any) (any, error) {
return input.([]int)[0], nil
})))
r, err := wf.Compile(context.Background())
@@ -618,18 +711,20 @@ t.Run("custom extract from array element", func(t *testing.T) {
})
```
-With `WithCustomExtractor`, compile-time type alignment checks cannot be performed and are deferred to runtime.
+When using `WithCustomExtractor`, all compile-time type alignment checks cannot be performed and can only be deferred to runtime validation.
### Constraints
-- Map key restrictions: only `string`, or string aliases convertible to `string`
-- Unsupported compile options:
- - `WithNodeTriggerMode` (fixed `AllPredecessor`)
- - `WithMaxRunSteps` (no cycles)
-- If the mapping source is a map key, the key must exist. For streams, existence across chunks cannot be verified ahead of time.
-- For struct fields as mapping sources or targets, fields must be exported (reflection is used internally).
-- Nil sources are generally supported; errors when the target cannot be nil (e.g., basic types).
+- Map key restrictions: only `string`, or string alias (types that can be converted to `string`).
+- Unsupported CompileOptions:
+ - `WithNodeTriggerMode`, because it's fixed to `AllPredecessor`.
+ - `WithMaxRunSteps`, because there are no cycles.
+- If the mapping source is a Map Key, the Map must contain this key. However, if the mapping source is a Stream, Eino cannot determine whether this key appears at least once across all frames in the stream, so validation cannot be performed for Streams.
+- If the mapping source field or target field belongs to a struct, these fields must be exported, because reflection is used internally.
+- Nil mapping source: generally supported, only errors when the mapping target cannot be nil, such as basic types (int, etc.).
## Real-world Usage
-Coze‑Studio’s open source workflow engine is built on Eino Workflow. See: https://github.com/coze-dev/coze-studio/wiki/11.-%E6%96%B0%E5%A2%9E%E5%B7%A5%E4%BD%9C%E6%B5%81%E8%8A%82%E7%82%B9%E7%B1%BB%E5%9E%8B%EF%BC%88%E5%90%8E%E7%AB%AF%EF%BC%89
+### Coze-Studio Workflow
+
+[Coze-Studio](https://github.com/coze-dev/coze-studio) open source version's workflow engine is built on the Eino Workflow orchestration framework. See: [11. Adding New Workflow Node Types (Backend)](https://github.com/coze-dev/coze-studio/wiki/11.-%E6%96%B0%E5%A2%9E%E5%B7%A5%E4%BD%9C%E6%B5%81%E8%8A%82%E7%82%B9%E7%B1%BB%E5%9E%8B%EF%BC%88%E5%90%8E%E7%AB%AF%EF%BC%89)
diff --git a/content/en/docs/eino/core_modules/components/_index.md b/content/en/docs/eino/core_modules/components/_index.md
index 4e93fe49d5b..4f76a3199ca 100644
--- a/content/en/docs/eino/core_modules/components/_index.md
+++ b/content/en/docs/eino/core_modules/components/_index.md
@@ -1,72 +1,75 @@
---
Description: ""
-date: "2025-07-21"
+date: "2026-01-20"
lastmod: ""
tags: []
title: 'Eino: Components'
weight: 1
---
-LLM application development differs from traditional app development primarily due to two core capabilities:
+The most significant difference between LLM application development and traditional application development lies in the two core capabilities of large language models:
-- **Semantic text processing**: understanding and generating human language, handling relationships within unstructured content.
-- **Intelligent decision-making**: reasoning from context and making appropriate action decisions.
+- **Semantic text processing**: the ability to understand and generate human language, handling semantic relationships in unstructured content.
+- **Intelligent decision-making**: the ability to reason and make judgments based on context, and make corresponding behavioral decisions.
-These capabilities lead to three major application patterns:
+These two core capabilities have given rise to three main application patterns:
-1. **Direct conversation**: process user input and produce responses.
-2. **Knowledge processing**: semantically process, store, and retrieve textual documents.
-3. **Tool calling**: reason from context and call appropriate tools.
+1. **Direct conversation mode**: process user input and generate corresponding responses.
+2. **Knowledge processing mode**: semantically process, store, and retrieve textual documents.
+3. **Tool calling mode**: make decisions based on context and call corresponding tools.
-These patterns summarize common LLM app scenarios and provide a basis for abstraction and standardization. Eino abstracts these into reusable **components**.
+These patterns highly summarize the main scenarios of current LLM applications and provide a foundation for abstraction and standardization. Based on this, Eino abstracts these common capabilities into reusable "Components".
-Mapping components to patterns:
+The relationship between component abstractions and these patterns is as follows:
-**Conversation components:**
+**Conversation processing components:**
-1. Template and parameter preparation for LLM interaction: `ChatTemplate`
+1. Component abstractions for templated processing and LLM interaction parameters: `ChatTemplate`, `AgenticChatTemplate`
- - See [Eino: ChatTemplate Guide](/docs/eino/core_modules/components/chat_template_guide)
+ > See [Eino: ChatTemplate Guide](/docs/eino/core_modules/components/chat_template_guide), [Eino: AgenticChatTemplate Guide [Beta]](/docs/eino/core_modules/components/agentic_chat_template_guide)
+ >
+2. Component abstractions for direct LLM interaction: `ChatModel`, `AgenticModel`
-2. Direct LLM interaction: `ChatModel`
+ > See [Eino: ChatModel Guide](/docs/eino/core_modules/components/chat_model_guide), [Eino: AgenticModel Guide [Beta]](/docs/eino/core_modules/components/agentic_chat_model_guide)
+ >
- - See [Eino: ChatModel Guide](/docs/eino/core_modules/components/chat_model_guide)
+**Text semantic processing components:**
-**Text semantics components:**
+1. Component abstractions for acquiring and processing text documents: `Document.Loader`, `Document.Transformer`
-1. Document acquisition and processing: `Document.Loader`, `Document.Transformer`
+ > See [Eino: Document Loader Guide](/docs/eino/core_modules/components/document_loader_guide), [Eino: Document Transformer Guide](/docs/eino/core_modules/components/document_transformer_guide)
+ >
+2. Component abstraction for semantic processing of text documents: `Embedding`
- - See [Document Loader Guide](/docs/eino/core_modules/components/document_loader_guide) and [Document Transformer Guide](/docs/eino/core_modules/components/document_transformer_guide)
+ > See [Eino: Embedding Guide](/docs/eino/core_modules/components/embedding_guide)
+ >
+3. Component abstraction for storing data indexes after embedding: `Indexer`
-2. Semantic embedding of documents: `Embedding`
+ > See [Eino: Indexer Guide](/docs/eino/core_modules/components/indexer_guide)
+ >
+4. Component abstraction for indexing and retrieving semantically related text documents: `Retriever`
- - See [Embedding Guide](/docs/eino/core_modules/components/embedding_guide)
-
-3. Indexing and storage of embeddings: `Indexer`
-
- - See [Indexer Guide](/docs/eino/core_modules/components/indexer_guide)
-
-4. Retrieval of semantically related documents: `Retriever`
-
- - See [Retriever Guide](/docs/eino/core_modules/components/retriever_guide)
+ > See [Eino: Retriever Guide](/docs/eino/core_modules/components/retriever_guide)
+ >
**Decision and execution components:**
-1. Tool-enabled decision making for LLMs: `ToolsNode`
-
- - See [ToolsNode Guide](/docs/eino/core_modules/components/tools_node_guide)
+1. Component abstractions for LLM decision-making and tool calling: `ToolsNode`, `AgenticToolsNode`
-**Custom logic:**
+ > See [Eino: ToolsNode&Tool Guide](/docs/eino/core_modules/components/tools_node_guide), [Eino: AgenticToolsNode&Tool Guide [Beta]](/docs/eino/core_modules/components/agentic_tools_node_guide)
+ >
-1. User-defined business logic: `Lambda`
+**Custom components:**
- - See [Lambda Guide](/docs/eino/core_modules/components/lambda_guide)
+1. Component abstraction for user-defined code logic: `Lambda`
-Components provide application capabilities — the bricks and mortar of LLM app construction. Eino’s component abstractions follow these principles:
+ > See [Eino: Lambda Guide](/docs/eino/core_modules/components/lambda_guide)
+ >
-1. **Modularity and standardization**: unify common capabilities into clear modules with well-defined boundaries for flexible composition.
-2. **Extensibility**: keep interfaces minimally constraining so developers can implement custom components easily.
-3. **Reusability**: package common capabilities and implementations as ready-to-use tooling.
+Components are the capability providers for LLM applications, serving as the bricks and mortar in the construction process of LLM applications. The quality of component abstractions determines the complexity of LLM application development. Eino's component abstractions adhere to the following design principles:
-These abstractions establish consistent development paradigms, reduce cognitive load, and improve collaboration efficiency, letting developers focus on business logic instead of reinventing the wheel.
+1. **Modularity and standardization**: abstract a series of capabilities with the same functionality into unified modules, with clear responsibilities and boundaries between components, supporting flexible composition.
+2. **Extensibility**: keep the interface design with minimal constraints on module capabilities, allowing component developers to easily implement custom component development.
+3. **Reusability**: encapsulate the most commonly used capabilities and implementations, providing developers with ready-to-use tools.
+Component abstractions enable LLM application development to form relatively fixed paradigms, reducing cognitive complexity and enhancing collaboration efficiency. Component encapsulation allows developers to focus on implementing business logic, avoiding reinventing the wheel, and quickly building high-quality LLM applications.
diff --git a/content/en/docs/eino/core_modules/components/agentic_chat_model_guide.md b/content/en/docs/eino/core_modules/components/agentic_chat_model_guide.md
new file mode 100644
index 00000000000..3724c41bbba
--- /dev/null
+++ b/content/en/docs/eino/core_modules/components/agentic_chat_model_guide.md
@@ -0,0 +1,1178 @@
+---
+Description: ""
+date: "2026-01-20"
+lastmod: ""
+tags: []
+title: 'Eino: AgenticModel Guide [Beta]'
+weight: 10
+---
+
+## Overview
+
+AgenticModel is a model capability abstraction centered on "goal-driven autonomous execution". As capabilities like caching and built-in tools gain native support in advanced provider APIs such as OpenAI Responses API and Claude API, models are evolving from "one-shot Q&A engines" to "goal-oriented autonomous agents": capable of closed-loop planning, tool invocation, and iterative execution around user goals to accomplish more complex tasks.
+
+### Differences from ChatModel
+
+| AgenticModel | ChatModel | |
| Positioning | Model capability abstraction centered on "goal-driven autonomous execution", an enhanced abstraction over ChatModel | One-shot Q&A engine |
| Core Entities | Message | |
| Capabilities | ||
| Related Components |
| AgenticModel | ChatModel | |
| Temperature | Supported | Supported |
| Model | Supported | Supported |
| TopP | Supported | Supported |
| Tools | Supported | Supported |
| ToolChoice | Supported | Supported |
| MaxTokens | Supported | Supported |
| AllowedToolNames | Not Supported | Supported |
| Stop | Supported by some implementations | Supported |
| AllowedTools | Supported | Not Supported |
| Mode | Description |
| Transfer | Hand off the task to another agent. The current agent exits and does not wait for the child agent’s completion. |
| ToolCall (AgentAsTool) | Treat an agent as a tool, wait for its response, and use the output for subsequent processing. |
| Mode | Description |
| Transfer | Directly transfer the task to another Agent. The current Agent exits after execution and does not care about the task execution status of the transferred Agent |
| ToolCall (AgentAsTool) | Treat an Agent as a ToolCall, wait for the Agent's response, and obtain the output result of the called Agent for the next round of processing |
| Strategy | Description |
| Upstream full dialogue | Receive complete conversation history from upstream agents. |
| New task description | Ignore upstream dialogue and generate a fresh summary as input for the child agent. |
| Context Strategy | Description |
| Upstream Agent Full Dialogue | Get the complete conversation history of the upstream Agent |
| New Task Description | Ignore the complete conversation history of the upstream Agent and provide a new task summary as the AgentInput for the sub-Agent |
| Autonomy | Description |
| Autonomous | Agent internally selects downstream agents (often via LLM) when needed. Even preset logic is considered autonomous from the outside. |
| Preset | Agent execution order is predetermined and predictable. |
| Decision Autonomy | Description |
| Autonomous Decision | Within the Agent, based on its available downstream Agents, autonomously select a downstream Agent for assistance when needed. Generally, the Agent makes decisions based on LLM internally, but even if selection is based on preset logic, it is still considered autonomous decision from the Agent's external perspective |
| Preset Decision | Pre-set the next Agent after an Agent executes a task. The execution order of Agents is predetermined and predictable |
| Type | Description | Diagram | Collab | Context | Decision |
| SubAgents | Build a parent agent with a list of named subagents, forming a tree. Agent names must be unique within the tree. |  | Transfer | Upstream full dialogue | Autonomous |
| Sequential | Run subagents in order once, then finish. |  | Transfer | Upstream full dialogue | Preset |
| Parallel | Run subagents concurrently over shared input; finish after all complete. |  | Transfer | Upstream full dialogue | Preset |
| Loop | Run subagents in sequence repeatedly until termination. |  | Transfer | Upstream full dialogue | Preset |
| AgentAsTool | Convert an agent into a tool callable by others. `ChatModelAgent` supports this directly. |  | ToolCall | New task description | Autonomous |
| Type | Description | Running Mode | Collaboration Mode | Context Strategy | Decision Autonomy |
| SubAgents | Combine the user-provided agent as the parent Agent and the user-provided subAgents list as child Agents to form an Agent capable of autonomous decision-making, where Name and Description serve as the Agent's name identifier and description. | | Transfer | Upstream Agent Full Dialogue | Autonomous Decision |
| Sequential | Combine the user-provided SubAgents list into a Sequential Agent that executes in order, where Name and Description serve as the Sequential Agent's name identifier and description. When the Sequential Agent executes, it runs the SubAgents list in order until all Agents have been executed once. | | Transfer | Upstream Agent Full Dialogue | Preset Decision |
| Parallel | Combine the user-provided SubAgents list into a Parallel Agent that executes concurrently based on the same context, where Name and Description serve as the Parallel Agent's name identifier and description. When the Parallel Agent executes, it runs the SubAgents list concurrently and finishes after all Agents complete execution. | | Transfer | Upstream Agent Full Dialogue | Preset Decision |
| Loop | Execute the user-provided SubAgents list in array order, cycling repeatedly, to form a Loop Agent, where Name and Description serve as the Loop Agent's name identifier and description. When the Loop Agent executes, it runs the SubAgents list in order and finishes after all Agents complete execution. | | Transfer | Upstream Agent Full Dialogue | Preset Decision |
| AgentAsTool | Convert an Agent into a Tool to be used as a regular Tool by other Agents. Whether an Agent can call other Agents as Tools depends on its own implementation. The ChatModelAgent provided in adk supports the AgentAsTool functionality | | ToolCall | New Task Description | Autonomous Decision |
+
-Filtering by RunPath
+In Eino ADK, when building AgentInput for an Agent, the History it can see is "all AgentEvents produced before me."
-- When building `AgentInput` for an Agent, only include History events whose `RunPath` belong to the current Agent’s `RunPath` (equal or prefix).
+It's worth mentioning ParallelWorkflowAgent: two parallel sub-Agents (A, B) cannot see each other's AgentEvents during parallel execution, because neither parallel A nor B comes before the other.
-Definition: RunPathA “belongs to” RunPathB when RunPathA equals RunPathB or RunPathA is a prefix of RunPathB.
+#### RunPath
-Examples of RunPath in different orchestrations:
+Each AgentEvent in History is "produced by a specific Agent in a specific execution sequence", meaning AgentEvent has its own RunPath. The purpose of RunPath is to convey this information; it does not carry other functions in the eino framework.
-| Example | RunPath |
 | |
 | |
| |
| |
 | |
 | |
|
-Three `ChatModelAgent`s: a router, a weather agent (tool‑enabled), and a general chat agent. The router uses Transfer to hand off requests based on intent.
+All three Agents are implemented using ChatModelAgent:
```go
import (
@@ -230,7 +267,7 @@ func NewRouterAgent() adk.Agent {
}
```
-Wire them together and run:
+Then use Eino ADK's Transfer capability to build Multi-Agent and run. ChatModelAgent implements the OnSubAgent interface, and in the adk.SetSubAgents method, this interface is used to register parent/child Agents with ChatModelAgent, without requiring users to handle TransferAction generation:
```go
import (
@@ -295,7 +332,7 @@ func main() {
}
```
-Sample output:
+Running result:
```yaml
>>>>>>>>>query weather<<<<<<<<<
@@ -336,10 +373,10 @@ usage: &{206 23 229}
======
```
-Other `OnSubAgents` callbacks when registering child agents:
+The other two methods of OnSubAgents are called when an Agent is used as a sub-Agent in SetSubAgents:
-- `OnSetAsSubAgent`: register parent info to the agent
-- `OnDisallowTransferToParent`: when `WithDisallowTransferToParent` is set, inform the agent not to transfer back to its parent
+- OnSetAsSubAgent is used to register parent Agent information with the Agent
+- OnDisallowTransferToParent is called when the Agent is set with WithDisallowTransferToParent option, used to inform the Agent not to produce TransferAction to the parent Agent.
```go
adk.SetSubAgents(
@@ -351,22 +388,24 @@ adk.SetSubAgents(
)
```
-### Deterministic Transfer
+### Static Configuration Transfer
-Wrap a subagent so it automatically transfers back to a target (e.g., supervisor) after finishing:
+AgentWithDeterministicTransferTo is an Agent Wrapper that generates preset TransferAction after the original Agent executes, enabling static configuration of Agent jumping:
```go
+// github.com/cloudwego/eino/adk/flow.go
+
type DeterministicTransferConfig struct {
- Agent Agent
- ToAgentNames []string
+ Agent Agent
+ ToAgentNames []string
}
func AgentWithDeterministicTransferTo(_ context.Context, config *DeterministicTransferConfig) Agent
```
-Used by Supervisor to ensure subagents return control deterministically:
+In Supervisor mode, after a sub-Agent finishes execution, it returns to the Supervisor, and the Supervisor generates the next task target. In this case, AgentWithDeterministicTransferTo can be used:
-
+
```go
// github.com/cloudwego/eino/adk/prebuilt/supervisor.go
@@ -392,19 +431,19 @@ func NewSupervisor(ctx context.Context, conf *SupervisorConfig) (adk.Agent, erro
## Workflow Agents
-WorkflowAgent runs agents according to a preset flow. ADK provides three base Workflow agents: Sequential, Parallel, and Loop. They can be nested to build more complex tasks.
+WorkflowAgent supports running Agents according to a preset flow in code. Eino ADK provides three basic Workflow Agents: Sequential, Parallel, and Loop. They can be nested with each other to complete more complex tasks.
-By default, each agent’s input in a Workflow is generated via the History mechanism described earlier; you can customize `AgentInput` generation with `WithHistoryRewriter`.
+By default, each Agent's input in a Workflow is generated using the method described in the History section. You can customize AgentInput generation using WithHistoryRewriter.
-When an agent emits an `ExitAction` event, the Workflow agent exits immediately, regardless of remaining agents.
+When an Agent produces an ExitAction Event, the Workflow Agent will exit immediately, regardless of whether there are other Agents that need to run afterward.
-See details and examples: `/docs/eino/core_modules/eino_adk/agent_implementation/workflow`
+For details and usage examples, see: [Eino ADK: Workflow Agents](/docs/eino/core_modules/eino_adk/agent_implementation/workflow)
### SequentialAgent
-Run a series of agents in the provided order:
+SequentialAgent executes a series of Agents in the order you provide:
-
+
```go
type SequentialAgentConfig struct {
@@ -418,9 +457,9 @@ func NewSequentialAgent(ctx context.Context, config *SequentialAgentConfig) (Age
### LoopAgent
-Based on SequentialAgent; after completing one run, it starts from the beginning again:
+LoopAgent is implemented based on SequentialAgent. After SequentialAgent completes running, it starts from the beginning again:
-
+
```go
type LoopAgentConfig struct {
@@ -428,7 +467,7 @@ type LoopAgentConfig struct {
Description string
SubAgents []Agent
- MaxIterations int
+ MaxIterations int // Maximum number of iterations
}
func NewLoopAgent(ctx context.Context, config *LoopAgentConfig) (Agent, error)
@@ -436,9 +475,9 @@ func NewLoopAgent(ctx context.Context, config *LoopAgentConfig) (Agent, error)
### ParallelAgent
-Run agents concurrently:
+ParallelAgent runs multiple Agents concurrently:
-
+
```go
type ParallelAgentConfig struct {
@@ -449,3 +488,31 @@ type ParallelAgentConfig struct {
func NewParallelAgent(ctx context.Context, config *ParallelAgentConfig) (Agent, error)
```
+
+## AgentAsTool
+
+When an Agent only needs clear and explicit instructions rather than complete running context (History), the Agent can be converted to a Tool for calling:
+
+```go
+func NewAgentTool(_ context.Context, agent Agent, options ...AgentToolOption) tool.BaseTool
+```
+
+After being converted to a Tool, the Agent can be called by ChatModels that support function calling, and can also be called by all LLM-driven Agents. The calling method depends on the Agent implementation.
+
+Message History Isolation: An Agent as a Tool will not inherit the message history (History) of the parent Agent.
+
+SessionValues Sharing: However, it will share the SessionValues of the parent Agent, i.e., read and write the same KV map.
+
+Internal Event Exposure: An Agent as a Tool is still an Agent and will produce AgentEvents. By default, these internal AgentEvents will not be exposed through the `AsyncIterator` returned by `Runner`. In some business scenarios, if you need to expose internal AgentTool's AgentEvents to users, you need to add configuration in the parent `ChatModelAgent`'s `ToolsConfig` to enable internal event exposure:
+
+```go
+// from adk/chatmodel.go
+
+**type **ToolsConfig **struct **{
+ // other configurations...
+
+ _// EmitInternalEvents indicates whether internal events from agentTool should be emitted_
+_ // to the parent generator via a tool option injection at run-time._
+_ _EmitInternalEvents bool
+}
+```
diff --git a/content/en/docs/eino/core_modules/eino_adk/agent_hitl.md b/content/en/docs/eino/core_modules/eino_adk/agent_hitl.md
index 3ccd7d8604c..d8e69127dda 100644
--- a/content/en/docs/eino/core_modules/eino_adk/agent_hitl.md
+++ b/content/en/docs/eino/core_modules/eino_adk/agent_hitl.md
@@ -1,66 +1,19 @@
---
Description: ""
-date: "2025-12-11"
+date: "2026-01-20"
lastmod: ""
tags: []
-title: 'Eino Human-in-the-Loop: Architecture Guide'
+title: 'Eino Human-in-the-Loop Framework: Architecture Guide'
weight: 6
---
## Overview
-Eino’s HITL framework provides robust interrupt/resume and an addressing system to route user approvals or inputs back to the exact interruption point.
+This document provides technical details of Eino's human-in-the-loop (Human-in-the-Loop, HITL) framework architecture, focusing on the interrupt/resume mechanism and the underlying addressing system.
-Covers:
+## Human-in-the-Loop Requirements
-- Developer: when to interrupt, what to persist, what to expose
-- Framework: where interruption happens, how to persist context/state
-- User: where interruption occurred, whether/how to resume, what data to provide
-
-## Quick Start
-
-Example: a ticket‑booking `ChatModelAgent` that pauses for user approval before calling a booking tool.
-
-1) Create a `ChatModelAgent` with an approvable tool (decorator adds approval interrupt):
-
-```go
-getWeather := &tool2.InvokableApprovableTool{InvokableTool: baseBookTool}
-a, _ := adk.NewChatModelAgent(ctx, &adk.ChatModelAgentConfig{ /* model + getWeather tool */ })
-```
-
-2) Create a `Runner` with `CheckPointStore`, run with `WithCheckPointID("1")`:
-
-```go
-runner := adk.NewRunner(ctx, adk.RunnerConfig{ Agent: a, EnableStreaming: true, CheckPointStore: store.NewInMemoryStore() })
-iter := runner.Query(ctx, "book a ticket for Martin...", adk.WithCheckPointID("1"))
-```
-
-3) Read `event.Action.Interrupted`; capture `interruptID` and show user info; collect approval result.
-
-```go
-interruptID := lastEvent.Action.Interrupted.InterruptContexts[0].ID
-```
-
-4) Resume with parameters mapping `interruptID` → approval result:
-
-```go
-iter, err := runner.ResumeWithParams(ctx, "1", &adk.ResumeParams{ Targets: map[string]any{ interruptID: approvalResult } })
-```
-
-For production, use a distributed store (e.g., Redis) and expose interrupt context via your API/UI.
-
-## APIs
-
-- Create interrupts: `adk.Interrupt`, `adk.StatefulInterrupt`, `adk.CompositeInterrupt`
-- Access interrupt info: `InterruptInfo` with a flat list of `InterruptCtx{ID, Address, Info, IsRootCause, Parent}`
-- Resume: `(*Runner).ResumeWithParams(ctx, checkPointID, params)` and `ResumeInfo` for agents
-- Compose‑level helpers: `compose.Interrupt`, `compose.StatefulInterrupt`, `compose.CompositeInterrupt`, `compose.ExtractInterruptInfo`, and resume context helpers
-
-Examples repository: https://github.com/cloudwego/eino-examples/pull/125 and the HITL series under `adk/human-in-the-loop`.
-
-## HITL Needs
-
-The following diagram illustrates the key questions each party must answer during interrupt/resume. Understanding these needs explains the architecture design choices.
+The following diagram illustrates the key questions each component must answer during the interrupt/resume process. Understanding these requirements is key to grasping the rationale behind the architecture design.
```mermaid
graph TD
@@ -76,14 +29,10 @@ graph TD
subgraph Fw1 [Framework]
direction TB
- F1[Where in the execution
- did the interrupt occur?]
- F2[How to associate state with
- the interrupt location?]
- F3[How to persist interrupt
- context and state?]
- F4[What information does the user
- need to understand the interrupt?]
+ F1[Where in the execution hierarchy
-Configure tools via `ToolsConfig`:
+Configure tools for ChatModelAgent via ToolsConfig:
```go
+// github.com/cloudwego/eino/adk/chatmodel.go
+
type ToolsConfig struct {
compose.ToolsNodeConfig
// Names of the tools that will make agent return directly when the tool is called.
// When multiple tools are called and more than one tool is in the return directly list, only the first one will be returned.
ReturnDirectly map[string]bool
+
+ // EmitInternalEvents indicates whether internal events from agentTool should be emitted
+ // to the parent generator via a tool option injection at run-time.
+ EmitInternalEvents bool
}
```
-ToolsConfig reuses Eino Graph `ToolsNodeConfig`. See: [Eino: ToolsNode & Tool Guide](/docs/eino/core_modules/components/tools_node_guide)
+ToolsConfig reuses Eino Graph ToolsNodeConfig. See: [Eino: ToolsNode & Tool Guide](/docs/eino/core_modules/components/tools_node_guide). Additionally, it provides the ReturnDirectly configuration - ChatModelAgent will exit directly after calling a tool configured in ReturnDirectly.
+
+## ChatModelAgent Configuration Fields
-## Configuration Fields
+> 💡
+> Note: By default, GenModelInput renders the Instruction using F-String format via adk.GetSessionValues(). To disable this behavior, you can customize the GenModelInput method.
```go
type ChatModelAgentConfig struct {
@@ -81,6 +90,12 @@ type ChatModelAgentConfig struct {
// The agent will terminate with an error if this limit is exceeded.
// Optional. Defaults to 20.
MaxIterations int
+
+ // ModelRetryConfig configures retry behavior for the ChatModel.
+ // When set, the agent will automatically retry failed ChatModel calls
+ // based on the configured policy.
+ // Optional. If nil, no retry will be performed.
+ ModelRetryConfig *ModelRetryConfig
}
type ToolsConfig struct {
@@ -89,68 +104,160 @@ type ToolsConfig struct {
// Names of the tools that will make agent return directly when the tool is called.
// When multiple tools are called and more than one tool is in the return directly list, only the first one will be returned.
ReturnDirectly map[string]bool
+
+ // EmitInternalEvents indicates whether internal events from agentTool should be emitted
+ // to the parent generator via a tool option injection at run-time.
+ EmitInternalEvents bool
}
type GenModelInput func(ctx context.Context, instruction string, input *AgentInput) ([]Message, error)
```
-- `Name`: agent name
-- `Description`: agent description
-- `Instruction`: system prompt when calling ChatModel; supports f‑string rendering
-- `Model`: ToolCallingChatModel is required to support tool calls
-- `ToolsConfig`: tool configuration
- - Reuses Eino Graph `ToolsNodeConfig` (see: [Eino: ToolsNode & Tool Guide](/docs/eino/core_modules/components/tools_node_guide)).
- - `ReturnDirectly`: after calling a listed tool, ChatModelAgent exits immediately with the result; if multiple tools are listed, only the first match returns. Map key is the tool name.
-- `GenModelInput`: transforms `Instruction` and `AgentInput` to ChatModel messages; default:
- 1. Prepend `Instruction` as a System Message to `AgentInput.Messages`
- 2. Render `SessionValues` variables into the message list
-- `OutputKey`: stores the final message’s content into `SessionValues` under `OutputKey`
-- `MaxIterations`: upper bound on the ReAct loop; default 20
-- `Exit`: special tool that causes immediate agent termination when invoked; ADK provides a default `ExitTool` implementation:
+- `Name`: Agent name
+- `Description`: Agent description
+- `Instruction`: System Prompt when calling ChatModel, supports f-string rendering
+- `Model`: The ChatModel used for execution, must support tool calling
+- `ToolsConfig`: Tool configuration
+ - ToolsConfig reuses Eino Graph ToolsNodeConfig. See: [Eino: ToolsNode & Tool Guide](/docs/eino/core_modules/components/tools_node_guide).
+ - ReturnDirectly: After ChatModelAgent calls a tool configured in ReturnDirectly, it will immediately exit with the result without returning to ChatModel in ReAct mode. If multiple tools match, only the first one returns. Map key is the tool name.
+ - EmitInternalEvents: When using adk.AgentTool() to call an Agent as a SubAgent via ToolCall, by default this SubAgent won't emit AgentEvents and will only return the final result as ToolResult.
+- `GenModelInput`: When the Agent is called, this method transforms `Instruction` and `AgentInput` into Messages for calling ChatModel. The Agent provides a default GenModelInput method:
+ 1. Prepend `Instruction` as a `System Message` to `AgentInput.Messages`
+ 2. Render `SessionValues` as variables into the message list from step 1
+
+> 💡
+> The default `GenModelInput` uses pyfmt rendering. Text in the message list is treated as a pyfmt template, meaning '{' and '}' are treated as keywords. To use these characters literally, escape them as '{{' and '}}'
+
+- `OutputKey`: When configured, the last Message produced by ChatModelAgent will be stored in `SessionValues` with `OutputKey` as the key
+- `MaxIterations`: Maximum number of ChatModel generations in ReAct mode. The Agent will exit with an error if exceeded. Default is 20
+- `Exit`: Exit is a special Tool. When the model calls and executes this tool, ChatModelAgent will exit directly, similar to `ToolsConfig.ReturnDirectly`. ADK provides a default ExitTool implementation:
```go
type ExitTool struct{}
-func (et ExitTool) Info(_ context.Context) (*schema.ToolInfo, error) { return ToolInfoExit, nil }
+func (et ExitTool) Info(_ context.Context) (*schema.ToolInfo, error) {
+ return ToolInfoExit, nil
+}
func (et ExitTool) InvokableRun(ctx context.Context, argumentsInJSON string, _ ...tool.Option) (string, error) {
- type exitParams struct { FinalResult string `json:"final_result"` }
+ type exitParams struct {
+ FinalResult string `json:"final_result"`
+ }
+
params := &exitParams{}
err := sonic.UnmarshalString(argumentsInJSON, params)
- if err != nil { return "", err }
+ if err != nil {
+ return "", err
+ }
+
err = SendToolGenAction(ctx, "exit", NewExitAction())
- if err != nil { return "", err }
+ if err != nil {
+ return "", err
+ }
+
return params.FinalResult, nil
}
```
+- `ModelRetryConfig`: When configured, various errors during ChatModel requests (including direct errors and errors during streaming responses) will be retried according to the configured policy. If an error occurs during streaming response, that streaming response will still be returned immediately via AgentEvent. If the error during streaming will be retried according to the policy, consuming the message stream in AgentEvent will yield a `WillRetryError`. Users can handle this error for display purposes, as shown below:
+
+```go
+iterator := agent.Run(ctx, input)
+for {
+ event, ok := iterator.Next()
+ if !ok {
+ break
+ }
+
+ if event.Err != nil {
+ handleFinalError(event.Err)
+ break
+ }
+
+ // Process streaming output
+ if event.Output != nil && event.Output.MessageOutput.IsStreaming {
+ stream := event.Output.MessageOutput.MessageStream
+ for {
+ msg, err := stream.Recv()
+ if err == io.EOF {
+ break // Stream completed successfully
+ }
+ if err != nil {
+ // Check if this error will be retried (more streams coming)
+ var willRetry *adk.WillRetryError
+ if errors.As(err, &willRetry) {
+ log.Printf("Attempt %d failed, retrying...", willRetry.RetryAttempt)
+ break // Wait for next event with new stream
+ }
+ // Original error - won't retry, agent will stop and the next AgentEvent probably will be an error
+ log.Printf("Final error (no retry): %v", err)
+ break
+ }
+ // Display chunk to user
+ displayChunk(msg)
+ }
+ }
+}
+```
+
## ChatModelAgent Transfer
-`ChatModelAgent` implements `OnSubAgents`. After `SetSubAgents`, it adds a Transfer Tool and instructs the model to call it with the target agent name when transfer is needed:
+`ChatModelAgent` supports converting other Agents' metadata into its own Tools, enabling dynamic Transfer through ChatModel decisions:
+
+- `ChatModelAgent` implements the `OnSubAgents` interface. After using `SetSubAgents` to set sub-Agents for `ChatModelAgent`, it adds a `Transfer Tool` and instructs the ChatModel in the prompt to call this Tool when transfer is needed, using the target AgentName as input.
```go
-const TransferToAgentInstruction = `Available other agents: %s
+const (
+ TransferToAgentInstruction = `Available other agents: %s
Decision rule:
- If you're best suited for the question according to your description: ANSWER
- If another agent is better according its description: CALL '%s' function with their agent name
When transferring: OUTPUT ONLY THE FUNCTION CALL`
+)
+
+func genTransferToAgentInstruction(ctx context.Context, agents []Agent) string {
+ var sb strings.Builder
+ for _, agent := range agents {
+ sb.WriteString(fmt.Sprintf("\n- Agent name: %s\n Agent description: %s",
+ agent.Name(ctx), agent.Description(ctx)))
+ }
+
+ return fmt.Sprintf(TransferToAgentInstruction, sb.String(), TransferToAgentToolName)
+}
```
-– `Transfer Tool` sets a Transfer Event to jump to the target Agent; then ChatModelAgent exits.
-– Agent Runner receives the Transfer Event and switches execution to the target Agent.
+- `Transfer Tool` execution sets a Transfer Event, specifying the jump to the target Agent, then ChatModelAgent exits.
+- Agent Runner receives the Transfer Event and jumps to the target Agent for execution, completing the Transfer operation.
## ChatModelAgent AgentAsTool
-Convert an agent to a tool when clear input suffices:
+When an Agent to be called doesn't need the full execution context and only requires clear input parameters to run correctly, that Agent can be converted to a Tool for `ChatModelAgent` to decide when to call:
+
+- ADK provides utility methods to conveniently convert an Eino ADK Agent to a Tool for ChatModelAgent to call:
```go
-// github.com/cloudwego/eino/adk/agent_tool.go
+// github.com/cloudwego/eino/adk/agent_tool.go
+
func NewAgentTool(_ context.Context, agent Agent, options ...AgentToolOption) tool.BaseTool
```
-Register the converted Agent tool in `ToolsConfig` so `ChatModelAgent` can decide when to call it.
+- The converted Agent Tool can be registered directly in ChatModelAgent via `ToolsConfig`
+
+```go
+bookRecommender := NewBookRecommendAgent()
+bookRecommendeTool := NewAgentTool(ctx, bookRecommender)
+
+a, err := adk.NewChatModelAgent(ctx, &adk.ChatModelAgentConfig{
+ // ...
+ ToolsConfig: adk.ToolsConfig{
+ ToolsNodeConfig: compose.ToolsNodeConfig{
+ Tools: []tool.BaseTool{bookRecommendeTool},
+ },
+ },
+})
+```
# ChatModelAgent Usage Example
@@ -160,9 +267,21 @@ Create a book recommendation Agent that can recommend relevant books based on us
## Implementation
-### Step 1: Define the tool
+### Step 1: Define the Tool
+
+The book recommendation Agent needs a tool `book_search` that can search for books based on user requirements (genre, rating, etc.).
+
+You can easily create tools using the utility methods provided by Eino (see [How to Create a Tool?](/docs/eino/core_modules/components/tools_node_guide/how_to_create_a_tool)):
```go
+import (
+ "context"
+ "log"
+
+ "github.com/cloudwego/eino/components/tool"
+ "github.com/cloudwego/eino/components/tool/utils"
+)
+
type BookSearchInput struct {
Genre string `json:"genre" jsonschema:"description=Preferred book genre,enum=fiction,enum=sci-fi,enum=mystery,enum=biography,enum=business"`
MaxPages int `json:"max_pages" jsonschema:"description=Maximum page length (0 for no limit)"`
@@ -188,7 +307,19 @@ func NewBookRecommender() tool.InvokableTool {
### Step 2: Create ChatModel
+Eino provides various ChatModel implementations (such as openai, gemini, doubao, etc. See [Eino: ChatModel Guide](/docs/eino/core_modules/components/chat_model_guide)). Here we use openai ChatModel as an example:
+
```go
+import (
+ "context"
+ "fmt"
+ "log"
+ "os"
+
+ "github.com/cloudwego/eino-ext/components/model/openai"
+ "github.com/cloudwego/eino/components/model"
+)
+
func NewChatModel() model.ToolCallingChatModel {
ctx := context.Background()
apiKey := os.Getenv("OPENAI_API_KEY")
@@ -207,20 +338,32 @@ func NewChatModel() model.ToolCallingChatModel {
### Step 3: Create ChatModelAgent
+In addition to configuring ChatModel and tools, you need to configure Name and Description to describe the Agent's functionality, as well as Instruction to guide the ChatModel. The Instruction will ultimately be passed to ChatModel as a system message.
+
```go
+import (
+ "context"
+ "fmt"
+ "log"
+
+ "github.com/cloudwego/eino/adk"
+ "github.com/cloudwego/eino/components/tool"
+ "github.com/cloudwego/eino/compose"
+)
+
func NewBookRecommendAgent() adk.Agent {
ctx := context.Background()
a, err := adk.NewChatModelAgent(ctx, &adk.ChatModelAgentConfig{
Name: "BookRecommender",
- Description: "An agent that can recommend books",
- Instruction: `You are an expert book recommender. Based on the user's request, use the "search_book" tool to find relevant books. Finally, present the results to the user.`,
- Model: NewChatModel(),
- ToolsConfig: adk.ToolsConfig{
- ToolsNodeConfig: compose.ToolsNodeConfig{
- Tools: []tool.BaseTool{NewBookRecommender()},
- },
- },
+ Description: "An agent that can recommend books",
+ Instruction: `You are an expert book recommender. Based on the user's request, use the "search_book" tool to find relevant books. Finally, present the results to the user.`,
+ Model: NewChatModel(),
+ ToolsConfig: adk.ToolsConfig{
+ ToolsNodeConfig: compose.ToolsNodeConfig{
+ Tools: []tool.BaseTool{NewBookRecommender()},
+ },
+ },
})
if err != nil {
log.Fatal(fmt.Errorf("failed to create chatmodel: %w", err))
@@ -233,14 +376,38 @@ func NewBookRecommendAgent() adk.Agent {
### Step 4: Run via Runner
```go
-runner := adk.NewRunner(ctx, adk.RunnerConfig{ Agent: a })
-iter := runner.Query(ctx, "recommend a fiction book to me")
-for {
- event, ok := iter.Next(); if !ok { break }
- if event.Err != nil { log.Fatal(event.Err) }
- msg, err := event.Output.MessageOutput.GetMessage()
- if err != nil { log.Fatal(err) }
- fmt.Printf("\nmessage:\n%v\n======", msg)
+import (
+ "context"
+ "fmt"
+ "log"
+ "os"
+
+ "github.com/cloudwego/eino/adk"
+
+ "github.com/cloudwego/eino-examples/adk/intro/chatmodel/internal"
+)
+
+func main() {
+ ctx := context.Background()
+ a := internal.NewBookRecommendAgent()
+ runner := adk.NewRunner(ctx, adk.RunnerConfig{
+ Agent: a,
+ })
+ iter := runner.Query(ctx, "recommend a fiction book to me")
+ for {
+ event, ok := iter.Next()
+ if !ok {
+ break
+ }
+ if event.Err != nil {
+ log.Fatal(event.Err)
+ }
+ msg, err := event.Output.MessageOutput.GetMessage()
+ if err != nil {
+ log.Fatal(err)
+ }
+ fmt.Printf("\nmessage:\n%v\n======", msg)
+ }
}
```
@@ -275,9 +442,9 @@ usage: &{185 31 216}
## Introduction
-`ChatModelAgent` uses Eino Graph internally, so the agent reuses Graph’s Interrupt & Resume capability.
+`ChatModelAgent` is implemented using Eino Graph, so the agent can reuse Eino Graph's Interrupt & Resume capability.
-– On interrupt, a tool returns a special error to trigger Graph interrupt and emit custom info; on resume, Graph reruns this tool:
+- On interrupt, return a special error from the tool to trigger Graph interrupt and emit custom info. On resume, Graph will rerun this tool:
```go
// github.com/cloudwego/eino/adk/interrupt.go
@@ -285,19 +452,213 @@ usage: &{185 31 216}
func NewInterruptAndRerunErr(extra any) error
```
-– On resume, pass custom `tool.Option`s to supply extra info to tools:
+- On resume, support custom ToolOptions to pass extra information to the Tool:
```go
import (
"github.com/cloudwego/eino/components/tool"
)
-type askForClarificationOptions struct { NewInput *string }
+type askForClarificationOptions struct {
+ NewInput *string
+}
+
func WithNewInput(input string) tool.Option {
- return tool.WrapImplSpecificOptFn(func(t *askForClarificationOptions) { t.NewInput = &input })
+ return tool.WrapImplSpecificOptFn(func(t *askForClarificationOptions) {
+ t.NewInput = &input
+ })
}
```
## Example
-Add `ask_for_clarification` tool that interrupts until new input is provided, then resumes and continues.
+Below we will add an `ask_for_clarification` tool to the `BookRecommendAgent` from the [ChatModelAgent Usage Example] section above. When the user provides insufficient information for recommendations, the Agent will call this tool to ask the user for more information. `ask_for_clarification` uses the Interrupt & Resume capability to implement "asking" the user.
+
+### Step 1: Add Tool with Interrupt Support
+
+```go
+import (
+ "context"
+ "log"
+
+ "github.com/cloudwego/eino/components/tool"
+ "github.com/cloudwego/eino/components/tool/utils"
+ "github.com/cloudwego/eino/compose"
+)
+
+type askForClarificationOptions struct {
+ NewInput *string
+}
+
+func WithNewInput(input string) tool.Option {
+ return tool.WrapImplSpecificOptFn(func(t *askForClarificationOptions) {
+ t.NewInput = &input
+ })
+}
+
+type AskForClarificationInput struct {
+ Question string `json:"question" jsonschema:"description=The specific question you want to ask the user to get the missing information"`
+}
+
+func NewAskForClarificationTool() tool.InvokableTool {
+ t, err := utils.InferOptionableTool(
+ "ask_for_clarification",
+ "Call this tool when the user's request is ambiguous or lacks the necessary information to proceed. Use it to ask a follow-up question to get the details you need, such as the book's genre, before you can use other tools effectively.",
+ func(ctx context.Context, input *AskForClarificationInput, opts ...tool.Option) (output string, err error) {
+ o := tool.GetImplSpecificOptions[askForClarificationOptions](nil, opts...)
+ if o.NewInput == nil {
+ return "", compose.NewInterruptAndRerunErr(input.Question)
+ }
+ return *o.NewInput, nil
+ })
+ if err != nil {
+ log.Fatal(err)
+ }
+ return t
+}
+```
+
+### Step 2: Add Tool to Agent
+
+```go
+func NewBookRecommendAgent() adk.Agent {
+ // xxx
+ a, err := adk.NewChatModelAgent(ctx, &adk.ChatModelAgentConfig{
+ // xxx
+ ToolsConfig: adk.ToolsConfig{
+ ToolsNodeConfig: compose.ToolsNodeConfig{
+ Tools: []tool.BaseTool{NewBookRecommender(), NewAskForClarificationTool()},
+ },
+ // Whether to output AgentEvents when calling SubAgent via AgentTool() internally
+ EmitInternalEvents: true,
+ },
+ })
+ // xxx
+}
+```
+
+### Step 3: Configure CheckPointStore in Agent Runner
+
+Configure `CheckPointStore` in the Runner (using the simplest InMemoryStore in this example), and pass `CheckPointID` when calling the Agent for use during resume. Also, on interrupt, Graph will place `InterruptInfo` in `Interrupted.Data`:
+
+```go
+func newInMemoryStore() compose.CheckPointStore {
+ return &inMemoryStore{
+ mem: map[string][]byte{},
+ }
+}
+
+func main() {
+ ctx := context.Background()
+ a := subagents.NewBookRecommendAgent()
+ runner := adk.NewRunner(ctx, adk.RunnerConfig{
+ EnableStreaming: true, // you can disable streaming here
+ Agent: a,
+ CheckPointStore: newInMemoryStore(),
+ })
+ iter := runner.Query(ctx, "recommend a book to me", adk.WithCheckPointID("1"))
+ for {
+ event, ok := iter.Next()
+ if !ok {
+ break
+ }
+ if event.Err != nil {
+ log.Fatal(event.Err)
+ }
+ if event.Action != nil && event.Action.Interrupted != nil {
+ fmt.Printf("\ninterrupt happened, info: %+v\n", event.Action.Interrupted.Data.(*adk.ChatModelAgentInterruptInfo).RerunNodesExtra["ToolNode"])
+ continue
+ }
+ msg, err := event.Output.MessageOutput.GetMessage()
+ if err != nil {
+ log.Fatal(err)
+ }
+ fmt.Printf("\nmessage:\n%v\n======\n\n", msg)
+ }
+
+ scanner := bufio.NewScanner(os.Stdin)
+ fmt.Print("\nyour input here: ")
+ scanner.Scan()
+ fmt.Println()
+ nInput := scanner.Text()
+
+ iter, err := runner.Resume(ctx, "1", adk.WithToolOptions([]tool.Option{subagents.WithNewInput(nInput)}))
+ if err != nil {
+ log.Fatal(err)
+ }
+ for {
+ event, ok := iter.Next()
+ if !ok {
+ break
+ }
+
+ if event.Err != nil {
+ log.Fatal(event.Err)
+ }
+
+ prints.Event(event)
+ }
+}
+```
+
+### Run Result
+
+Running will trigger an interrupt
+
+```
+message:
+assistant:
+tool_calls:
+{Index:
@@ -39,31 +39,31 @@ SequentialAgent executes sub‑agents strictly in the order provided. Each sub
type SequentialAgentConfig struct {
Name string // Agent name
Description string // Agent description
- SubAgents []Agent // Sub‑agents in execution order
+ SubAgents []Agent // List of sub-agents, arranged in execution order
}
func NewSequentialAgent(ctx context.Context, config *SequentialAgentConfig) (Agent, error)
```
-Execution rules:
+SequentialAgent execution follows these rules:
-1. Linear execution: strictly in `SubAgents` order
-2. History passing: each agent’s result is added to History; subsequent agents can access prior history
-3. Early termination: if any sub‑agent emits ExitAction / Interrupt, the whole Sequential flow ends immediately
+1. **Linear execution**: Strictly follows the order of the SubAgents array
+2. **History passing**: Each agent's execution result is added to History, allowing subsequent agents to access the execution history of previous agents
+3. **Early exit**: If any sub-agent produces an ExitAction / Interrupt, the entire Sequential flow terminates immediately
-Suitable for:
+SequentialAgent is suitable for the following scenarios:
-- Multi‑step pipelines: e.g., preprocessing → analysis → report
-- Pipeline processing: each step’s output feeds the next
-- Dependent task sequences: later tasks rely on earlier results
+- **Multi-step processing flows**: Such as data preprocessing -> analysis -> report generation
+- **Pipeline processing**: Each step's output serves as the next step's input
+- **Task sequences with dependencies**: Subsequent tasks depend on results from previous tasks
## Example
-Create a three‑step document processing pipeline:
+This example demonstrates how to use SequentialAgent to create a three-step document processing pipeline:
-1. DocumentAnalyzer — analyze document content
-2. ContentSummarizer — summarize analysis
-3. ReportGenerator — generate final report
+1. **DocumentAnalyzer**: Analyzes document content
+2. **ContentSummarizer**: Summarizes analysis results
+3. **ReportGenerator**: Generates the final report
```go
package main
@@ -80,7 +80,7 @@ import (
"github.com/cloudwego/eino/schema"
)
-// 创建 ChatModel 实例
+// Create ChatModel instance
func newChatModel() model.ToolCallingChatModel {
cm, err := openai.NewChatModel(context.Background(), &openai.ChatModelConfig{
APIKey: os.Getenv("OPENAI_API_KEY"),
@@ -92,12 +92,12 @@ func newChatModel() model.ToolCallingChatModel {
return cm
}
-// 文档分析 Agent
+// Document analysis Agent
func NewDocumentAnalyzerAgent() adk.Agent {
a, err := adk.NewChatModelAgent(context.Background(), &adk.ChatModelAgentConfig{
Name: "DocumentAnalyzer",
- Description: "分析文档内容并提取关键信息",
- Instruction: "你是一个文档分析专家。请仔细分析用户提供的文档内容，提取其中的关键信息、主要观点和重要数据。",
+ Description: "Analyzes document content and extracts key information",
+ Instruction: "You are a document analysis expert. Please carefully analyze the document content provided by the user, extracting key information, main points, and important data.",
Model: newChatModel(),
})
if err != nil {
@@ -106,12 +106,12 @@ func NewDocumentAnalyzerAgent() adk.Agent {
return a
}
-// 内容总结 Agent
+// Content summarization Agent
func NewContentSummarizerAgent() adk.Agent {
a, err := adk.NewChatModelAgent(context.Background(), &adk.ChatModelAgentConfig{
Name: "ContentSummarizer",
- Description: "对分析结果进行总结",
- Instruction: "基于前面的文档分析结果，生成一个简洁明了的总结，突出最重要的发现和结论。",
+ Description: "Summarizes analysis results",
+ Instruction: "Based on the previous document analysis results, generate a concise and clear summary highlighting the most important findings and conclusions.",
Model: newChatModel(),
})
if err != nil {
@@ -120,12 +120,12 @@ func NewContentSummarizerAgent() adk.Agent {
return a
}
-// 报告生成 Agent
+// Report generation Agent
func NewReportGeneratorAgent() adk.Agent {
a, err := adk.NewChatModelAgent(context.Background(), &adk.ChatModelAgentConfig{
Name: "ReportGenerator",
- Description: "生成最终的分析报告",
- Instruction: "基于前面的分析和总结，生成一份结构化的分析报告，包含执行摘要、详细分析和建议。",
+ Description: "Generates the final analysis report",
+ Instruction: "Based on the previous analysis and summary, generate a structured analysis report including an executive summary, detailed analysis, and recommendations.",
Model: newChatModel(),
})
if err != nil {
@@ -137,30 +137,30 @@ func NewReportGeneratorAgent() adk.Agent {
func main() {
ctx := context.Background()
- // 创建三个处理步骤的 Agent
+ // Create three processing step Agents
analyzer := NewDocumentAnalyzerAgent()
summarizer := NewContentSummarizerAgent()
generator := NewReportGeneratorAgent()
- // 创建 SequentialAgent
+ // Create SequentialAgent
sequentialAgent, err := adk.NewSequentialAgent(ctx, &adk.SequentialAgentConfig{
Name: "DocumentProcessingPipeline",
- Description: "文档处理流水线：分析 → 总结 → 报告生成",
+ Description: "Document processing pipeline: Analysis → Summary → Report Generation",
SubAgents: []adk.Agent{analyzer, summarizer, generator},
})
if err != nil {
log.Fatal(err)
}
- // 创建 Runner
+ // Create Runner
runner := adk.NewRunner(ctx, adk.RunnerConfig{
Agent: sequentialAgent,
})
- // 执行文档处理流程
- input := "请分析以下市场报告：2024年第三季度，公司营收增长15%，主要得益于新产品线的成功推出。但运营成本也上升了8%，需要优化效率。"
+ // Execute document processing flow
+ input := "Please analyze the following market report: In Q3 2024, company revenue grew 15%, mainly due to the successful launch of new product lines. However, operating costs also increased by 8%, requiring efficiency optimization."
- fmt.Println("开始执行文档处理流水线...")
+ fmt.Println("Starting document processing pipeline...")
iter := runner.Query(ctx, input)
stepCount := 1
@@ -175,79 +175,79 @@ func main() {
}
if event.Output != nil && event.Output.MessageOutput != nil {
- fmt.Printf("\n=== 步骤 %d: %s ===\n", stepCount, event.AgentName)
+ fmt.Printf("\n=== Step %d: %s ===\n", stepCount, event.AgentName)
fmt.Printf("%s\n", event.Output.MessageOutput.Message.Content)
stepCount++
}
}
- fmt.Println("\n文档处理流水线执行完成！")
+ fmt.Println("\nDocument processing pipeline completed!")
}
```
Run result:
```markdown
-开始执行文档处理流水线...
+Starting document processing pipeline...
-=== 步骤 1: DocumentAnalyzer ===
-市场报告关键信息分析：
+=== Step 1: DocumentAnalyzer ===
+Market Report Key Information Analysis:
-1. 营收增长情况：
- - 2024年第三季度，公司营收同比增长15%。
- - 营收增长的主要驱动力是新产品线的成功推出。
+1. Revenue Growth:
+ - In Q3 2024, company revenue grew 15% year-over-year.
+ - The main driver of revenue growth was the successful launch of new product lines.
-2. 成本情况：
- - 运营成本上涨了8%。
- - 成本上升提醒公司需要进行效率优化。
+2. Cost Situation:
+ - Operating costs increased by 8%.
+ - The cost increase reminds the company of the need for efficiency optimization.
-主要观点总结：
-- 新产品线推出显著推动了营收增长，显示公司在产品创新方面取得良好成果。
-- 虽然营收提升，但运营成本的增加在一定程度上影响了盈利能力，指出了提升运营效率的重要性。
+Key Points Summary:
+- The new product line launch significantly drove revenue growth, showing good results in product innovation.
+- Although revenue increased, the rise in operating costs somewhat affected profitability, highlighting the importance of improving operational efficiency.
-重要数据：
-- 营收增长率：15%
-- 运营成本增长率：8%
+Important Data:
+- Revenue growth rate: 15%
+- Operating cost growth rate: 8%
-=== 步骤 2: ContentSummarizer ===
-总结：2024年第三季度，公司实现了15%的营收增长，主要归功于新产品线的成功推出，体现了公司产品创新能力的显著提升。然而，运营成本同时上涨了8%，对盈利能力构成一定压力，强调了优化运营效率的迫切需求。整体来看，公司在增长与成本控制之间需寻求更好的平衡以保障持续健康发展。
+=== Step 2: ContentSummarizer ===
+Summary: In Q3 2024, the company achieved 15% revenue growth, mainly attributed to the successful launch of new product lines, demonstrating significant improvement in product innovation capability. However, operating costs also increased by 8%, putting some pressure on profitability and emphasizing the urgent need for operational efficiency optimization. Overall, the company needs to seek a better balance between growth and cost control to ensure sustainable healthy development.
-=== 步骤 3: ReportGenerator ===
-分析报告
+=== Step 3: ReportGenerator ===
+Analysis Report
-一、执行摘要
-2024年第三季度，公司实现营收同比增长15%，主要得益于新产品线的成功推出，展现了强劲的产品创新能力。然而，运营成本也同比提升了8%，对利润空间形成一定压力。为确保持续的盈利增长，需重点关注运营效率的优化，推动成本控制与收入增长的平衡发展。
+I. Executive Summary
+In Q3 2024, the company achieved 15% year-over-year revenue growth, mainly driven by the successful launch of new product lines, demonstrating strong product innovation capability. However, operating costs also increased 8% year-over-year, putting some pressure on profit margins. To ensure continued profitable growth, focus should be on optimizing operational efficiency and promoting balanced development of cost control and revenue growth.
-二、详细分析
-1. 营收增长分析
-- 公司营收增长15%，反映出新产品线市场接受度良好，有效拓展了收入来源。
-- 新产品线的推出体现了公司研发及市场响应能力的提升，为未来持续增长奠定基础。
+II. Detailed Analysis
+1. Revenue Growth Analysis
+- The company's 15% revenue growth reflects good market acceptance of new product lines, effectively expanding revenue sources.
+- The launch of new product lines demonstrates improved R&D and market responsiveness, laying a foundation for future sustained growth.
-2. 运营成本情况
-- 运营成本上升8%，可能来自原材料价格上涨、生产效率下降或销售推广费用增加等多个方面。
-- 该成本提升在一定程度上抵消了收入增长带来的利润增益，影响整体盈利能力。
+2. Operating Cost Situation
+- The 8% increase in operating costs may come from various aspects including raw material price increases, decreased production efficiency, or increased sales and promotion expenses.
+- This cost increase somewhat offsets the profit gains from revenue growth, affecting overall profitability.
-3. 盈利能力及效率考量
-- 营收与成本增长的不匹配显示出当前运营效率存在改进空间。
-- 优化供应链管理、提升生产自动化及加强成本控制将成为关键措施。
+3. Profitability and Efficiency Considerations
+- The mismatch between revenue and cost growth indicates room for improvement in current operational efficiency.
+- Optimizing supply chain management, improving production automation, and strengthening cost control will become key measures.
-三、建议
-1. 加强新产品线后续支持，包括市场推广和客户反馈机制，持续推动营收增长。
-2. 深入分析运营成本构成，识别主要成本驱动因素，制定针对性降低成本的策略。
-3. 推动内部流程优化与技术升级，提升生产及运营效率，缓解成本压力。
-4. 建立动态的财务监控体系，实现对营收与成本的实时跟踪与调整，确保公司财务健康。
+III. Recommendations
+1. Strengthen follow-up support for new product lines, including marketing and customer feedback mechanisms, to continue driving revenue growth.
+2. Conduct in-depth analysis of operating cost composition, identify main cost drivers, and develop targeted cost reduction strategies.
+3. Promote internal process optimization and technology upgrades to improve production and operational efficiency and alleviate cost pressure.
+4. Establish a dynamic financial monitoring system to achieve real-time tracking and adjustment of revenue and costs, ensuring company financial health.
-四、结论
-公司在2024年第三季度展现出了良好的增长动力，但同时面临成本上升带来的挑战。通过持续的产品创新结合有效的成本管理，未来有望实现盈利能力和市场竞争力的双重提升，推动公司稳健发展。
+IV. Conclusion
+The company demonstrated good growth momentum in Q3 2024 but also faces challenges from rising costs. Through continuous product innovation combined with effective cost management, there is potential to achieve dual improvement in profitability and market competitiveness, driving steady company development.
-文档处理流水线执行完成！
+Document processing pipeline completed!
```
# LoopAgent
-## Functionality
+## Features
-LoopAgent builds on SequentialAgent and repeats the sub‑agent sequence until reaching `MaxIterations` or a sub‑agent emits ExitAction. Ideal for iterative optimization, repeated processing, or continuous monitoring.
+LoopAgent is built on SequentialAgent. It repeatedly executes the configured sub-agent sequence until the maximum iteration count is reached or a sub-agent produces an ExitAction. LoopAgent is particularly suitable for scenarios requiring iterative optimization, repeated processing, or continuous monitoring.
@@ -255,33 +255,35 @@ LoopAgent builds on SequentialAgent and repeats the sub‑agent sequence until r
type LoopAgentConfig struct {
Name string // Agent name
Description string // Agent description
- SubAgents []Agent // Sub‑agent list
- MaxIterations int // Max iterations; 0 for infinite loop
+ SubAgents []Agent // List of sub-agents
+ MaxIterations int // Maximum iteration count, 0 means infinite loop
}
func NewLoopAgent(ctx context.Context, config *LoopAgentConfig) (Agent, error)
```
-Execution rules:
+LoopAgent execution follows these rules:
-1. Loop execution: repeat the `SubAgents` sequence; each loop is a full Sequential run
-2. History accumulation: results from each iteration accumulate into History
-3. Exit conditions: ExitAction or reaching `MaxIterations` stops the loop; `MaxIterations=0` means infinite loop
+1. **Loop execution**: Repeatedly executes the SubAgents sequence, with each loop being a complete Sequential execution process
+2. **History accumulation**: Results from each iteration accumulate in History, allowing subsequent iterations to access all historical information
+3. **Conditional exit**: Supports terminating the loop via ExitAction or reaching maximum iteration count; setting `MaxIterations=0` means infinite loop
-Suitable for:
+LoopAgent is suitable for the following scenarios:
-- Iterative optimization
-- Continuous monitoring
-- Repeated processing to reach a satisfactory result
-- Self‑improvement based on prior outputs
+- **Iterative optimization**: Tasks requiring repeated improvement such as code optimization, parameter tuning
+- **Continuous monitoring**: Periodically checking status and executing corresponding operations
+- **Repeated processing**: Tasks that need multiple rounds of processing to achieve satisfactory results
+- **Self-improvement**: Agent continuously improves its output based on previous execution results
## Example
-An iterative code optimization loop:
+This example demonstrates how to use LoopAgent to create a code optimization loop:
-1. CodeAnalyzer — analyze code issues
-2. CodeOptimizer — optimize based on analysis
-3. ExitController — decide whether to exit the loop
+1. **CodeAnalyzer**: Analyzes code issues
+2. **CodeOptimizer**: Optimizes code based on analysis results
+3. **ExitController**: Determines whether to exit the loop
+
+The loop continues until code quality meets standards or maximum iteration count is reached.
```go
package main
@@ -309,19 +311,19 @@ func newChatModel() model.ToolCallingChatModel {
return cm
}
-// 代码分析 Agent
+// Code analysis Agent
func NewCodeAnalyzerAgent() adk.Agent {
a, err := adk.NewChatModelAgent(context.Background(), &adk.ChatModelAgentConfig{
Name: "CodeAnalyzer",
- Description: "分析代码质量和性能问题",
- Instruction: `你是一个代码分析专家。请分析提供的代码，识别以下问题：
-1. 性能瓶颈
-2. 代码重复
-3. 可读性问题
-4. 潜在的 bug
-5. 不符合最佳实践的地方
-
-如果代码已经足够优秀，请输出 "EXIT: 代码质量已达到标准" 来结束优化流程。`,
+ Description: "Analyzes code quality and performance issues",
+ Instruction: `You are a code analysis expert. Please analyze the provided code and identify the following issues:
+1. Performance bottlenecks
+2. Code duplication
+3. Readability issues
+4. Potential bugs
+5. Non-compliance with best practices
+
+If the code is already excellent, output "EXIT: Code quality has met standards" to end the optimization process.`,
Model: newChatModel(),
})
if err != nil {
@@ -330,19 +332,19 @@ func NewCodeAnalyzerAgent() adk.Agent {
return a
}
-// 代码优化 Agent
+// Code optimization Agent
func NewCodeOptimizerAgent() adk.Agent {
a, err := adk.NewChatModelAgent(context.Background(), &adk.ChatModelAgentConfig{
Name: "CodeOptimizer",
- Description: "根据分析结果优化代码",
- Instruction: `基于前面的代码分析结果，对代码进行优化改进：
-1. 修复识别出的性能问题
-2. 消除代码重复
-3. 提高代码可读性
-4. 修复潜在 bug
-5. 应用最佳实践
-
-请提供优化后的完整代码。`,
+ Description: "Optimizes code based on analysis results",
+ Instruction: `Based on the previous code analysis results, optimize and improve the code:
+1. Fix identified performance issues
+2. Eliminate code duplication
+3. Improve code readability
+4. Fix potential bugs
+5. Apply best practices
+
+Please provide the complete optimized code.`,
Model: newChatModel(),
})
if err != nil {
@@ -351,13 +353,13 @@ func NewCodeOptimizerAgent() adk.Agent {
return a
}
-// 创建一个特殊的 Agent 来处理退出逻辑
+// Create a special Agent to handle exit logic
func NewExitControllerAgent() adk.Agent {
a, err := adk.NewChatModelAgent(context.Background(), &adk.ChatModelAgentConfig{
Name: "ExitController",
- Description: "控制优化循环的退出",
- Instruction: `检查前面的分析结果，如果代码分析师认为代码质量已达到标准（包含"EXIT"关键词），
-则输出 "TERMINATE" 并生成退出动作来结束循环。否则继续下一轮优化。`,
+ Description: "Controls the exit of the optimization loop",
+ Instruction: `Check the previous analysis results. If the code analyst believes the code quality has met standards (contains "EXIT" keyword),
+output "TERMINATE" and generate an exit action to end the loop. Otherwise continue to the next optimization round.`,
Model: newChatModel(),
})
if err != nil {
@@ -369,28 +371,28 @@ func NewExitControllerAgent() adk.Agent {
func main() {
ctx := context.Background()
- // 创建优化流程的 Agent
+ // Create optimization flow Agents
analyzer := NewCodeAnalyzerAgent()
optimizer := NewCodeOptimizerAgent()
controller := NewExitControllerAgent()
- // 创建 LoopAgent，最多执行 5 轮优化
+ // Create LoopAgent, execute up to 5 optimization rounds
loopAgent, err := adk.NewLoopAgent(ctx, &adk.LoopAgentConfig{
Name: "CodeOptimizationLoop",
- Description: "代码优化循环：分析 → 优化 → 检查退出条件",
+ Description: "Code optimization loop: Analysis → Optimization → Check exit condition",
SubAgents: []adk.Agent{analyzer, optimizer, controller},
- MaxIterations: 5, // 最多 5 轮优化
+ MaxIterations: 5, // Maximum 5 optimization rounds
})
if err != nil {
log.Fatal(err)
}
- // 创建 Runner
+ // Create Runner
runner := adk.NewRunner(ctx, adk.RunnerConfig{
Agent: loopAgent,
})
- // 待优化的代码示例
+ // Code example to optimize
codeToOptimize := `
func processData(data []int) []int {
result := []int{}
@@ -406,8 +408,8 @@ func processData(data []int) []int {
}
`
- fmt.Println("开始代码优化循环...")
- iter := runner.Query(ctx, "请优化以下 Go 代码：\n"+codeToOptimize)
+ fmt.Println("Starting code optimization loop...")
+ iter := runner.Query(ctx, "Please optimize the following Go code:\n"+codeToOptimize)
iteration := 1
for {
@@ -421,13 +423,13 @@ func processData(data []int) []int {
}
if event.Output != nil && event.Output.MessageOutput != nil {
- fmt.Printf("\n=== 第 %d 轮 - %s ===\n", iteration, event.AgentName)
+ fmt.Printf("\n=== Round %d - %s ===\n", iteration, event.AgentName)
fmt.Printf("%s\n", event.Output.MessageOutput.Message.Content)
- // 检查是否需要退出
+ // Check if exit is needed
if event.AgentName == "ExitController" {
if event.Action != nil && event.Action.Exit {
- fmt.Println("\n优化循环提前结束！")
+ fmt.Println("\nOptimization loop ended early!")
break
}
iteration++
@@ -435,17 +437,17 @@ func processData(data []int) []int {
}
}
- fmt.Println("\n代码优化循环执行完成！")
+ fmt.Println("\nCode optimization loop completed!")
}
```
-Run result (excerpt):
+Run result:
```java
-开始代码优化循环...
+Starting code optimization loop...
-=== 第 1 轮 - CodeAnalyzer ===
-分析提供的代码：
+=== Round 1 - CodeAnalyzer ===
+Analyzing the provided code:
```go
func processData(data []int) []int {
@@ -462,15 +464,258 @@ func processData(data []int) []int {
}
```
-... (analysis and optimized code)
+### 1. Performance Bottlenecks
+
+- Double nested loop with O(n²) time complexity, poor performance for large datasets.
+- Inner loop breaks immediately when condition is met, reducing some unnecessary comparisons, but overall still quadratic complexity.
+
+### 2. Code Duplication
+
+- No obvious code duplication, but logic can be simplified.
+
+### 3. Readability Issues
+
+- The code's intent is not very clear. Based on the code logic, the function filters out elements that have at least one element smaller than them in the array. In other words, it removes all elements that are equal to or are the minimum.
+- Readability can be improved through comments or renaming.
+
+### 4. Potential Bugs
+
+- No obvious bugs currently, but returning an empty array for empty input is expected behavior.
+
+### 5. Non-compliance with Best Practices
+
+- The process is relatively inefficient, not using Go language features to optimize performance.
+- Missing comments or documentation.
+
+---
+
+## Optimization Suggestions
+
+If the goal is indeed to return all elements greater than some element in the array (excluding the minimum element), it can be changed to:
+
+- Find the minimum element minVal in the array.
+- Iterate through the array, adding elements greater than minVal to the result.
+- This reduces time complexity to O(n).
+
+Optimized code:
+
+```go
+func processData(data []int) []int {
+ if len(data) == 0 {
+ return nil
+ }
+
+ minVal := data[0]
+ for _, v := range data[1:] {
+ if v < minVal {
+ minVal = v
+ }
+ }
+
+ result := make([]int, 0, len(data))
+ for _, v := range data {
+ if v > minVal {
+ result = append(result, v)
+ }
+ }
+
+ return result
+}
+```
+
+### Optimization Points Explained
+
+- Use one traversal to determine minimum value, O(n).
+- Another traversal to construct result, O(n).
+- Properly use slice capacity initialization to reduce memory allocation.
+- Code intent is clear and performance improvement is significant.
+
+---
+
+If you expect different logic, please let me know for further adjustments.
+
+=== Round 1 - CodeOptimizer ===
+
+```go
+// processData returns all values in the input slice that are greater than the minimum element.
+// Returns nil if input is empty.
+func processData(data []int) []int {
+ if len(data) == 0 {
+ return nil
+ }
+
+ // Find the minimum value in the slice
+ minVal := data[0]
+ for _, v := range data[1:] {
+ if v < minVal {
+ minVal = v
+ }
+ }
+
+ // Collect all elements greater than the minimum value
+ result := make([]int, 0, len(data))
+ for _, v := range data {
+ if v > minVal {
+ result = append(result, v)
+ }
+ }
+
+ return result
+}
+```
+
+=== Round 1 - ExitController ===
+TERMINATE
+
+=== Round 2 - CodeAnalyzer ===
+EXIT: Code quality has met standards
+
+=== Round 2 - CodeOptimizer ===
+
+```go
+// processData returns all values in the input slice that are greater than the minimum element.
+// Returns nil if input is empty.
+func processData(data []int) []int {
+ if len(data) == 0 {
+ return nil
+ }
+
+ // Find the minimum value in the slice
+ minVal := data[0]
+ for _, v := range data[1:] {
+ if v < minVal {
+ minVal = v
+ }
+ }
+
+ // Collect all elements greater than the minimum value
+ result := make([]int, 0, len(data))
+ for _, v := range data {
+ if v > minVal {
+ result = append(result, v)
+ }
+ }
+
+ return result
+}
+```
+
+=== Round 2 - ExitController ===
+TERMINATE
+
+=== Round 3 - CodeAnalyzer ===
+EXIT: Code quality has met standards
+
+=== Round 3 - CodeOptimizer ===
+
+```go
+// processData returns all values in the input slice that are greater than the minimum element.
+// Returns nil if input is empty.
+func processData(data []int) []int {
+ if len(data) == 0 {
+ return nil
+ }
+
+ minVal := data[0]
+ for _, v := range data[1:] {
+ if v < minVal {
+ minVal = v
+ }
+ }
+
+ result := make([]int, 0, len(data))
+ for _, v := range data {
+ if v > minVal {
+ result = append(result, v)
+ }
+ }
+
+ return result
+}
+```
+
+=== Round 3 - ExitController ===
+TERMINATE
+
+=== Round 4 - CodeAnalyzer ===
+EXIT: Code quality has met standards
+
+=== Round 4 - CodeOptimizer ===
+
+```go
+// processData returns all elements in the slice that are greater than the minimum element.
+// Returns nil if input is empty.
+func processData(data []int) []int {
+ if len(data) == 0 {
+ return nil
+ }
+
+ minVal := data[0]
+ for _, v := range data[1:] {
+ if v < minVal {
+ minVal = v
+ }
+ }
+
+ result := make([]int, 0, len(data))
+ for _, v := range data {
+ if v > minVal {
+ result = append(result, v)
+ }
+ }
+
+ return result
+}
+```
+
+=== Round 4 - ExitController ===
+TERMINATE
+
+=== Round 5 - CodeAnalyzer ===
+EXIT: Code quality has met standards
+
+=== Round 5 - CodeOptimizer ===
+
+```go
+// processData returns all values in the input slice that are greater than the minimum element.
+// Returns nil if input is empty.
+func processData(data []int) []int {
+ if len(data) == 0 {
+ return nil
+ }
+
+ minVal := data[0]
+ for _, v := range data[1:] {
+ if v < minVal {
+ minVal = v
+ }
+ }
+
+ result := make([]int, 0, len(data))
+ for _, v := range data {
+ if v > minVal {
+ result = append(result, v)
+ }
+ }
+
+ return result
+}
+```
-=== 第 1 轮 - ExitController ===
+=== Round 5 - ExitController ===
TERMINATE
+
+Code optimization loop completed!
+
```
+
+
+
## BreakLoop
-Use ADK’s Break Action to stop a loop without affecting outer agents:
+
+In a Loop Agent, when an Agent needs to interrupt the loop execution, you can use the corresponding Break Action provided by ADK.
```go
// BreakLoopAction is a programmatic-only agent action used to prematurely
@@ -496,21 +741,24 @@ type BreakLoopAction struct {
func NewBreakLoopAction(agentName string) *AgentAction {
return &AgentAction{BreakLoop: &BreakLoopAction{
From: agentName,
- }}}
+ }}
+}
```
-Illustration:
+Break Action achieves the interruption purpose without affecting other Agents outside the Loop Agent, while Exit Action immediately interrupts all subsequent Agent execution.
+
+Using the following diagram as an example:
-- If Agent1 emits BreakAction, the Loop Agent stops and Sequential continues to Agent3
-- If Agent1 emits ExitAction, the overall Sequential flow terminates; Agent2 / Agent3 do not run
+- When Agent1 issues a BreakAction, the Loop Agent will be interrupted, and Sequential continues to run Agent3
+- When Agent1 issues an ExitAction, the Sequential execution flow terminates entirely, and neither Agent2 nor Agent3 will run
# ParallelAgent
-## Functionality
+## Features
-ParallelAgent runs multiple sub‑agents concurrently over shared input; all start together and it waits for all to finish. Best for independently processable tasks.
+ParallelAgent allows multiple sub-agents to execute concurrently based on the same input context. All sub-agents start execution simultaneously and wait for all to complete before ending. This pattern is particularly suitable for tasks that can be processed independently in parallel, significantly improving execution efficiency.
@@ -518,39 +766,39 @@ ParallelAgent runs multiple sub‑agents concurrently over shared input; all sta
type ParallelAgentConfig struct {
Name string // Agent name
Description string // Agent description
- SubAgents []Agent // Concurrent sub‑agents
+ SubAgents []Agent // List of sub-agents to execute concurrently
}
func NewParallelAgent(ctx context.Context, config *ParallelAgentConfig) (Agent, error)
```
-Execution rules:
+ParallelAgent execution follows these rules:
-1. Concurrent execution: each sub‑agent runs in its own goroutine
-2. Shared input: all sub‑agents receive the same initial input and context
-3. Wait and aggregate: use sync.WaitGroup to wait for completion; collect outputs and emit in received order
+1. **Concurrent execution**: All sub-agents start simultaneously, executing in parallel in independent goroutines
+2. **Shared input**: All sub-agents receive the same initial input and context
+3. **Wait and result aggregation**: Internally uses sync.WaitGroup to wait for all sub-agents to complete, collecting all sub-agent execution results and outputting them in the order received
-Defaults include:
+Additionally, Parallel internally includes exception handling mechanisms by default:
-- Panic recovery per goroutine
-- Error isolation: one sub‑agent’s error does not affect others
-- Interrupt handling: supports sub‑agent interrupt/resume
+- **Panic recovery**: Each goroutine has independent panic recovery mechanism
+- **Error isolation**: Errors from a single sub-agent do not affect execution of other sub-agents
+- **Interrupt handling**: Supports sub-agent interrupt and resume mechanisms
-Suitable for:
+ParallelAgent is suitable for the following scenarios:
-- Independent task parallelism
-- Multi‑perspective analysis
-- Performance optimization
-- Multi‑expert consultation
+- **Independent task parallel processing**: Multiple unrelated tasks can execute simultaneously
+- **Multi-angle analysis**: Analyzing the same problem from different angles simultaneously
+- **Performance optimization**: Reducing overall execution time through parallel execution
+- **Multi-expert consultation**: Consulting multiple specialized domain Agents simultaneously
## Example
-Analyze a product proposal from four perspectives:
+This example demonstrates how to use ParallelAgent to analyze a product proposal from four different angles simultaneously:
-1. TechnicalAnalyst — technical feasibility
-2. BusinessAnalyst — business value
-3. UXAnalyst — user experience
-4. SecurityAnalyst — security risks
+1. **TechnicalAnalyst**: Technical feasibility analysis
+2. **BusinessAnalyst**: Business value analysis
+3. **UXAnalyst**: User experience analysis
+4. **SecurityAnalyst**: Security risk analysis
```go
package main
@@ -578,18 +826,18 @@ func newChatModel() model.ToolCallingChatModel {
return cm
}
-// 技术分析 Agent
+// Technical analysis Agent
func NewTechnicalAnalystAgent() adk.Agent {
a, err := adk.NewChatModelAgent(context.Background(), &adk.ChatModelAgentConfig{
Name: "TechnicalAnalyst",
- Description: "从技术角度分析内容",
- Instruction: `你是一个技术专家。请从技术实现、架构设计、性能优化等技术角度分析提供的内容。
-重点关注：
-1. 技术可行性
-2. 架构合理性
-3. 性能考量
-4. 技术风险
-5. 实现复杂度`,
+ Description: "Analyzes content from a technical perspective",
+ Instruction: `You are a technical expert. Please analyze the provided content from technical implementation, architecture design, and performance optimization perspectives.
+Focus on:
+1. Technical feasibility
+2. Architecture rationality
+3. Performance considerations
+4. Technical risks
+5. Implementation complexity`,
Model: newChatModel(),
})
if err != nil {
@@ -598,18 +846,18 @@ func NewTechnicalAnalystAgent() adk.Agent {
return a
}
-// 商业分析 Agent
+// Business analysis Agent
func NewBusinessAnalystAgent() adk.Agent {
a, err := adk.NewChatModelAgent(context.Background(), &adk.ChatModelAgentConfig{
Name: "BusinessAnalyst",
- Description: "从商业角度分析内容",
- Instruction: `你是一个商业分析专家。请从商业价值、市场前景、成本效益等商业角度分析提供的内容。
-重点关注：
-1. 商业价值
-2. 市场需求
-3. 竞争优势
-4. 成本分析
-5. 盈利模式`,
+ Description: "Analyzes content from a business perspective",
+ Instruction: `You are a business analysis expert. Please analyze the provided content from business value, market prospects, and cost-effectiveness perspectives.
+Focus on:
+1. Business value
+2. Market demand
+3. Competitive advantages
+4. Cost analysis
+5. Revenue model`,
Model: newChatModel(),
})
if err != nil {
@@ -618,18 +866,18 @@ func NewBusinessAnalystAgent() adk.Agent {
return a
}
-// 用户体验分析 Agent
+// User experience analysis Agent
func NewUXAnalystAgent() adk.Agent {
a, err := adk.NewChatModelAgent(context.Background(), &adk.ChatModelAgentConfig{
Name: "UXAnalyst",
- Description: "从用户体验角度分析内容",
- Instruction: `你是一个用户体验专家。请从用户体验、易用性、用户满意度等角度分析提供的内容。
-重点关注：
-1. 用户友好性
-2. 操作便利性
-3. 学习成本
-4. 用户满意度
-5. 可访问性`,
+ Description: "Analyzes content from a user experience perspective",
+ Instruction: `You are a user experience expert. Please analyze the provided content from user experience, usability, and user satisfaction perspectives.
+Focus on:
+1. User friendliness
+2. Operational convenience
+3. Learning cost
+4. User satisfaction
+5. Accessibility`,
Model: newChatModel(),
})
if err != nil {
@@ -638,18 +886,18 @@ func NewUXAnalystAgent() adk.Agent {
return a
}
-// 安全分析 Agent
+// Security analysis Agent
func NewSecurityAnalystAgent() adk.Agent {
a, err := adk.NewChatModelAgent(context.Background(), &adk.ChatModelAgentConfig{
Name: "SecurityAnalyst",
- Description: "从安全角度分析内容",
- Instruction: `你是一个安全专家。请从信息安全、数据保护、隐私合规等安全角度分析提供的内容。
-重点关注：
-1. 数据安全
-2. 隐私保护
-3. 访问控制
-4. 安全漏洞
-5. 合规要求`,
+ Description: "Analyzes content from a security perspective",
+ Instruction: `You are a security expert. Please analyze the provided content from information security, data protection, and privacy compliance perspectives.
+Focus on:
+1. Data security
+2. Privacy protection
+3. Access control
+4. Security vulnerabilities
+5. Compliance requirements`,
Model: newChatModel(),
})
if err != nil {
@@ -661,54 +909,54 @@ func NewSecurityAnalystAgent() adk.Agent {
func main() {
ctx := context.Background()
- // 创建四个不同角度的分析 Agent
+ // Create four analysis Agents from different angles
techAnalyst := NewTechnicalAnalystAgent()
bizAnalyst := NewBusinessAnalystAgent()
uxAnalyst := NewUXAnalystAgent()
secAnalyst := NewSecurityAnalystAgent()
- // 创建 ParallelAgent，同时进行多角度分析
+ // Create ParallelAgent for simultaneous multi-angle analysis
parallelAgent, err := adk.NewParallelAgent(ctx, &adk.ParallelAgentConfig{
Name: "MultiPerspectiveAnalyzer",
- Description: "多角度并行分析：技术 + 商业 + 用户体验 + 安全",
+ Description: "Multi-angle parallel analysis: Technical + Business + User Experience + Security",
SubAgents: []adk.Agent{techAnalyst, bizAnalyst, uxAnalyst, secAnalyst},
})
if err != nil {
log.Fatal(err)
}
- // 创建 Runner
+ // Create Runner
runner := adk.NewRunner(ctx, adk.RunnerConfig{
Agent: parallelAgent,
})
- // 要分析的产品方案
+ // Product proposal to analyze
productProposal := `
-产品方案：智能客服系统
-
-概述：开发一个基于大语言模型的智能客服系统，能够自动回答用户问题，处理常见业务咨询，并在必要时转接人工客服。
-
-主要功能：
-1. 自然语言理解和回复
-2. 多轮对话管理
-3. 知识库集成
-4. 情感分析
-5. 人工客服转接
-6. 对话历史记录
-7. 多渠道接入（网页、微信、APP）
-
-技术架构：
-- 前端：React + TypeScript
-- 后端：Go + Gin 框架
-- 数据库：PostgreSQL + Redis
-- AI模型：GPT-4 API
-- 部署：Docker + Kubernetes
+Product Proposal: Intelligent Customer Service System
+
+Overview: Develop an intelligent customer service system based on large language models that can automatically answer user questions, handle common business inquiries, and transfer to human agents when necessary.
+
+Main Features:
+1. Natural language understanding and response
+2. Multi-turn conversation management
+3. Knowledge base integration
+4. Sentiment analysis
+5. Human agent transfer
+6. Conversation history recording
+7. Multi-channel access (Web, WeChat, App)
+
+Technical Architecture:
+- Frontend: React + TypeScript
+- Backend: Go + Gin framework
+- Database: PostgreSQL + Redis
+- AI Model: GPT-4 API
+- Deployment: Docker + Kubernetes
`
- fmt.Println("开始多角度并行分析...")
- iter := runner.Query(ctx, "请分析以下产品方案：\n"+productProposal)
+ fmt.Println("Starting multi-angle parallel analysis...")
+ iter := runner.Query(ctx, "Please analyze the following product proposal:\n"+productProposal)
- // 使用 map 来收集不同分析师的结果
+ // Use map to collect results from different analysts
results := make(map[string]string)
var mu sync.Mutex
@@ -719,7 +967,7 @@ func main() {
}
if event.Err != nil {
- log.Printf("分析过程中出现错误: %v", event.Err)
+ log.Printf("Error during analysis: %v", event.Err)
continue
}
@@ -728,21 +976,21 @@ func main() {
results[event.AgentName] = event.Output.MessageOutput.Message.Content
mu.Unlock()
- fmt.Printf("\n=== %s 分析完成 ===\n", event.AgentName)
+ fmt.Printf("\n=== %s analysis completed ===\n", event.AgentName)
}
}
- // 输出所有分析结果
+ // Output all analysis results
fmt.Println("\n" + "============================================================")
- fmt.Println("多角度分析结果汇总")
+ fmt.Println("Multi-angle Analysis Results Summary")
fmt.Println("============================================================")
analysisOrder := []string{"TechnicalAnalyst", "BusinessAnalyst", "UXAnalyst", "SecurityAnalyst"}
analysisNames := map[string]string{
- "TechnicalAnalyst": "技术分析",
- "BusinessAnalyst": "商业分析",
- "UXAnalyst": "用户体验分析",
- "SecurityAnalyst": "安全分析",
+ "TechnicalAnalyst": "Technical Analysis",
+ "BusinessAnalyst": "Business Analysis",
+ "UXAnalyst": "User Experience Analysis",
+ "SecurityAnalyst": "Security Analysis",
}
for _, agentName := range analysisOrder {
@@ -753,33 +1001,265 @@ func main() {
}
}
- fmt.Println("\n多角度并行分析完成！")
- fmt.Printf("共收到 %d 个分析结果\n", len(results))
+ fmt.Println("\nMulti-angle parallel analysis completed!")
+ fmt.Printf("Received %d analysis results\n", len(results))
}
```
-Run result (excerpt):
+Run result:
```markdown
-开始多角度并行分析...
+Starting multi-angle parallel analysis...
-=== BusinessAnalyst 分析完成 ===
+=== BusinessAnalyst analysis completed ===
-=== UXAnalyst 分析完成 ===
+=== UXAnalyst analysis completed ===
-=== SecurityAnalyst 分析完成 ===
+=== SecurityAnalyst analysis completed ===
-=== TechnicalAnalyst 分析完成 ===
+=== TechnicalAnalyst analysis completed ===
============================================================
-多角度分析结果汇总
+Multi-angle Analysis Results Summary
============================================================
-【技术分析】
-针对该智能客服系统方案，下面从技术实现、架构设计及性能优化等角度进行详细分析：
-...
+【Technical Analysis】
+For this intelligent customer service system proposal, here is a detailed analysis from technical implementation, architecture design, and performance optimization perspectives:
+
+---
+
+### I. Technical Feasibility
+
+1. **Natural Language Understanding and Response**
+ - Using GPT-4 API for natural language understanding and automatic response is a mature and feasible solution. GPT-4 has strong language understanding and generation capabilities, suitable for handling complex and diverse questions.
+
+2. **Multi-turn Conversation Management**
+ - Relies on backend to maintain context state, combined with GPT-4 model can handle multi-turn interactions well. Need to design reasonable context management mechanism (such as conversation history maintenance, key slot extraction, etc.) to ensure context information integrity.
+
+3. **Knowledge Base Integration**
+ - Can add specific knowledge base retrieval results to GPT-4 API (retrieval-augmented generation), or integrate knowledge base through local retrieval interface. Technically feasible, but has high requirements for real-time and accuracy.
+
+4. **Sentiment Analysis**
+ - Sentiment analysis function can be implemented with independent lightweight models (such as fine-tuned BERT), or try using GPT-4 output, but cost is higher. Sentiment analysis capability helps intelligent customer service better understand user emotions and improve user experience.
+
+5. **Human Agent Transfer**
+ - Technically achievable through establishing event trigger rules (such as turn count, emotion threshold, keyword detection) to implement automatic transfer to human. System needs to support ticket or session transfer mechanism and ensure seamless session switching.
+
+6. **Multi-channel Access**
+ - Multi-channel access including web, WeChat, App can all be achieved through unified API gateway, technology is mature, while needing to handle channel differences (message format, authentication, push mechanism, etc.).
+
+---
+
+### II. Architecture Rationality
+
+- **Frontend React + TypeScript**
+ Very suitable for building responsive customer service interface, mature ecosystem, convenient for multi-channel component sharing.
+
+- **Backend Go + Gin**
+ Go language has excellent performance, Gin framework is lightweight and high-performance, suitable for high-concurrency scenarios. Backend handles GPT-4 API integration, state management, multi-channel message forwarding and other responsibilities, reasonable choice.
+
+- **Database PostgreSQL + Redis**
+ - PostgreSQL handles structured data storage, such as user information, conversation history, knowledge base metadata.
+ - Redis handles session state caching, hot knowledge base, rate limiting, etc., improving access performance.
+ Architecture design follows common large internet product patterns, with clear component division.
+
+- **AI Model GPT-4 API**
+ Using mature API reduces development difficulty and model maintenance cost; disadvantage is high dependency on network and API calls.
+
+- **Deployment Docker + Kubernetes**
+ Containerization and K8s orchestration ensure system elastic scaling, high availability and canary deployment, suitable for production environment, follows modern microservices architecture trends.
+
+---
+
+### III. Performance Considerations
+
+1. **Response Time**
+ - GPT-4 API calls have inherent latency (usually hundreds of milliseconds to 1 second), significantly affecting response time. Need to handle interface asynchronously and design frontend experience well (such as loading animations, partial progressive response).
+
+2. **Concurrent Processing Capability**
+ - Backend Go has high concurrent processing advantages, combined with Redis caching hot data, can greatly improve overall throughput.
+ - But GPT-4 API calls are limited by OpenAI service QPS limits and call costs, need to reasonably design call frequency and degradation strategies.
+
+3. **Caching Strategy**
+ - Cache user conversation context and common question answers to reduce repeated API calls.
+ - Match key questions locally first, call GPT-4 only on failure, improving efficiency.
+
+4. **Multi-channel Load Balancing**
+ - Need to design unified message bus and reliable async queue to prevent traffic spikes from one channel affecting overall system stability.
+
+---
+
+### IV. Technical Risks
+
+1. **GPT-4 API Dependency**
+ - High dependency on third-party API, risks include service interruption, interface changes and cost fluctuations.
+ - Recommend designing local cache and limited alternative response logic to handle API exceptions.
+
+2. **Multi-turn Conversation Context Management Difficulty**
+ - Context too long or complex will reduce answer quality, need to design context length limits and selective important information retention mechanism.
+
+3. **Knowledge Base Integration Complexity**
+ - How to achieve knowledge base and...
+----------------------------------------
+
+【Business Analysis】
+Here is the business perspective analysis of the intelligent customer service system product proposal:
+
+1. Business Value
+- Improve customer service efficiency: Automatically answer user questions and common inquiries, reduce human agent pressure, lower labor costs.
+- Improve user experience: Multi-turn conversation and sentiment analysis make interactions more natural, enhance customer satisfaction and stickiness.
+- Data-driven decision support: Conversation history and knowledge base integration provide valuable user feedback and behavior data for enterprises, optimizing products and services.
+- Support business expansion: Multi-channel access (web, WeChat, App) meets different customer access habits, improving coverage.
+
+2. Market Demand
+- Market demand for intelligent customer service continues to grow, especially in e-commerce, finance, healthcare, education and other industries, customer service automation is an important direction for enterprise digital transformation.
+- With the maturity of AI technology, enterprises expect to use large language models to improve customer service intelligence level.
+- Users' demand for instant response and 24/7 service is increasing, driving widespread adoption of intelligent customer service systems.
+
+3. Competitive Advantages
+- Using advanced GPT-4 large language model, has strong natural language understanding and generation capabilities, improving Q&A accuracy and conversation naturalness.
+- Sentiment analysis function helps accurately identify user emotions, dynamically adjust response strategies, improve customer satisfaction.
+- Multi-channel access design meets enterprise diversified customer reach needs, enhancing product applicability.
+- Technical architecture uses microservices, containerized deployment, convenient for elastic scaling and maintenance, improving system stability and scalability.
+
+4. Cost Analysis
+- AI model call cost is high, depends on GPT-4 API, need to adjust budget based on call volume and response speed.
+- Technical R&D investment is large, involving frontend and backend, multi-channel integration, AI and knowledge base management.
+- Operation and server costs need to consider multi-channel concurrent access.
+- In the long term, human agent count can be significantly reduced, saving labor costs.
+- Can reduce initial hardware investment through cloud services, but cloud resource usage needs careful management to control costs.
+
+5. Revenue Model
+- SaaS subscription service: Charge monthly/yearly service fees to enterprise customers, tiered pricing based on access channels, concurrency, and feature levels.
+- Charge by call count or conversation count, suitable for customers with large business fluctuations.
+- Value-added services: Data analysis report customization, industry knowledge base integration, human agent collaboration tools, etc.
+- For medium and large customers, can provide custom development and technical support, charging project fees.
+- Through continuous model and service optimization, increase customer retention and renewal rates.
+
+In summary, this intelligent customer service system based on mature technology and AI advantages has good business value and market potential. Its multi-channel access and sentiment analysis features enhance competitiveness, but need to reasonably control AI call costs and operating expenses. Recommend focusing on SaaS subscription and value-added services, combined with marketing, quickly capture customer resources and improve profitability.
+----------------------------------------
+
+【User Experience Analysis】
+For this intelligent customer service system proposal, I will analyze from user experience, usability, user satisfaction and accessibility perspectives:
+
+1. User Friendliness
+- Natural language understanding and response capability improves user communication experience with the system, allowing users to express needs in natural language, reducing communication barriers.
+- Multi-turn conversation management allows the system to understand context, reducing repeated explanations, enhancing conversation coherence, further improving user experience.
+- Sentiment analysis function helps the system identify user emotions, making more thoughtful responses, improving interaction personalization and humanization.
+- Multi-channel access covers users' commonly used access paths, convenient for users to get service anytime anywhere, improving friendliness.
+
+2. Operational Convenience
+- Automatically answering common business inquiries can reduce user waiting time and operational burden, improving response speed.
+- Human agent transfer mechanism ensures complex issues can be handled timely, ensuring service continuity and seamless operation handoff.
+- Conversation history recording convenient for users to review consultation content, avoiding repeated queries, improving operational convenience.
+- Using modern tech stack (React, TypeScript) provides good frontend interaction performance and response speed, indirectly enhancing operational smoothness.
+
+3. Learning Cost
+- Based on natural language processing, users don't need to learn special commands, lowering usage threshold.
+- Multi-turn conversation natural connection makes it easier for users to understand system response logic, reducing confusion and frustration.
+- Consistent interface across different channels (such as keeping similar experience on web and WeChat) helps users get started quickly.
+- More precise feedback provided through sentiment analysis reduces time cost of users frequently trying due to misunderstanding.
+
+4. User Satisfaction
+- Fast and accurate automatic replies and multi-turn conversation reduce user waiting and repeated input, improving satisfaction.
+- Sentiment analysis makes the system better understand user emotions, bringing warmer interaction experience, increasing user stickiness.
+- Human agent intervention ensures complex issues are properly handled, improving service quality perception.
+- Multi-channel coverage meets different users' usage scenarios, enhancing overall satisfaction.
+
+5. Accessibility
+- Multi-channel access covers web, WeChat, App, adapting to different users' devices and environments, improving accessibility.
+- The proposal doesn't explicitly mention accessibility design (such as screen reader compatibility, high contrast mode, etc.), which may be an area to supplement in the future.
+- Frontend using React and TypeScript is conducive to implementing responsive design and accessibility features, but need to ensure development standards are implemented.
+- Backend architecture and deployment solution ensure system stability and scalability, indirectly improving user continuous accessibility.
+
+Summary:
+This intelligent customer service system proposal is fairly comprehensive in user experience and usability considerations, using large language models to achieve natural multi-turn conversation, sentiment analysis and knowledge base integration, meeting users' diverse needs. Meanwhile, multi-channel access enhances system coverage. Recommend strengthening accessibility design in specific implementation to achieve more comprehensive accessibility assurance, while continuing to optimize conversation strategies to improve user satisfaction.
+----------------------------------------
+
+【Security Analysis】
+For this intelligent customer service system proposal, here is the analysis from information security, data protection and privacy compliance perspectives:
+
+I. Data Security
+
+1. Data Transmission Security
+- Recommend all client-server communications use TLS/SSL encryption to ensure data confidentiality and integrity during transmission.
+- Since multi-channel access is supported (web, WeChat, App), need to ensure each entry point strictly implements encrypted transmission.
+
+2. Data Storage Security
+- PostgreSQL stores sensitive information like conversation history and user data, need to enable database encryption (such as transparent data encryption TDE or field-level encryption) to prevent data leakage.
+- Redis as cache may store temporary session data, also need to enable access authentication and encrypted transmission.
+- Implement minimum storage principle for user sensitive data, avoid storing unrelated data beyond scope.
+- Data backup process needs encrypted storage, and backup access should also be controlled.
+
+3. API Call Security
+- GPT-4 API calls generate large amounts of user data interaction, should evaluate its data processing and storage policies to ensure compliance with data security requirements.
+- Add call permission management, limit API key access scope and permissions to prevent abuse.
+
+4. Log Security
+- System logs should avoid storing plaintext sensitive information, especially personal identity information and conversation content. Log access needs strict control.
+
+II. Privacy Protection
+
+1. Personal Data Processing
+- Collection and storage of user personal data (name, contact information, account information, etc.) must clearly inform users and obtain user consent.
+- Implement data anonymization/de-identification technology, especially for identity information processing in conversation history.
+
+2. User Privacy Rights
+- Meet users' rights to access, correct, and delete data in relevant laws and regulations (such as Personal Information Protection Law, GDPR).
+- Provide privacy policy clearly disclosing data collection, use and sharing situations.
+
+3. Interaction Privacy
+- Multi-turn conversation and sentiment analysis features should consider avoiding excessive invasion of user privacy, such as transparent notification and restriction of sensitive emotion data usage.
+
+4. Third-party Compliance
+- GPT-4 API is provided by third party, need to ensure its service complies with relevant privacy compliance requirements and data protection standards.
+
+III. Access Control
+
+1. User Identity Verification
+- When system involves user identity information query and management, need to establish reliable identity authentication mechanism.
+- Support multi-factor authentication to enhance security.
+
+2. Permission Management
+- Backend management interface and human agent transfer module need to use role-based access control (RBAC) to ensure minimum operation permissions.
+- Operations accessing sensitive data need detailed audit and monitoring.
+
+3. Session Management
+- Need effective session management mechanism for multi-channel sessions to prevent session hijacking.
+- Conversation history access permissions should be limited to only relevant users or authorized personnel.
+
+IV. Security Vulnerabilities
+
+1. Application Security
+- Frontend React+TypeScript should prevent XSS, CSRF attacks, reasonably use Content Security Policy (CSP).
+- Backend Go application needs to prevent SQL injection, request forgery and permission deficiency. Gin framework provides middleware support, recommend fully utilizing security modules.
+
+2. AI Model Risks
+- GPT-4 API input/output may have sensitive information leakage or model misuse risks, need to limit input content and filter sensitive information.
+- Prevent generating malicious answers or information leakage, establish content review mechanism.
+
+3. Container and Deployment Security
+- Docker containers must use secure images and patch timely. Kubernetes cluster network policies and access control need to be complete.
+- Container runtime permissions minimized to avoid container escape risks.
+
+V. Compliance Requirements
+
+1. Data Protection Regulations
+- Based on operating region, need to comply with Personal Information Protection Law (PIPL), EU General Data Protection Regulation (GDPR) or other relevant legal requirements.
+- Clearly define user data collection, processing, transmission and storage processes comply with regulations.
+
+2. User Privacy Notice and Consent
+- Should provide clear privacy policy and terms of use, explaining data purposes and processing methods.
+- Implement user consent management mechanism.
+
+3. Cross-border Data Transfer Compliance
+- If system involves cross-border data flow, need to assess compliance risks and take corresponding technical...
+----------------------------------------
+
+Multi-angle parallel analysis completed!
+Received 4 analysis results
```
# Summary
-Workflow Agents provide robust multi‑agent collaboration in Eino ADK. By choosing and composing these agents appropriately, developers can build efficient, reliable multi‑agent systems for complex business needs.
+Workflow Agents provide powerful multi-agent collaboration capabilities for Eino ADK. By reasonably selecting and combining these Workflow Agents, developers can build efficient and reliable multi-agent collaboration systems to meet various complex business requirements.
diff --git a/content/en/docs/eino/core_modules/eino_adk/agent_interface.md b/content/en/docs/eino/core_modules/eino_adk/agent_interface.md
index d26bfe2b9a0..619d2ad330a 100644
--- a/content/en/docs/eino/core_modules/eino_adk/agent_interface.md
+++ b/content/en/docs/eino/core_modules/eino_adk/agent_interface.md
@@ -1,15 +1,15 @@
---
Description: ""
-date: "2025-12-09"
+date: "2026-01-20"
lastmod: ""
tags: []
title: 'Eino ADK: Agent Interface'
weight: 3
---
-## Agent Definition
+# Agent Definition
-Implementing the following interface makes a struct an agent:
+Eino defines a basic interface for Agents. Any struct implementing this interface can be considered an Agent:
```go
// github.com/cloudwego/eino/adk/interface.go
@@ -23,13 +23,15 @@ type Agent interface {
| Method | Description |
| Name | Agent identifier (name) |
| Description | Capabilities description to help other agents understand its role |
| Run | Core execution method; returns an iterator to continuously receive Agent events |
| Name | The name of the Agent, serving as its identifier |
| Description | Description of the Agent's capabilities, mainly used to help other Agents understand and determine this Agent's responsibilities or functions |
| Run | The core execution method of the Agent, returns an iterator through which the caller can continuously receive events produced by the Agent |
## AgentRunOption
-Options can adjust configuration or behavior per request. ADK provides common options:
+`AgentRunOption` is defined by the Agent implementation and can modify Agent configuration or control Agent behavior at the request level.
+
+Eino ADK provides some commonly defined Options for users:
+
+- `WithSessionValues`: Set cross-Agent read/write data
+- `WithSkipTransferMessages`: When configured, when the Event is Transfer SubAgent, the messages in the Event will not be appended to History
-- `WithSessionValues` — set cross‑agent KV state
-- `WithSkipTransferMessages` — when transferring, do not append transfer event messages to history
+Eino ADK provides two methods `WrapImplSpecificOptFn` and `GetImplSpecificOptions` for Agents to wrap and read custom `AgentRunOption`.
-Wrapping and reading implementation‑specific options:
+When using the `GetImplSpecificOptions` method to read `AgentRunOptions`, AgentRunOptions that don't match the required type (like options in the example) will be ignored.
+
+For example, you can define `WithModelName` to require the Agent to change the model being called at the request level:
```go
// github.com/cloudwego/eino/adk/call_option.go
// func WrapImplSpecificOptFn[T any](optFn func(*T)) AgentRunOption
// func GetImplSpecificOptions[T any](base *T, opts ...AgentRunOption) *T
-type options struct { modelName string }
+import "github.com/cloudwego/eino/adk"
+
+type options struct {
+ modelName string
+}
func WithModelName(name string) adk.AgentRunOption {
- return adk.WrapImplSpecificOptFn(func(t *options) { t.modelName = name })
+ return adk.WrapImplSpecificOptFn(func(t *options) {
+ t.modelName = name
+ })
}
func (m *MyAgent) Run(ctx context.Context, input *adk.AgentInput, opts ...adk.AgentRunOption) *adk.AsyncIterator[*adk.AgentEvent] {
@@ -95,42 +109,61 @@ func (m *MyAgent) Run(ctx context.Context, input *adk.AgentInput, opts ...adk.Ag
}
```
-Designate option targets for specific agents in a multi‑agent system:
+Additionally, AgentRunOption has a `DesignateAgent` method. Calling this method allows you to specify which Agents the Option takes effect on when calling a multi-Agent system:
```go
-opt := adk.WithSessionValues(map[string]any{}).DesignateAgent("agent_1", "agent_2")
+func genOpt() {
+ // Specify that the option only takes effect for agent_1 and agent_2
+ opt := adk.WithSessionValues(map[string]any{}).DesignateAgent("agent_1", "agent_2")
+}
```
## AsyncIterator
-`Agent.Run` returns `AsyncIterator[*AgentEvent]`, an asynchronous iterator (production and consumption are decoupled) for consuming events in order:
+`Agent.Run` returns an iterator `AsyncIterator[*AgentEvent]`:
```go
// github.com/cloudwego/eino/adk/utils.go
-type AsyncIterator[T any] struct { /* ... */ }
+type AsyncIterator[T any] struct {
+ ...
+}
-func (ai *AsyncIterator[T]) Next() (T, bool) { /* ... */ }
+func (ai *AsyncIterator[T]) Next() (T, bool) {
+ ...
+}
```
-Consume with a blocking `Next()` loop until closed:
+It represents an asynchronous iterator (asynchronous means there is no synchronization control between production and consumption), allowing the caller to consume a series of events produced by the Agent during execution in an ordered, blocking manner.
+
+- `AsyncIterator` is a generic struct that can be used to iterate over any type of data. Currently in the Agent interface, the iterator type returned by the Run method is fixed as `AsyncIterator[*AgentEvent]`. This means that every element you get from this iterator will be a pointer to an `AgentEvent` object. `AgentEvent` will be explained in detail in the following sections.
+- The main way to interact with the iterator is by calling its `Next()` method. This method's behavior is blocking. Each time you call `Next()`, the program pauses execution until one of the following two situations occurs:
+ - The Agent produces a new `AgentEvent`: The `Next()` method returns this event, and the caller can process it immediately.
+ - The Agent actively closes the iterator: When the Agent will no longer produce any new events (usually when the Agent finishes running), it closes this iterator. At this point, the `Next()` call ends blocking and returns false in the second return value, informing the caller that iteration has ended.
+
+Typically, you need to use a for loop to process `AsyncIterator`:
```go
-iter := myAgent.Run(xxx)
+iter := myAgent.Run(xxx) // get AsyncIterator from Agent.Run
+
for {
event, ok := iter.Next()
- if !ok { break }
+ if !ok {
+ break
+ }
// handle event
}
```
-Create with `NewAsyncIteratorPair` and produce via `AsyncGenerator`:
+`AsyncIterator` can be created by `NewAsyncIteratorPair`. The other parameter returned by this function, `AsyncGenerator`, is used to produce data:
```go
+// github.com/cloudwego/eino/adk/utils.go
+
func NewAsyncIteratorPair[T any]() (*AsyncIterator[T], *AsyncGenerator[T])
```
-Agents usually run in a goroutine and return the iterator immediately, so the caller can start consuming events in real time:
+Agent.Run returns AsyncIterator to allow the caller to receive a series of AgentEvents produced by the Agent in real-time. Therefore, Agent.Run usually runs the Agent in a Goroutine to immediately return the AsyncIterator for the caller to listen to:
```go
import "github.com/cloudwego/eino/adk"
@@ -152,31 +185,37 @@ func (m *MyAgent) Run(ctx context.Context, input *adk.AgentInput, opts ...adk.Ag
## AgentWithOptions
-Configure common behaviors before running via `AgentWithOptions`:
+Using the `AgentWithOptions` method allows you to perform some common configurations in Eino ADK Agent.
+
+Unlike `AgentRunOption`, `AgentWithOptions` takes effect before running and does not support custom options.
```go
// github.com/cloudwego/eino/adk/flow.go
func AgentWithOptions(ctx context.Context, agent Agent, opts ...AgentOption) Agent
```
-Built‑in options:
+Currently built-in configurations supported by Eino ADK:
-- `WithDisallowTransferToParent` — disallow transferring to parent; triggers `OnDisallowTransferToParent`
-- `WithHistoryRewriter` — rewrite history into input messages before execution
+- `WithDisallowTransferToParent`: Configure that this SubAgent is not allowed to Transfer to ParentAgent, which will trigger the SubAgent's `OnDisallowTransferToParent` callback method
+- `WithHistoryRewriter`: When configured, this Agent will rewrite the received context information through this method before execution
# AgentEvent
-Core event structure produced by agents:
+AgentEvent is the core event data structure produced by the Agent during its execution. It contains the Agent's meta information, output, behavior, and errors:
```go
// github.com/cloudwego/eino/adk/interface.go
type AgentEvent struct {
AgentName string
- RunPath []RunStep
- Output *AgentOutput
- Action *AgentAction
- Err error
+
+ RunPath []RunStep
+
+ Output *AgentOutput
+
+ Action *AgentAction
+
+ Err error
}
// EventFromMessage builds a standard event
@@ -185,48 +224,64 @@ func EventFromMessage(msg Message, msgStream MessageStream, role schema.RoleType
## AgentName & RunPath
-Filled by the framework to provide event provenance in multi‑agent systems:
+The `AgentName` and `RunPath` fields are automatically filled by the framework. They provide important context information about the event source, which is crucial in complex systems composed of multiple Agents.
```go
-type RunStep struct { agentName string }
+type RunStep struct {
+ agentName string
+}
```
-- `AgentName` — which agent produced the event
-- `RunPath` — chain from entry agent to current agent
+- `AgentName` indicates which Agent instance produced the current AgentEvent.
+- `RunPath` records the complete call chain to reach the current Agent. `RunPath` is a slice of `RunStep` that records all `AgentName`s in order from the initial entry Agent to the Agent that produced the current event.
## AgentOutput
-Encapsulates agent output:
+`AgentOutput` encapsulates the output produced by the Agent.
+
+Message output is set in the MessageOutput field, while other types of custom output are set in the CustomizedOutput field:
```go
+// github.com/cloudwego/eino/adk/interface.go
+
type AgentOutput struct {
- MessageOutput *MessageVariant
+ MessageOutput *MessageVariant
+
CustomizedOutput any
}
type MessageVariant struct {
- IsStreaming bool
+ IsStreaming bool
+
Message Message
MessageStream MessageStream
- Role schema.RoleType
- ToolName string // when Role is Tool
+ // message role: Assistant or Tool
+ Role schema.RoleType
+ // only used when Role is Tool
+ ToolName string
}
```
-`MessageVariant`:
+The type `MessageVariant` of the `MessageOutput` field is a core data structure with the main functions of:
+
+1. Unified handling of streaming and non-streaming messages: `IsStreaming` is a flag. A value of true indicates that the current `MessageVariant` contains a streaming message (read from MessageStream), while false indicates it contains a non-streaming message (read from Message):
+
+ - Streaming: Returns a series of message fragments over time that eventually form a complete message (MessageStream).
+ - Non-streaming: Returns a complete message at once (Message).
+2. Providing convenient metadata access: The Message struct contains some important meta information internally, such as the message's Role (Assistant or Tool). To quickly identify message types and sources, MessageVariant elevates these commonly used metadata to the top level:
-1. Unifies streaming vs non‑streaming messages via `IsStreaming`:
- - Streaming: return chunks over time that form a complete message (read from `MessageStream`).
- - Non‑streaming: return a complete message at once (read from `Message`).
-2. Surfaces convenient metadata at top level:
- - `Role`: Assistant or Tool
- - `ToolName`: when `Role` is Tool, provide the tool’s name
+ - `Role`: The role of the message, Assistant / Tool
+ - `ToolName`: If the message role is Tool, this field directly provides the tool's name.
+
+The benefit of this is that when code needs to route or make decisions based on message type, it doesn't need to deeply parse the specific content of the Message object. It can directly get the required information from the top-level fields of MessageVariant, thus simplifying the logic and improving code readability and efficiency.
## AgentAction
-Control multi‑agent collaboration: exit, interrupt, transfer, or custom:
+An Agent producing an Event containing AgentAction can control multi-Agent collaboration, such as immediate exit, interruption, transfer, etc.:
```go
+// github.com/cloudwego/eino/adk/interface.go
+
type AgentAction struct {
Exit bool
@@ -239,27 +294,42 @@ type AgentAction struct {
CustomizedAction any
}
-type InterruptInfo struct { Data any }
+type InterruptInfo struct {
+ Data any
+}
+
+type TransferToAgentAction struct {
+ DestAgentName string
+}
+```
+
+Eino ADK currently has four preset Actions:
+
+1. Exit: When an Agent produces an Exit Action, the Multi-Agent will exit immediately
-type TransferToAgentAction struct { DestAgentName string }
+```go
+func NewExitAction() *AgentAction {
+ return &AgentAction{Exit: true}
+}
```
-Prebuilt actions:
+2. Transfer: When an Agent produces a Transfer Action, it will transfer to the target Agent to run
```go
-func NewExitAction() *AgentAction { return &AgentAction{Exit: true} }
-func NewTransferToAgentAction(dest string) *AgentAction {
- return &AgentAction{TransferToAgent: &TransferToAgentAction{DestAgentName: dest}}
+func NewTransferToAgentAction(destAgentName string) *AgentAction {
+ return &AgentAction{TransferToAgent: &TransferToAgentAction{DestAgentName: destAgentName}}
}
```
-Interrupt sends custom info for checkpoint/resume flows (see Runner docs). For example, ChatModelAgent sends an interrupt event as:
+3. Interrupt: When an Agent produces an Interrupt Action, it will interrupt the Runner's execution. Since interruption can occur at any position and unique information needs to be passed out during interruption, the Action provides an `Interrupted` field for the Agent to set custom data. When the Runner receives an Action with non-empty Interrupted, it considers an interruption has occurred. The internal mechanism of Interrupt & Resume is relatively complex and will be detailed in the [Eino ADK: Agent Runner] - [Eino ADK: Interrupt & Resume] section.
```go
-// e.g., when ChatModelAgent interrupts, it emits:
+// For example, when ChatModelAgent interrupts, it sends the following AgentEvent:
h.Send(&AgentEvent{AgentName: h.agentName, Action: &AgentAction{
Interrupted: &InterruptInfo{
Data: &ChatModelAgentInterruptInfo{Data: data, Info: info},
},
}})
```
+
+4. Break Loop: When a sub-Agent of LoopAgent emits a BreakLoopAction, the corresponding LoopAgent will stop looping and exit normally.
diff --git a/content/en/docs/eino/core_modules/eino_adk/agent_preview.md b/content/en/docs/eino/core_modules/eino_adk/agent_preview.md
index b3b95c4741e..4dcd230afb4 100644
--- a/content/en/docs/eino/core_modules/eino_adk/agent_preview.md
+++ b/content/en/docs/eino/core_modules/eino_adk/agent_preview.md
@@ -1,25 +1,27 @@
---
Description: ""
-date: "2025-12-11"
+date: "2026-01-20"
lastmod: ""
tags: []
title: 'Eino ADK: Overview'
-weight: 1
+weight: 2
---
# What is Eino ADK?
-Eino ADK, inspired by Google ADK, is a flexible Go framework for building Agents and Multi‑Agent applications. It standardizes context passing, event streaming and conversion, task transfer, interrupts & resume, and cross‑cutting aspects. It is model‑agnostic and deployment‑agnostic, aiming to make Agent and Multi‑Agent development simpler and more robust while offering production‑grade governance capabilities.
+Eino ADK, inspired by [Google-ADK](https://google.github.io/adk-docs/agents/), provides a flexible composition framework for Agent development in Go, i.e., an Agent and Multi-Agent development framework. Eino ADK has accumulated common capabilities for multi-Agent interaction, including context passing, event stream distribution and conversion, task control transfer, interrupt and resume, and common aspects. It is widely applicable, model-agnostic, and deployment-agnostic, making Agent and Multi-Agent development simpler and more convenient while providing comprehensive production-grade application governance capabilities.
-Eino ADK helps developers build and manage agent applications, providing a resilient development environment to support conversational and non‑conversational agents, complex tasks, and workflows.
+Eino ADK aims to help developers develop and manage Agent applications. It provides a flexible and robust development environment to help developers build various Agent applications such as conversational agents, non-conversational agents, complex tasks, workflows, and more.
-# Architecture
+# ADK Framework
+
+The overall module structure of Eino ADK is shown in the diagram below:
## Agent Interface
-The core of ADK is the `Agent` abstraction. See the full details in [Eino ADK: Agent Interface](/docs/eino/core_modules/eino_adk/agent_interface).
+The core of Eino ADK is the Agent abstraction (Agent Interface). All ADK functionality is designed around the Agent abstraction. For details, see [Eino ADK: Agent Interface](/docs/eino/core_modules/eino_adk/agent_interface)
```go
type Agent interface {
@@ -36,57 +38,63 @@ type Agent interface {
}
```
-`Agent.Run`:
+The definition of `Agent.Run` is:
+
+1. Get task details and related data from the input AgentInput, AgentRunOption, and optional Context Session
+2. Execute the task and write the execution process and results to the AgentEvent Iterator
+
+`Agent.Run` requires the Agent implementation to execute asynchronously in a Future pattern. The core is divided into three steps. For specifics, refer to the implementation of the Run method in ChatModelAgent:
-1. Reads task details and related data from `AgentInput`, `AgentRunOption`, and optional session context
-2. Executes the task and writes progress/results into an `AgentEvent` iterator
-3. Requires a future‑style asynchronous execution. In practice (see ChatModelAgent `Run`):
- - Create a pair of Iterator/Generator
- - Start the agent’s async task with the Generator, process `AgentInput` (e.g., call LLM) and emit events into the Generator
- - Return the Iterator immediately to the caller
+1. Create a pair of Iterator and Generator
+2. Start the Agent's asynchronous task and pass in the Generator to process AgentInput. The Agent executes core logic in this asynchronous task (e.g., ChatModelAgent calls LLM) and writes new events to the Generator for the Agent caller to consume from the Iterator
+3. Return the Iterator immediately after starting the task in step 2
-## Collaboration
+## Multi-Agent Collaboration
-ADK provides rich composition primitives to build Multi‑Agent systems: Supervisor, Plan‑Execute, Group‑Chat, etc. See [Eino ADK: Agent Collaboration](/docs/eino/core_modules/eino_adk/agent_collaboration).
+Around the Agent abstraction, Eino ADK provides various simple, easy-to-use composition primitives for rich scenarios, supporting the development of diverse Multi-Agent collaboration strategies such as Supervisor, Plan-Execute, Group-Chat, and other Multi-Agent scenarios. This enables different Agent division of labor and cooperation patterns to handle more complex tasks. For details, see [Eino ADK: Agent Collaboration](/docs/eino/core_modules/eino_adk/agent_collaboration)
-Primitives:
+The collaboration primitives defined by Eino ADK during Agent collaboration are as follows:
+
+- Collaboration methods between Agents
| Collaboration | Description |
| Transfer | Directly transfer the task to another Agent; current Agent exits and does not track the transferred task |
| ToolCall (AgentAsTool) | Treat an Agent as a tool call, wait for its response, consume its output, and continue processing |
| Collaboration Method | Description |
| Transfer | Directly transfer the task to another Agent. The current Agent exits after execution and does not care about the task execution status of the transferred Agent |
| ToolCall(AgentAsTool) | Call an Agent as a ToolCall, wait for the Agent's response, and obtain the output result of the called Agent for the next round of processing |
| Context Strategy | Description |
| Upstream full dialogue | Provide the child Agent with the complete upstream conversation |
| New task description | Ignore upstream conversation and provide a fresh summarized task as the child Agent’s input |
| Upstream Agent Full Dialogue | Get the complete dialogue record of this Agent's upstream Agent |
| New Task Description | Ignore the complete dialogue record of the upstream Agent and provide a new task summary as the sub-Agent's AgentInput |
| Autonomy | Description |
| Autonomous | Inside the Agent, choose downstream Agents as needed (often via LLM). Even if decisions are based on preset logic, from the outside this is treated as autonomous. |
| Preset | Pre‑define the next Agent. Execution order is fixed and predictable. |
| Decision Autonomy | Description |
| Autonomous Decision | Inside the Agent, based on its available downstream Agents, when assistance is needed, autonomously select downstream Agents for assistance. Generally, the Agent makes decisions based on LLM internally, but even if selection is based on preset logic, it is still considered autonomous decision from outside the Agent |
| Preset Decision | Pre-set the next Agent after an Agent executes a task. The execution order of Agents is predetermined and predictable |
| Type | Description | Run Mode | Collaboration | Context | Autonomy |
| SubAgents | Treat a user‑provided Agent as the parent, and its subAgents list as children, forming an autonomously deciding Agent. Name/Description identify the Agent. |  | Transfer | Upstream full dialogue | Autonomous |
| Sequential | Compose SubAgents to execute in order. Name/Description identify the Sequential Agent. Executes subagents sequentially until all finish. |  | Transfer | Upstream full dialogue | Preset |
| Parallel | Compose SubAgents to run concurrently under the same context. Name/Description identify the Parallel Agent. Executes subagents in parallel, ends after all complete. |  | Transfer | Upstream full dialogue | Preset |
| Loop | Compose SubAgents to run in array order, repeat cyclically. Name/Description identify the Loop Agent. Executes subagents in sequence per loop. |  | Transfer | Upstream full dialogue | Preset |
| AgentAsTool | Convert an Agent into a Tool for use by other Agents. Whether an Agent can call other Agents as Tools depends on its implementation. ChatModelAgent supports AgentAsTool. |  | ToolCall | New task description | Autonomous |
| Type | Description | Run Mode | Collaboration Method | Context Strategy | Decision Autonomy |
| SubAgents | Use the user-provided agent as the Parent Agent and the user-provided subAgents list as Child Agents to form an autonomously deciding Agent, where Name and Description serve as the Agent's name identifier and description. | | Transfer | Upstream Agent Full Dialogue | Autonomous Decision |
| Sequential | Combine the user-provided SubAgents list into a Sequential Agent that executes in order, where Name and Description serve as the Sequential Agent's name identifier and description. When the Sequential Agent executes, it runs the SubAgents list in order until all Agents have been executed. | | Transfer | Upstream Agent Full Dialogue | Preset Decision |
| Parallel | Combine the user-provided SubAgents list into a Parallel Agent that executes concurrently based on the same context, where Name and Description serve as the Parallel Agent's name identifier and description. When the Parallel Agent executes, it runs the SubAgents list concurrently and ends after all Agents complete execution. | | Transfer | Upstream Agent Full Dialogue | Preset Decision |
| Loop | Execute the user-provided SubAgents list in array order, cycling repeatedly, to form a Loop Agent, where Name and Description serve as the Loop Agent's name identifier and description. When the Loop Agent executes, it runs the SubAgents list in sequence and ends after all Agents complete execution. | | Transfer | Upstream Agent Full Dialogue | Preset Decision |
| AgentAsTool | Convert an Agent into a Tool to be used by other Agents as a regular Tool. Whether an Agent can call other Agents as Tools depends on its own implementation. The ChatModelAgent provided in Eino ADK supports the AgentAsTool functionality | | ToolCall | New Task Description | Autonomous Decision |
+The table and diagram below provide the characteristics, differences, and relationships of these base extensions and encapsulations. Subsequent chapters will detail the principles and specifics of each type:
| Category | ChatModel Agent | Workflow Agents | Custom Logic | EinoBuiltInAgent (supervisor, plan‑execute) |
| Function | Thinking, generation, tool calls | Control execution flow among agents | Run custom logic | Out‑of‑the‑box multi‑agent patterns |
| Core | LLM | Predetermined flows (sequential/parallel/loop) | Custom code | High‑level encapsulation based on Eino practice |
| Purpose | Generation, dynamic decisions | Structured orchestration | Specific customization | Turnkey solutions for common scenarios |
| Function | Thinking, generation, tool calls | Control execution flow among agents | Run custom logic | Out‑of‑the‑box multi‑agent pattern encapsulation |
| Core | LLM | Predetermined execution flows (sequential/parallel/loop) | Custom code | High‑level encapsulation based on Eino practical experience |
| Purpose | Generation, dynamic decisions | Structured processing, orchestration | Customization needs | Turnkey solutions for specific scenarios |
+
# ADK Examples
-Explore examples in [Eino‑examples](https://github.com/cloudwego/eino-examples/tree/main/adk). The table summarizes project paths, key points, and diagrams:
+The [Eino‑examples](https://github.com/cloudwego/eino-examples/tree/main/adk) project provides various ADK implementation examples. You can refer to the example code and descriptions to build an initial understanding of ADK capabilities:
| Project Path | Intro | Diagram |
| Sequential workflow | This example shows a sequential multi‑agent workflow built with Eino ADK’s Workflow paradigm. |  |
| Loop workflow | Built with LoopAgent to form a reflection‑iteration framework. |  |
| Parallel workflow | Built with ParallelAgent for concurrent data collection. |  |
| supervisor | Single‑layer Supervisor manages two composite subagents: Research Agent (retrieval) and Math Agent (math operations: add/multiply/divide). All math ops are handled by one Math Agent rather than splitting into many; suitable for focused tasks and quick deployment. |  |
| layered‑supervisor | Multi‑tier supervision: top Supervisor manages Research Agent and Math Agent; Math Agent further manages Subtract/Multiply/Divide subagents. |  |
| plan‑execute example | Implements a plan‑execute‑replan travel planner: Planner generates stepwise plan; Executor calls mock tools (get_weather/search_flights/search_hotels/search_attractions/ask_for_clarification); Replanner decides replan or finish. Two layers: |  |
| book recommendation agent (interrupt/resume) | Demonstrates a ChatModel agent with tools and checkpointing. |  |
| Project Path | Introduction | Diagram |
| Sequential workflow example | This example code demonstrates a sequential multi-agent workflow built using Eino ADK's Workflow paradigm. | |
| Loop workflow example | This example code builds a reflection-iteration agent framework based on Eino ADK's Workflow paradigm using LoopAgent. | |
| Parallel workflow example | This example code builds a concurrent information collection framework based on Eino ADK's Workflow paradigm using ParallelAgent: | |
| supervisor | This use case employs a single-layer Supervisor managing two relatively comprehensive sub-Agents: Research Agent handles retrieval tasks, Math Agent handles various mathematical operations (add, multiply, divide), but all math operations are uniformly processed within the same Math Agent rather than being split into multiple sub-Agents. This design simplifies the agent hierarchy, suitable for scenarios where tasks are relatively concentrated and don't require excessive decomposition, facilitating rapid deployment and maintenance. | |
| layered‑supervisor | This use case implements a multi-tier intelligent agent supervision system, where the top-level Supervisor manages Research Agent and Math Agent, and Math Agent is further subdivided into three sub-Agents: Subtract, Multiply, and Divide. The top-level Supervisor is responsible for assigning research tasks and math tasks to lower-level Agents, while Math Agent as a mid-tier supervisor further dispatches specific math operation tasks to its sub-Agents. | |
| plan‑execute example | This example implements a multi-Agent travel planning system using the plan-execute-replan pattern based on Eino ADK. The core function is to process complex user travel requests (such as "3-day Beijing trip, need flights from New York, hotel recommendations, must-see attractions") through a "plan-execute-replan" loop to complete tasks: 1. Plan: Planner Agentgenerates a step-by-step execution plan based on the large model (e.g., "Step 1: check Beijing weather, Step 2: search New York to Beijing flights"); 2. Execute: Executor Agentcalls mock tools **weather (get_weather), flights (search_flights), hotels (search_hotels), attractions (search_attractions)** to execute each step. If user input information is missing (e.g., budget not specified), it calls ask_for_clarificationtool to ask follow-up questions; 3. Replan: Replanner Agentevaluates whether the plan needs adjustment based on tool execution results (e.g., if no flight tickets available, reselect dates). Execute and Replan continuously loop until all steps in the plan are completed; 4. Supports session trajectory tracking (CozeLoop callback) and state management, ultimately outputting a complete travel plan. Structurally, plan-execute-replan has two layers: | |
| book recommendation agent (interrupt and resume) | This code demonstrates a book recommendation chat agent implementation built on the Eino ADK framework, showcasing Agent interrupt and resume functionality. | |
diff --git a/content/en/docs/eino/core_modules/flow_integration_components/multi_agent_hosting.md b/content/en/docs/eino/core_modules/flow_integration_components/multi_agent_hosting.md
index 89543a16f30..c8636f27a76 100644
--- a/content/en/docs/eino/core_modules/flow_integration_components/multi_agent_hosting.md
+++ b/content/en/docs/eino/core_modules/flow_integration_components/multi_agent_hosting.md
@@ -1,17 +1,17 @@
---
Description: ""
-date: "2025-12-09"
+date: "2026-01-20"
lastmod: ""
tags: []
title: 'Eino Tutorial: Host Multi-Agent'
weight: 2
---
-Host Multi-Agent is a pattern where a Host recognizes intent and hands off to a specialist agent to perform the actual generation.
+Host Multi-Agent is a pattern where a Host recognizes intent and hands off to a specialist agent to perform the actual generation. It only forwards requests, without generating subtasks.
-Example: a “journal assistant” that can write journal, read journal, and answer questions based on journal.
+Example: a "journal assistant" that can write journal, read journal, and answer questions based on journal.
-Full sample: https://github.com/cloudwego/eino-examples/tree/main/flow/agent/multiagent/host/journal
+Full sample: [https://github.com/cloudwego/eino-examples/tree/main/flow/agent/multiagent/host/journal](https://github.com/cloudwego/eino-examples/tree/main/flow/agent/multiagent/host/journal)
Host:
@@ -34,7 +34,7 @@ func newHost(ctx context.Context, baseURL, apiKey, modelName string) (*host.Host
}
```
-Write-journal specialist:
+Write-journal specialist: after the host recognizes that the user's intent is to write a journal, it hands off to this specialist to write the content to a file.
```go
func newWriteJournalSpecialist(ctx context.Context) (*host.Specialist, error) {
@@ -95,7 +95,7 @@ func newWriteJournalSpecialist(ctx context.Context) (*host.Specialist, error) {
}
```
-Read-journal specialist (streams lines):
+Read-journal specialist: after the host recognizes that the user's intent is to read a journal, it hands off to this specialist to read the journal file content and output it line by line. This is a local function.
```go
func newReadJournalSpecialist(ctx context.Context) (*host.Specialist, error) {
@@ -141,7 +141,7 @@ func newReadJournalSpecialist(ctx context.Context) (*host.Specialist, error) {
}
```
-Answer-with-journal specialist:
+Answer-with-journal specialist: answers questions based on journal content.
```go
func newAnswerWithJournalSpecialist(ctx context.Context) (*host.Specialist, error) {
@@ -269,48 +269,48 @@ func main() {
cb := &logCallback{}
- for { // multi-turn until user enters "exit"
+ for { // multi-turn conversation, loops until user enters "exit"
println("\n\nYou: ") // prompt for user input
- var message string
- scanner := bufio.NewScanner(os.Stdin) // read from CLI
- for scanner.Scan() {
- message += scanner.Text()
- break
- }
-
- if err := scanner.Err(); err != nil {
- panic(err)
- }
-
- if message == "exit" {
- return
- }
-
- msg := &schema.Message{
- Role: schema._User_,
- Content: message,
- }
-
- out, err := hostMA.Stream(ctx, []*schema.Message{msg}, host.WithAgentCallbacks(cb))
- if err != nil {
- panic(err)
- }
-
- defer out.Close()
-
- println("\nAnswer:")
-
- for {
- msg, err := out.Recv()
- if err != nil {
- if err == io.EOF {
- break
- }
- }
-
- print(msg.Content)
- }
+ var message string
+ scanner := bufio.NewScanner(os.Stdin) // read user input from CLI
+ for scanner.Scan() {
+ message += scanner.Text()
+ break
+ }
+
+ if err := scanner.Err(); err != nil {
+ panic(err)
+ }
+
+ if message == "exit" {
+ return
+ }
+
+ msg := &schema.Message{
+ Role: schema._User_,
+ Content: message,
+ }
+
+ out, err := hostMA.Stream(ctx, []*schema.Message{msg}, host.WithAgentCallbacks(cb))
+ if err != nil {
+ panic(err)
+ }
+
+ defer out.Close()
+
+ println("\nAnswer:")
+
+ for {
+ msg, err := out.Recv()
+ if err != nil {
+ if err == io.EOF {
+ break
+ }
+ }
+
+ print(msg.Content)
+ }
}
}
```
@@ -318,7 +318,7 @@ func main() {
Console output example:
```go
-You:
+You:
write journal: I got up at 7:00 in the morning
HandOff to write_journal with argument {"reason":"I got up at 7:00 in the morning"}
@@ -326,7 +326,7 @@ HandOff to write_journal with argument {"reason":"I got up at 7:00 in the mornin
Answer:
Journal written successfully: I got up at 7:00 in the morning
-You:
+You:
read journal
HandOff to view_journal_content with argument {"reason":"User wants to read the journal content."}
@@ -334,7 +334,8 @@ HandOff to view_journal_content with argument {"reason":"User wants to read the
Answer:
I got up at 7:00 in the morning
-You:
+
+You:
when did I get up in the morning?
HandOff to answer_with_journal with argument {"reason":"To find out the user's morning wake-up times"}
@@ -351,7 +352,7 @@ Host Multi-Agent provides `StreamToolCallChecker` to determine whether Host outp
Different providers in streaming mode may output tool calls differently: some output tool calls directly (e.g., OpenAI); some output text first then tool calls (e.g., Claude). Configure a checker accordingly.
-Default checker (first non-empty chunk must be tool-call):
+Optional. If not set, the default checks whether the first "non-empty chunk" contains a tool call:
```go
func firstChunkStreamToolCallChecker(_ context.Context, sr *schema.StreamReader[*schema.Message]) (bool, error) {
@@ -379,9 +380,9 @@ func firstChunkStreamToolCallChecker(_ context.Context, sr *schema.StreamReader[
}
```
-The default fits providers whose Tool Call messages contain only tool calls.
+The default implementation is suitable for: models whose Tool Call Message contains only Tool Calls.
-When a provider outputs non-empty content before tool calls, define a custom checker:
+The default implementation is NOT suitable for: cases where there are non-empty content chunks before the Tool Call output. In such cases, you need to define a custom tool call checker:
```go
toolCallChecker := func(ctx context.Context, sr *schema.StreamReader[*schema.Message]) (bool, error) {
@@ -405,12 +406,15 @@ toolCallChecker := func(ctx context.Context, sr *schema.StreamReader[*schema.Mes
}
```
-Note: in extreme cases you may need to scan all chunks, degrading the streaming decision. To preserve streaming behavior as much as possible:
+The custom StreamToolCallChecker above may need to check **all chunks** for ToolCalls in extreme cases, which can cause the "streaming decision" effect to be lost. To preserve the "streaming decision" effect as much as possible, the recommendation is:
-> Tip: add a prompt such as “If you need to call tools, output the tool calls only, do not output text.” Prompt effectiveness varies; adjust and verify with your provider.
+> 💡
+> Try adding a prompt to constrain the model not to output additional text when calling tools, for example: "If you need to call a tool, output the tool directly, do not output text."
+>
+> Different models may be affected differently by prompts, so you need to adjust the prompt and verify the effect in actual use.
### Host selects multiple Specialists
-Host may select multiple specialists via a list of tool calls. In that case, Host Multi-Agent routes to all selected specialists in parallel, and after they finish, summarizes multiple messages into one via a Summarizer node as the final output.
+Host selects Specialists in the form of Tool Calls, so it may select multiple Specialists simultaneously as a list of Tool Calls. In this case, Host Multi-Agent routes the request to all selected Specialists simultaneously, and after all Specialists complete, it summarizes multiple Messages into one Message through the Summarizer node as the final output of Host Multi-Agent.
-Users can configure a Summarizer (ChatModel + SystemPrompt) to customize behavior. If unspecified, Host Multi-Agent concatenates contents from multiple specialists.
+Users can configure a Summarizer by specifying a ChatModel and SystemPrompt to customize the Summarizer's behavior. If not specified, Host Multi-Agent will concatenate the Message Contents from multiple Specialists and return them.
diff --git a/content/en/docs/eino/core_modules/flow_integration_components/react_agent_manual.md b/content/en/docs/eino/core_modules/flow_integration_components/react_agent_manual.md
index a4ab703d3d5..0b817b66b8a 100644
--- a/content/en/docs/eino/core_modules/flow_integration_components/react_agent_manual.md
+++ b/content/en/docs/eino/core_modules/flow_integration_components/react_agent_manual.md
@@ -1,6 +1,6 @@
---
Description: ""
-date: "2025-12-09"
+date: "2026-01-20"
lastmod: ""
tags: []
title: 'Eino: ReAct Agent Manual'
@@ -9,21 +9,22 @@ weight: 1
# Introduction
-Eino’s ReAct Agent implements the [ReAct logic](https://react-lm.github.io/), enabling fast, flexible agent construction and invocation.
+Eino ReAct Agent is an agent framework that implements the [ReAct logic](https://react-lm.github.io/), allowing users to quickly and flexibly build and invoke ReAct Agents.
-> Code: [Implementation Directory](https://github.com/cloudwego/eino/tree/main/flow/agent/react)
+> 💡
+> See the code implementation at: [Implementation Directory](https://github.com/cloudwego/eino/tree/main/flow/agent/react)
-## Topology and Data Flow
+## Node Topology & Data Flow Diagram
-Under the hood, ReAct Agent uses `compose.Graph`. Typically two nodes: `ChatModel` and `Tools`. All historical messages are stored in `state`. Before passing history to `ChatModel`, messages are copied and processed by `MessageModifier`. When `ChatModel` returns without any tool call, the final message is returned.
+ReAct Agent uses `compose.Graph` as its orchestration scheme under the hood. Generally, there are 2 nodes: ChatModel and Tools. All historical messages during the intermediate running process are stored in state. Before passing all historical messages to ChatModel, the messages are copied and processed by MessageModifier, and the processed result is then passed to ChatModel. The process continues until ChatModel returns a message without any tool call, at which point the final message is returned.
-If any tool is marked `ReturnDirectly`, a `Branch` follows `ToolsNode` to short-circuit and end when such a tool is invoked; otherwise the flow returns to `ChatModel`.
+When at least one Tool in the Tools list is configured with ReturnDirectly, the ReAct Agent structure becomes more complex: a Branch is added after ToolsNode to determine whether a ReturnDirectly Tool was called. If so, it goes directly to END; otherwise, it proceeds to ChatModel as usual.
## Initialization
-Provide a `ToolCallingChatModel` and `ToolsConfig`. Optional: `MessageModifier`, `MaxStep`, `ToolReturnDirectly`, `StreamToolCallChecker`.
+The ReactAgent initialization function is provided. Required parameters are Model and ToolsConfig. Optional parameters are MessageModifier, MaxStep, ToolReturnDirectly, and StreamToolCallChecker.
```bash
go get github.com/cloudwego/eino-ext/components/model/openai@latest
@@ -42,10 +43,10 @@ import (
)
func main() {
- // initialize chat model
+ // first initialize the required chatModel
toolableChatModel, err := openai.NewChatModel(...)
- // initialize tools
+ // initialize the required tools
tools := compose.ToolsNodeConfig{
InvokableTools: []tool.InvokableTool{mytool},
StreamableTools: []tool.StreamableTool{myStreamTool},
@@ -62,7 +63,9 @@ func main() {
### Model
-ReAct requires a `ToolCallingChatModel`. Inside the agent, `WithTools` is called to bind the agent’s tools to the model:
+Since ReAct Agent needs to make tool calls, the Model needs to have ToolCall capability, so you need to configure a ToolCallingChatModel.
+
+Inside the Agent, the WithTools interface is called to register the Agent's tool list with the model. The definition is:
```go
// BaseChatModel defines the basic interface for chat models.
@@ -88,7 +91,7 @@ type ToolCallingChatModel interface {
}
```
-Supported implementations include OpenAI and Ark (any provider that supports tool calls).
+Currently, eino provides implementations such as openai and ark, as long as the underlying model supports tool call.
```bash
go get github.com/cloudwego/eino-ext/components/model/openai@latest
go get github.com/cloudwego/eino-ext/components/model/ark@latest
@@ -128,7 +131,7 @@ func arkExample() {
### ToolsConfig
-`toolsConfig` is `compose.ToolsNodeConfig`. To build a Tools node, provide Tool info and a run function. Tool interfaces:
+toolsConfig type is `compose.ToolsNodeConfig`. In eino, to build a Tool node, you need to provide the Tool's information and the function to call the Tool. The tool interface definition is as follows:
```go
type InvokableRun func(ctx context.Context, arguments string, opts ...Option) (content string, err error)
type StreamableRun func(ctx context.Context, arguments string, opts ...Option) (content *schema.StreamReader[string], err error)
@@ -150,7 +153,7 @@ type StreamableTool interface {
}
```
-You can implement tools per the interfaces, or use helpers to construct tools:
+Users can implement the required tools according to the tool interface definition. The framework also provides a more convenient method to build tools:
```go
userInfoTool := utils.NewTool(
&schema.ToolInfo{
@@ -184,14 +187,14 @@ toolConfig := &compose.ToolsNodeConfig{
### MessageModifier
-Executed before each call to `ChatModel`:
+MessageModifier is executed before each time all historical messages are passed to ChatModel. The definition is:
```go
// modify the input messages before the model is called.
type MessageModifier func(ctx context.Context, input []*schema.Message) []*schema.Message
```
-Configure `MessageModifier` inside the Agent to adjust the messages passed to the model:
+Configuring MessageModifier in the Agent can modify the messages passed to the model, commonly used to add a preceding system message:
```go
import (
@@ -213,57 +216,130 @@ func main() {
},
})
- agent.Generate(ctx, []*schema.Message{schema.UserMessage("写一个 hello world 的代码")})
- // 实际输入：
+ agent.Generate(ctx, []*schema.Message{schema.UserMessage("Write a hello world code")})
+ // The actual input to the model is:
// []*schema.Message{
- // {Role: schema.System, Content:"你是一个 golang 开发专家."},
- // {Role: schema.Human, Content: "写一个 hello world 的代码"}
- // }
+ // {Role: schema.System, Content:"You are a golang development expert."},
+ // {Role: schema.Human, Content: "Write a hello world code"}
+ //}
}
```
+### MessageRewriter
+
+MessageRewriter is executed before each ChatModel call and modifies and updates the historical messages saved in the global state:
+
+```go
+// MessageRewriter modifies message in the state, before the ChatModel is called.
+// It takes the messages stored accumulated in state, modify them, and put the modified version back into state.
+// Useful for compressing message history to fit the model context window,
+// or if you want to make changes to messages that take effect across multiple model calls.
+// NOTE: if both MessageModifier and MessageRewriter are set, MessageRewriter will be called before MessageModifier.
+MessageRewriter MessageModifier
+```
+
+Commonly used for context compression, which is a message change that needs to take effect continuously across multiple ReAct loops.
+
+Compared to MessageModifier (which only changes without persisting, thus suitable for system prompts), MessageRewriter's changes are visible in subsequent ReAct loops.
+
### MaxStep
-Specify the maximum number of steps. One loop is `ChatModel` + `Tools` (2 steps). Default is `node count + 2`.
+Specify the Agent's maximum running step length. Each transition from one node to the next node counts as one step. The default value is node count + 2.
-Since one loop is 2 steps, default `12` supports up to 6 loops. The final step must be a `ChatModel` result (no tool call), so at most 5 Tools.
+Since one loop in the Agent is ChatModel + Tools, which equals 2 steps, the default value of 12 allows up to 6 loops. However, since the last step must be a ChatModel return (because ChatModel must determine that no tool needs to run before returning the final result), at most 5 tools can be run.
-For 10 loops (10×ChatModel + 9×Tools), set `MaxStep` to 20; for 20 loops set `MaxStep` to 40.
+Similarly, if you want to run at most 10 loops (10 ChatModel + 9 Tools), you need to set MaxStep to 20. If you want to run at most 20 loops, MaxStep needs to be 40.
-### ToolReturnDirectly and Stream Tool Call Checking
+```go
+func main() {
+ agent, err := react.NewAgent(ctx, &react.AgentConfig{
+ ToolCallingModel: toolableChatModel,
+ ToolsConfig: tools,
+ MaxStep: 20,
+ }
+}
+```
-If a tool is `ReturnDirectly`, its output is returned immediately; configure `ToolReturnDirectly` with the tool name. For streaming models, set `StreamToolCallChecker` to determine tool-call presence in streams (model-dependent behavior).
+### ToolReturnDirectly
+
+If you want the Agent to directly return the Tool's Response ToolMessage after ChatModel selects a specific Tool and executes it, you can configure this Tool in ToolReturnDirectly.
+
+```go
+a, err = NewAgent(ctx, &AgentConfig{
+ Model: cm,
+ ToolsConfig: compose.ToolsNodeConfig{
+ Tools: []tool.BaseTool{fakeTool, fakeStreamTool},
+ },
+
+ MaxStep: 40,
+ ToolReturnDirectly: map[string]struct{}{fakeToolName: {}}, // one of the two tools is return directly
+})
+```
+
+### StreamToolCallChecker
+
+Different models may output tool calls differently in streaming mode: some models (like OpenAI) output tool calls directly; some models (like Claude) output text first, then output tool calls. Therefore, different methods are needed to determine this. This field is used to specify the function that determines whether the model's streaming output contains tool calls.
+
+Optional. If not set, the default checks whether the first "non-empty chunk" contains a tool call:
-Default checker (first non-empty chunk must be tool-call):
```go
func firstChunkStreamToolCallChecker(_ context.Context, sr *schema.StreamReader[*schema.Message]) (bool, error) {
defer sr.Close()
+
for {
- msg, err := sr.Recv()
- if errors.Is(err, io.EOF) { return false, nil }
- if err != nil { return false, err }
- if len(msg.ToolCalls) > 0 { return true, nil }
- if len(msg.Content) == 0 { continue }
- return false, nil
+ msg, err := sr.Recv()
+ if err == io.EOF {
+ return false, nil
+ }
+ if err != nil {
+ return false, err
+ }
+
+ if len(msg.ToolCalls) > 0 {
+ return true, nil
+ }
+
+ if len(msg.Content) == 0 { // skip empty chunks at the front
+ continue
+ }
+
+ return false, nil
}
}
```
-If the provider outputs non-empty text before tool-calls, implement a custom checker that scans all chunks for tool-calls:
+The default implementation is suitable for: models whose Tool Call Message contains only Tool Calls.
+
+The default implementation is NOT suitable for: cases where there are non-empty content chunks before the Tool Call output. In such cases, you need to define a custom tool call checker:
+
```go
toolCallChecker := func(ctx context.Context, sr *schema.StreamReader[*schema.Message]) (bool, error) {
defer sr.Close()
for {
- msg, err := sr.Recv()
- if errors.Is(err, io.EOF) { break }
- if err != nil { return false, err }
- if len(msg.ToolCalls) > 0 { return true, nil }
+ msg, err := sr.Recv()
+ if err != nil {
+ if errors.Is(err, io.EOF) {
+ // finish
+ break
+ }
+
+ return false, err
+ }
+
+ if len(msg.ToolCalls) > 0 {
+ return true, nil
+ }
}
return false, nil
}
```
-Tip: add a prompt like “If you need to call tools, output only tool-calls, not text” to preserve a streaming experience where possible.
+The custom StreamToolCallChecker above may need to check **all chunks** for ToolCalls in extreme cases, which can cause the "streaming decision" effect to be lost. To preserve the "streaming decision" effect as much as possible, the recommendation is:
+
+> 💡
+> Try adding a prompt to constrain the model not to output additional text when calling tools, for example: "If you need to call a tool, output the tool directly, do not output text."
+>
+> Different models may be affected differently by prompts, so you need to adjust the prompt and verify the effect in actual use.
## Invocation
@@ -307,7 +383,7 @@ for {
### WithCallbacks
-Callback handlers run at defined timings. Since the agent graph has only ChatModel and ToolsNode, the agent’s callbacks are those two component callbacks. A helper is provided to build them:
+Callback is a callback executed at specific timings during Agent runtime. Since the Agent Graph only has ChatModel and ToolsNode, the Agent's Callback is the Callback for ChatModel and Tool. The react package provides a helper function to help users quickly build Callback Handlers for these two component types.
```go
import (
@@ -326,9 +402,9 @@ func BuildAgentCallback(modelHandler *template.ModelCallbackHandler, toolHandler
### Options
-React agent supports dynamic runtime options.
+React agent supports dynamic modification through runtime Options.
-Scenario 1: modify the model config at runtime:
+Scenario 1: Modify the Model configuration in the Agent at runtime:
```go
// WithChatModelOptions returns an agent option that specifies model.Option for the chat model in agent.
@@ -337,7 +413,7 @@ func WithChatModelOptions(opts ...model.Option) agent.AgentOption {
}
```
-Scenario 2: modify the Tool list at runtime:
+Scenario 2: Modify the Tool list at runtime:
```go
// WithToolList returns an agent option that specifies the list of tools can be called which are BaseTool but must implement InvokableTool or StreamableTool.
@@ -346,9 +422,9 @@ func WithToolList(tools ...tool.BaseTool) agent.AgentOption {
}
```
-Also update ChatModel’s bound tools: `WithChatModelOptions(model.WithTools(...))`
+Additionally, you also need to modify the tools bound in ChatModel: `WithChatModelOptions(model.WithTools(...))`
-Scenario 3: modify options for a specific Tool:
+Scenario 3: Modify the options for a specific Tool at runtime:
```go
// WithToolOptions returns an agent option that specifies tool.Option for the tools in agent.
@@ -357,9 +433,13 @@ func WithToolOptions(opts ...tool.Option) agent.AgentOption {
}
```
+### Prompt
+
+Modifying the prompt at runtime is essentially passing different Message lists when calling Generate or Stream.
+
### Get Intermediate Results
-Use `WithMessageFuture` to capture intermediate `*schema.Message` during execution:
+If you want to get the `*schema.Message` generated during the ReAct Agent execution process in real-time, you can first obtain a runtime Option and a MessageFuture through WithMessageFuture:
```go
// WithMessageFuture returns an agent option and a MessageFuture interface instance.
@@ -390,57 +470,83 @@ func WithMessageFuture() (agent.AgentOption, MessageFuture) {
}
```
-Pass the option into Generate or Stream. Use `GetMessages` or `GetMessageStreams` to read intermediate messages.
+This runtime Option is passed normally to the Generate or Stream method. The MessageFuture can use GetMessages or GetMessageStreams to get the Messages of various intermediate states.
+
+> 💡
+> After passing the MessageFuture Option, the Agent will still run in a blocking manner. Receiving intermediate results through MessageFuture needs to be asynchronous with the Agent running (read MessageFuture in a goroutine or run the Agent in a goroutine).
+
+## Agent In Graph/Chain
+
+Agent can be embedded into other Graphs as a Lambda:
+
+```go
+agent, _ := NewAgent(ctx, &AgentConfig{
+ ToolCallingModel: cm,
+ ToolsConfig: compose.ToolsNodeConfig{
+ Tools: []tool.BaseTool{fakeTool, &fakeStreamToolGreetForTest{}},
+ },
-Tip: the agent still runs synchronously. Read the future in a goroutine or run the agent in a goroutine.
+ MaxStep: 40,
+})
-### Agent In Graph/Chain
+chain := compose.NewChain[[]*schema.Message, string]()
+agentLambda, _ := compose.AnyLambda(agent.Generate, agent.Stream, nil, nil)
-Agent can be embedded via `compose.AnyLambda` and appended to Chain/Graph.
+chain.
+ AppendLambda(agentLambda).
+ AppendLambda(compose.InvokableLambda(func(ctx context.Context, input *schema.Message) (string, error) {
+ t.Log("got agent response: ", input.Content)
+ return input.Content, nil
+ }))
+r, _ := chain.Compile(ctx)
+
+res, _ := r.Invoke(ctx, []*schema.Message{{Role: schema.User, Content: "hello"}},
+ compose.WithCallbacks(callbackForTest))
+```
## Demo
### Basic Info
-简介：这是一个拥有两个 tool (query_restaurants 和 query_dishes ) 的 `美食推荐官`
+Description: This is a `Food Recommender` with two tools (query_restaurants and query_dishes).
-地址：[eino-examples/flow/agent/react](https://github.com/cloudwego/eino-examples/tree/main/flow/agent/react)
+Repository: [eino-examples/flow/agent/react](https://github.com/cloudwego/eino-examples/tree/main/flow/agent/react)
-使用方式:
+Usage:
-1. clone eino-examples repo，并 cd 到根目录
-2. 提供一个 `OPENAI_API_KEY`: `export OPENAI_API_KEY=xxxxxxx`
-3. 运行 demo: `go run flow/agent/react/react.go`
+1. Clone the eino-examples repo and cd to the root directory
+2. Provide an `OPENAI_API_KEY`: `export OPENAI_API_KEY=xxxxxxx`
+3. Run the demo: `go run flow/agent/react/react.go`
-### 运行过程
+### Running Process
-### 运行过程解释
+### Running Process Explanation
-- 模拟用户输入了 `我在海淀区，给我推荐一些菜，需要有口味辣一点的菜，至少推荐有 2 家餐厅`
-- agent 运行第一个节点 `ChatModel`，大模型判断出需要做一次 ToolCall 调用来查询餐厅，并且给出的参数为：
+- Simulating user input: `I'm in Haidian District, recommend some dishes for me, need some spicy dishes, recommend at least 2 restaurants`
+- The agent runs the first node `ChatModel`, the LLM determines that a ToolCall needs to be made to query restaurants, with the following parameters:
```json
"function": {
"name": "query_restaurants",
- "arguments": "{\"location\":\"海淀区\",\"topn\":2}"
+ "arguments": "{\"location\":\"Haidian District\",\"topn\":2}"
}
```
-- 进入 `Tools` 节点，调用 查询餐厅 的 tool，并且得到结果，结果返回了 2 家海淀区的餐厅信息:
+- Entering the `Tools` node, calling the query_restaurants tool and getting the result. The result returns information about 2 restaurants in Haidian District:
```json
-[{"id":"1001","name":"老地方餐厅","place":"北京老胡同 5F, 左转进入","desc":"","score":3},{"id":"1002","name":"人间味道餐厅","place":"北京大世界商城-1F","desc":"","score":5}]
+[{"id":"1001","name":"Old Place Restaurant","place":"Beijing Old Hutong 5F, turn left to enter","desc":"","score":3},{"id":"1002","name":"Human Taste Restaurant","place":"Beijing Big World Mall -1F","desc":"","score":5}]
```
-- 得到 tool 的结果后，此时对话的 history 中包含了 tool 的结果，再次运行 `ChatModel`，大模型判断出需要再次调用另一个 ToolCall，用来查询餐厅有哪些菜品，注意，由于有两家餐厅，因此大模型返回了 2 个 ToolCall，如下：
+- After getting the tool result, the conversation history now contains the tool result. Running `ChatModel` again, the LLM determines that another ToolCall needs to be made to query what dishes the restaurants have. Note that since there are two restaurants, the LLM returns 2 ToolCalls as follows:
```json
"Message": {
"role": "ai",
"content": "",
- "tool_calls": [ // <= 这里有 2 个 tool call
+ "tool_calls": [ // <= there are 2 tool calls here
{
"index": 1,
"id": "call_wV7zA3vGGJBhuN7r9guhhAfF",
@@ -461,20 +567,20 @@ Agent can be embedded via `compose.AnyLambda` and appended to Chain/Graph.
}
```
-- 再次进入到 `Tools` 节点，由于有 2 个 tool call，Tools 节点内部并发执行这两个调用，并且均加入到对话的 history 中，从 callback 的调试日志中可以看到结果如下：
+- Entering the `Tools` node again. Since there are 2 tool calls, the Tools node executes these two calls concurrently internally, and both are added to the conversation history. From the callback debug logs, you can see the results as follows:
```json
=========[OnToolStart]=========
{"restaurant_id": "1001", "topn": 5}
=========[OnToolEnd]=========
-[{"name":"红烧肉","desc":"一块红烧肉","price":20,"score":8},{"name":"清泉牛肉","desc":"很多的水煮牛肉","price":50,"score":8},{"name":"清炒小南瓜","desc":"炒的糊糊的南瓜","price":5,"score":5},{"name":"韩式辣白菜","desc":"这可是开过光的辣白菜，好吃得很","price":20,"score":9},{"name":"酸辣土豆丝","desc":"酸酸辣辣的土豆丝","price":10,"score":9}]
+[{"name":"Braised Pork","desc":"A piece of braised pork","price":20,"score":8},{"name":"Spring Beef","desc":"Lots of boiled beef","price":50,"score":8},{"name":"Stir-fried Pumpkin","desc":"Mushy stir-fried pumpkin","price":5,"score":5},{"name":"Korean Spicy Cabbage","desc":"This is blessed spicy cabbage, very delicious","price":20,"score":9},{"name":"Hot and Sour Potato Shreds","desc":"Sour and spicy potato shreds","price":10,"score":9}]
=========[OnToolStart]=========
{"restaurant_id": "1002", "topn": 5}
=========[OnToolEnd]=========
-[{"name":"红烧排骨","desc":"一块一块的排骨","price":43,"score":7},{"name":"大刀回锅肉","desc":"经典的回锅肉, 肉很大","price":40,"score":8},{"name":"火辣辣的吻","desc":"凉拌猪嘴，口味辣而不腻","price":60,"score":9},{"name":"辣椒拌皮蛋","desc":"擂椒皮蛋，下饭的神器","price":15,"score":8}]
+[{"name":"Braised Spare Ribs","desc":"Piece by piece spare ribs","price":43,"score":7},{"name":"Big Knife Twice-cooked Pork","desc":"Classic twice-cooked pork, big pieces of meat","price":40,"score":8},{"name":"Fiery Kiss","desc":"Cold pig snout, spicy but not greasy","price":60,"score":9},{"name":"Chili Mixed with Preserved Egg","desc":"Pounded chili preserved egg, a rice killer","price":15,"score":8}]
```
-- 得到所有 tool call 返回的结果后，再次进入 `ChatModel` 节点，这次大模型发现已经拥有了回答用户提问的所有信息，因此整合信息后输出结论，由于调用时使用的 `Stream` 方法，因此流式返回的大模型结果。
+- After getting all the tool call results, entering the `ChatModel` node again. This time the LLM finds that it has all the information needed to answer the user's question, so it integrates the information and outputs the conclusion. Since the `Stream` method was used for the call, the LLM result is returned in a streaming manner.
## Related Reading
diff --git a/content/en/docs/eino/ecosystem_integration/_index.md b/content/en/docs/eino/ecosystem_integration/_index.md
index 4ebdb86df53..dc0bcd8820a 100644
--- a/content/en/docs/eino/ecosystem_integration/_index.md
+++ b/content/en/docs/eino/ecosystem_integration/_index.md
@@ -1,6 +1,6 @@
---
Description: ""
-date: "2025-07-21"
+date: "2026-01-20"
lastmod: ""
tags: []
title: 'Eino: Ecosystem Integration'
@@ -12,7 +12,7 @@ weight: 5
### ChatModel
- openai: [ChatModel - OpenAI](/docs/eino/ecosystem_integration/chat_model/chat_model_openai)
-- ark: [ChatModel - Ark](/docs/eino/ecosystem_integration/chat_model/chat_model_ark)
+- ark: [ChatModel - ARK](/docs/eino/ecosystem_integration/chat_model/chat_model_ark)
- ollama: [ChatModel - Ollama](/docs/eino/ecosystem_integration/chat_model/chat_model_ollama)
### Document
@@ -36,16 +36,30 @@ weight: 5
### Embedding
-- ark: [Embedding - Ark](/docs/eino/ecosystem_integration/embedding/embedding_ark)
+- ark: [Embedding - ARK](/docs/eino/ecosystem_integration/embedding/embedding_ark)
- openai: [Embedding - OpenAI](/docs/eino/ecosystem_integration/embedding/embedding_openai)
### Indexer
- volc vikingdb: [Indexer - volc VikingDB](/docs/eino/ecosystem_integration/indexer/indexer_volc_vikingdb)
+- Milvus 2.5+: [Indexer - Milvus 2 (v2.5+)](/docs/eino/ecosystem_integration/indexer/indexer_milvusv2)
+- Milvus 2.4: [Indexer - Milvus](/docs/eino/ecosystem_integration/indexer/indexer_milvus)
+- OpenSearch 3: [Indexer - OpenSearch 3](/docs/eino/ecosystem_integration/indexer/indexer_opensearch3)
+- OpenSearch 2: [Indexer - OpenSearch 2](/docs/eino/ecosystem_integration/indexer/indexer_opensearch2)
+- ElasticSearch 9: [Indexer - Elasticsearch 9](/docs/eino/ecosystem_integration/indexer/indexer_elasticsearch9)
+- Elasticsearch 8: [Indexer - ES8](/docs/eino/ecosystem_integration/indexer/indexer_es8)
+- ElasticSearch 7: [Indexer - Elasticsearch 7 ](/docs/eino/ecosystem_integration/indexer/indexer_elasticsearch7)
### Retriever
- volc vikingdb: [Retriever - volc VikingDB](/docs/eino/ecosystem_integration/retriever/retriever_volc_vikingdb)
+- Milvus 2.5+: [Retriever - Milvus 2 (v2.5+) ](/docs/eino/ecosystem_integration/retriever/retriever_milvusv2)
+- Milvus 2.4: [Retriever - Milvus](/docs/eino/ecosystem_integration/retriever/retriever_milvus)
+- OpenSearch 3: [Retriever - OpenSearch 3](/docs/eino/ecosystem_integration/retriever/retriever_opensearch3)
+- OpenSearch 2: [Retriever - OpenSearch 2](/docs/eino/ecosystem_integration/retriever/retriever_opensearch2)
+- ElasticSearch 9: [Retriever - Elasticsearch 9](/docs/eino/ecosystem_integration/retriever/retriever_elasticsearch9)
+- ElasticSearch 8: [Retriever - ES8](/docs/eino/ecosystem_integration/retriever/retriever_es8)
+- ElasticSearch 7: [Retriever - ES 7](/docs/eino/ecosystem_integration/retriever/retriever_elasticsearch7)
### Tools
diff --git a/content/en/docs/eino/ecosystem_integration/callbacks/callback_cozeloop.md b/content/en/docs/eino/ecosystem_integration/callbacks/callback_cozeloop.md
index 281bc44f888..960bd16fa76 100644
--- a/content/en/docs/eino/ecosystem_integration/callbacks/callback_cozeloop.md
+++ b/content/en/docs/eino/ecosystem_integration/callbacks/callback_cozeloop.md
@@ -1,6 +1,6 @@
---
Description: ""
-date: "2025-12-11"
+date: "2026-01-20"
lastmod: ""
tags: []
title: Callback - CozeLoop
@@ -51,4 +51,5 @@ func main() {
```
## **More Details**
+
- [CozeLoop Docs](https://github.com/coze-dev/cozeloop-go)
diff --git a/content/en/docs/eino/ecosystem_integration/chat_model/_index.md b/content/en/docs/eino/ecosystem_integration/chat_model/_index.md
index 3c2219951cf..ece88e1fd03 100644
--- a/content/en/docs/eino/ecosystem_integration/chat_model/_index.md
+++ b/content/en/docs/eino/ecosystem_integration/chat_model/_index.md
@@ -1,6 +1,6 @@
---
Description: ""
-date: "2025-07-21"
+date: "2026-01-20"
lastmod: ""
tags: []
title: ChatModel
diff --git a/content/en/docs/eino/ecosystem_integration/chat_model/chat_model_ark.md b/content/en/docs/eino/ecosystem_integration/chat_model/chat_model_ark.md
index 22597cdf915..5f221dc0dff 100644
--- a/content/en/docs/eino/ecosystem_integration/chat_model/chat_model_ark.md
+++ b/content/en/docs/eino/ecosystem_integration/chat_model/chat_model_ark.md
@@ -1,15 +1,18 @@
---
Description: ""
-date: "2025-12-02"
+date: "2026-01-20"
lastmod: ""
tags: []
-title: ChatModel - ark
+title: ChatModel - ARK
weight: 0
---
+## **ARK Model**
+
A Volcengine Ark model implementation for [Eino](https://github.com/cloudwego/eino) that implements the `ToolCallingChatModel` interface. This enables seamless integration with Eino's LLM capabilities to enhance natural language processing and generation.
-This package provides three components:
+This package provides two different models:
+
- **ChatModel**: for text-based and multi-modal chat completion.
- **ImageGenerationModel**: for generating images from text prompts or images.
- **ResponseAPI**: methods and helpers for interacting with the OpenAI-compatible API.
@@ -30,12 +33,6 @@ This package provides three components:
go get github.com/cloudwego/eino-ext/components/model/ark@latest
```
----
-
-## Chat
-
-This model is used for standard chat and text generation tasks.
-
### Quick Start
Here's a quick example of how to use `ChatModel`:
@@ -44,73 +41,73 @@ Here's a quick example of how to use `ChatModel`:
package main
import (
- "context"
- "encoding/json"
- "errors"
- "io"
- "log"
- "os"
+ "context"
+ "encoding/json"
+ "errors"
+ "io"
+ "log"
+ "os"
- "github.com/cloudwego/eino/schema"
+ "github.com/cloudwego/eino/schema"
- "github.com/cloudwego/eino-ext/components/model/ark"
+ "github.com/cloudwego/eino-ext/components/model/ark"
)
func main() {
- ctx := context.Background()
-
- chatModel, err := ark.NewChatModel(ctx, &ark.ChatModelConfig{
- APIKey: os.Getenv("ARK_API_KEY"),
- Model: os.Getenv("ARK_MODEL_ID"),
- })
+ ctx := context.Background()
- if err != nil {
- log.Fatalf("NewChatModel failed, err=%v", err)
- }
+ chatModel, err := ark.NewChatModel(ctx, &ark.ChatModelConfig{
+ APIKey: os.Getenv("ARK_API_KEY"),
+ Model: os.Getenv("ARK_MODEL_ID"),
+ })
- inMsgs := []*schema.Message{
- {
- Role: schema.User,
- Content: "how do you generate answer for user question as a machine, please answer in short?",
- },
- }
+ if err != nil {
+ log.Fatalf("NewChatModel failed, err=%v", err)
+ }
- msg, err := chatModel.Generate(ctx, inMsgs)
- if err != nil {
- log.Fatalf("Generate failed, err=%v", err)
- }
+ inMsgs := []*schema.Message{
+ {
+ Role: schema.User,
+ Content: "how do you generate answer for user question as a machine, please answer in short?",
+ },
+ }
- log.Printf("generate output: \n")
- respBody, _ := json.MarshalIndent(msg, " ", " ")
- log.Printf(" body: %s \n", string(respBody))
+ msg, err := chatModel.Generate(ctx, inMsgs)
+ if err != nil {
+ log.Fatalf("Generate failed, err=%v", err)
+ }
- sr, err := chatModel.Stream(ctx, inMsgs)
- if err != nil {
- log.Fatalf("Stream failed, err=%v", err)
- }
+ log.Printf("generate output: \n")
+ respBody, _ := json.MarshalIndent(msg, " ", " ")
+ log.Printf(" body: %s \n", string(respBody))
- chunks := make([]*schema.Message, 0, 1024)
- for {
- msgChunk, err := sr.Recv()
- if errors.Is(err, io.EOF) {
- break
- }
+ sr, err := chatModel.Stream(ctx, inMsgs)
if err != nil {
- log.Fatalf("Stream Recv failed, err=%v", err)
+ log.Fatalf("Stream failed, err=%v", err)
}
- chunks = append(chunks, msgChunk)
- }
+ chunks := make([]*schema.Message, 0, 1024)
+ for {
+ msgChunk, err := sr.Recv()
+ if errors.Is(err, io.EOF) {
+ break
+ }
+ if err != nil {
+ log.Fatalf("Stream Recv failed, err=%v", err)
+ }
+
+ chunks = append(chunks, msgChunk)
+ }
- msg, err = schema.ConcatMessages(chunks)
- if err != nil {
- log.Fatalf("ConcatMessages failed, err=%v", err)
- }
+ msg, err = schema.ConcatMessages(chunks)
+ if err != nil {
+ log.Fatalf("ConcatMessages failed, err=%v", err)
+ }
- log.Printf("stream final output: \n")
- log.Printf(" request_id: %s \n")
- respBody, _ = json.MarshalIndent(msg, " ", " ")
- log.Printf("body: %s \n", string(respBody))
+ log.Printf("stream final output: \n")
+ log.Printf(" request_id: %s \n")
+ respBody, _ = json.MarshalIndent(msg, " ", " ")
+ log.Printf("body: %s \n", string(respBody))
}
```
@@ -188,12 +185,8 @@ type ChatModelConfig struct {
}
```
----
-
## Image Generation
-This model is specifically designed for generating images from text prompts.
-
### Quick Start
Here's a quick example of how to use `ImageGenerationModel`:
@@ -202,65 +195,65 @@ Here's a quick example of how to use `ImageGenerationModel`:
package main
import (
- "context"
- "encoding/json"
- "log"
- "os"
+ "context"
+ "encoding/json"
+ "log"
+ "os"
- "github.com/cloudwego/eino/schema"
- "github.com/cloudwego/eino-ext/components/model/ark"
+ "github.com/cloudwego/eino/schema"
+ "github.com/cloudwego/eino-ext/components/model/ark"
)
func main() {
- ctx := context.Background()
-
- // Get ARK_API_KEY and an image generation model ID
- imageGenerationModel, err := ark.NewImageGenerationModel(ctx, &ark.ImageGenerationConfig{
- APIKey: os.Getenv("ARK_API_KEY"),
- Model: os.Getenv("ARK_IMAGE_MODEL_ID"), // Use an appropriate image model ID
- })
+ ctx := context.Background()
- if err != nil {
- log.Fatalf("NewImageGenerationModel failed, err=%v", err)
- }
+ // Get ARK_API_KEY and an image generation model ID
+ imageGenerationModel, err := ark.NewImageGenerationModel(ctx, &ark.ImageGenerationConfig{
+ APIKey: os.Getenv("ARK_API_KEY"),
+ Model: os.Getenv("ARK_IMAGE_MODEL_ID"), // Use an appropriate image model ID
+ })
- inMsgs := []*schema.Message{
- {
- Role: schema.User,
- Content: "a photo of a cat sitting on a table",
- },
- }
+ if err != nil {
+ log.Fatalf("NewImageGenerationModel failed, err=%v", err)
+ }
- msg, err := imageGenerationModel.Generate(ctx, inMsgs)
- if err != nil {
- log.Fatalf("Generate failed, err=%v", err)
- }
+ inMsgs := []*schema.Message{
+ {
+ Role: schema.User,
+ Content: "a photo of a cat sitting on a table",
+ },
+ }
- log.Printf("generate output:")
- log.Printf(" request_id: %s", ark.GetArkRequestID(msg))
- respBody, _ := json.MarshalIndent(msg, " ", " ")
- log.Printf(" body: %s", string(respBody))
+ msg, err := imageGenerationModel.Generate(ctx, inMsgs)
+ if err != nil {
+ log.Fatalf("Generate failed, err=%v", err)
+ }
- sr, err := imageGenerationModel.Stream(ctx, inMsgs)
- if err != nil {
- log.Fatalf("Stream failed, err=%v", err)
- }
+ log.Printf("generate output:")
+ log.Printf(" request_id: %s", ark.GetArkRequestID(msg))
+ respBody, _ := json.MarshalIndent(msg, " ", " ")
+ log.Printf(" body: %s", string(respBody))
- log.Printf("stream output:")
- index := 0
- for {
- msgChunk, err := sr.Recv()
- if errors.Is(err, io.EOF) {
- break
- }
+ sr, err := imageGenerationModel.Stream(ctx, inMsgs)
if err != nil {
- log.Fatalf("Stream Recv failed, err=%v", err)
+ log.Fatalf("Stream failed, err=%v", err)
}
- respBody, _ = json.MarshalIndent(msgChunk, " ", " ")
- log.Printf("stream chunk %d: body: %s \n", index, string(respBody))
- index++
- }
+ log.Printf("stream output:")
+ index := 0
+ for {
+ msgChunk, err := sr.Recv()
+ if errors.Is(err, io.EOF) {
+ break
+ }
+ if err != nil {
+ log.Fatalf("Stream Recv failed, err=%v", err)
+ }
+
+ respBody, _ = json.MarshalIndent(msgChunk, " ", " ")
+ log.Printf("stream chunk %d: body: %s \n", index, string(respBody))
+ index++
+ }
}
```
@@ -347,77 +340,76 @@ type ImageGenerationConfig struct {
package main
import (
- "context"
- "encoding/json"
- "errors"
- "io"
- "log"
- "os"
+ "context"
+ "encoding/json"
+ "errors"
+ "io"
+ "log"
+ "os"
- "github.com/cloudwego/eino/schema"
+ "github.com/cloudwego/eino/schema"
- "github.com/cloudwego/eino-ext/components/model/ark"
+ "github.com/cloudwego/eino-ext/components/model/ark"
)
func main() {
- ctx := context.Background()
-
- // Get ARK_API_KEY and ARK_MODEL_ID: https://www.volcengine.com/docs/82379/1399008
- chatModel, err := ark.NewChatModel(ctx, &ark.ChatModelConfig{
- APIKey: os.Getenv("ARK_API_KEY"),
- Model: os.Getenv("ARK_MODEL_ID"),
- })
+ ctx := context.Background()
- if err != nil {
- log.Fatalf("NewChatModel failed, err=%v", err)
- }
+ // Get ARK_API_KEY and ARK_MODEL_ID: https://www.volcengine.com/docs/82379/1399008
+ chatModel, err := ark.NewChatModel(ctx, &ark.ChatModelConfig{
+ APIKey: os.Getenv("ARK_API_KEY"),
+ Model: os.Getenv("ARK_MODEL_ID"),
+ })
- inMsgs := []*schema.Message{
- {
- Role: schema.User,
- Content: "how do you generate answer for user question as a machine, please answer in short?",
- },
- }
+ if err != nil {
+ log.Fatalf("NewChatModel failed, err=%v", err)
+ }
- msg, err := chatModel.Generate(ctx, inMsgs)
- if err != nil {
- log.Fatalf("Generate failed, err=%v", err)
- }
+ inMsgs := []*schema.Message{
+ {
+ Role: schema.User,
+ Content: "how do you generate answer for user question as a machine, please answer in short?",
+ },
+ }
- log.Printf("generate output:")
- log.Printf(" request_id: %s", ark.GetArkRequestID(msg))
- respBody, _ := json.MarshalIndent(msg, " ", " ")
- log.Printf(" body: %s", string(respBody))
+ msg, err := chatModel.Generate(ctx, inMsgs)
+ if err != nil {
+ log.Fatalf("Generate failed, err=%v", err)
+ }
- sr, err := chatModel.Stream(ctx, inMsgs)
- if err != nil {
- log.Fatalf("Stream failed, err=%v", err)
- }
+ log.Printf("generate output:")
+ log.Printf(" request_id: %s", ark.GetArkRequestID(msg))
+ respBody, _ := json.MarshalIndent(msg, " ", " ")
+ log.Printf(" body: %s", string(respBody))
- chunks := make([]*schema.Message, 0, 1024)
- for {
- msgChunk, err := sr.Recv()
- if errors.Is(err, io.EOF) {
- break
- }
+ sr, err := chatModel.Stream(ctx, inMsgs)
if err != nil {
- log.Fatalf("Stream Recv failed, err=%v", err)
+ log.Fatalf("Stream failed, err=%v", err)
}
- chunks = append(chunks, msgChunk)
- }
+ chunks := make([]*schema.Message, 0, 1024)
+ for {
+ msgChunk, err := sr.Recv()
+ if errors.Is(err, io.EOF) {
+ break
+ }
+ if err != nil {
+ log.Fatalf("Stream Recv failed, err=%v", err)
+ }
+
+ chunks = append(chunks, msgChunk)
+ }
- msg, err = schema.ConcatMessages(chunks)
- if err != nil {
- log.Fatalf("ConcatMessages failed, err=%v", err)
- }
+ msg, err = schema.ConcatMessages(chunks)
+ if err != nil {
+ log.Fatalf("ConcatMessages failed, err=%v", err)
+ }
- log.Printf("stream final output:")
- log.Printf(" request_id: %s", ark.GetArkRequestID(msg))
- respBody, _ = json.MarshalIndent(msg, " ", " ")
- log.Printf(" body: %s \n", string(respBody))
+ log.Printf("stream final output:")
+ log.Printf(" request_id: %s", ark.GetArkRequestID(msg))
+ respBody, _ = json.MarshalIndent(msg, " ", " ")
+ log.Printf(" body: %s \n", string(respBody))
}
-
```
### Multimodal Support (Image Understanding)
@@ -426,104 +418,103 @@ func main() {
package main
import (
- "context"
- "encoding/base64"
- "log"
- "os"
+ "context"
+ "encoding/base64"
+ "log"
+ "os"
- "github.com/cloudwego/eino/components/prompt"
- "github.com/cloudwego/eino/compose"
- "github.com/cloudwego/eino/schema"
+ "github.com/cloudwego/eino/components/prompt"
+ "github.com/cloudwego/eino/compose"
+ "github.com/cloudwego/eino/schema"
- "github.com/cloudwego/eino-ext/components/model/ark"
+ "github.com/cloudwego/eino-ext/components/model/ark"
)
func main() {
- ctx := context.Background()
-
- // Get ARK_API_KEY and ARK_MODEL_ID: https://www.volcengine.com/docs/82379/1399008
- chatModel, err := ark.NewChatModel(ctx, &ark.ChatModelConfig{
- APIKey: os.Getenv("ARK_API_KEY"),
- Model: os.Getenv("ARK_MODEL_ID"),
- })
- if err != nil {
- log.Fatalf("NewChatModel failed, err=%v", err)
- }
-
- multiModalMsg := schema.UserMessage("")
- image, err := os.ReadFile("./examples/generate_with_image/eino.png")
- if err != nil {
- log.Fatalf("os.ReadFile failed, err=%v \n", err)
- }
-
- imageStr := base64.StdEncoding.EncodeToString(image)
+ ctx := context.Background()
- multiModalMsg.UserInputMultiContent = []schema.MessageInputPart{
- {
- Type: schema.ChatMessagePartTypeText,
- Text: "What do you see in this image?",
- },
- {
- Type: schema.ChatMessagePartTypeImageURL,
- Image: &schema.MessageInputImage{
- MessagePartCommon: schema.MessagePartCommon{
- Base64Data: &imageStr,
- MIMEType: "image/png",
- },
- Detail: schema.ImageURLDetailAuto,
- },
- },
- }
+ // Get ARK_API_KEY and ARK_MODEL_ID: https://www.volcengine.com/docs/82379/1399008
+ chatModel, err := ark.NewChatModel(ctx, &ark.ChatModelConfig{
+ APIKey: os.Getenv("ARK_API_KEY"),
+ Model: os.Getenv("ARK_MODEL_ID"),
+ })
+ if err != nil {
+ log.Fatalf("NewChatModel failed, err=%v", err)
+ }
- resp, err := chatModel.Generate(ctx, []*schema.Message{
- multiModalMsg,
- })
- if err != nil {
- log.Fatalf("Generate failed, err=%v", err)
- }
+ multiModalMsg := schema.UserMessage("")
+ image, err := os.ReadFile("./examples/generate_with_image/eino.png")
+ if err != nil {
+ log.Fatalf("os.ReadFile failed, err=%v \n", err)
+ }
- log.Printf("Ark ChatModel output:%v \n", resp)
+ imageStr := base64.StdEncoding.EncodeToString(image)
- // demonstrate how to use ChatTemplate to generate with image
- imgPlaceholder := "{img}"
- ctx = context.Background()
- chain := compose.NewChain[map[string]any, *schema.Message]()
- _ = chain.AppendChatTemplate(prompt.FromMessages(schema.FString,
- &schema.Message{
- Role: schema.User,
- UserInputMultiContent: []schema.MessageInputPart{
+ multiModalMsg.UserInputMultiContent = []schema.MessageInputPart{
{
- Type: schema.ChatMessagePartTypeText,
- Text: "What do you see in this image?",
+ Type: schema.ChatMessagePartTypeText,
+ Text: "What do you see in this image?",
},
{
- Type: schema.ChatMessagePartTypeImageURL,
- Image: &schema.MessageInputImage{
- MessagePartCommon: schema.MessagePartCommon{
- Base64Data: &imgPlaceholder,
- MIMEType: "image/png",
+ Type: schema.ChatMessagePartTypeImageURL,
+ Image: &schema.MessageInputImage{
+ MessagePartCommon: schema.MessagePartCommon{
+ Base64Data: &imageStr,
+ MIMEType: "image/png",
+ },
+ Detail: schema.ImageURLDetailAuto,
},
- Detail: schema.ImageURLDetailAuto,
- },
},
- },
- }))
- _ = chain.AppendChatModel(chatModel)
- r, err := chain.Compile(ctx)
- if err != nil {
- log.Fatalf("Compile failed, err=%v", err)
- }
+ }
- resp, err = r.Invoke(ctx, map[string]any{
- "img": imageStr,
- })
- if err != nil {
- log.Fatalf("Run failed, err=%v", err)
- }
+ resp, err := chatModel.Generate(ctx, []*schema.Message{
+ multiModalMsg,
+ })
+ if err != nil {
+ log.Fatalf("Generate failed, err=%v", err)
+ }
- log.Printf("Ark ChatModel output with ChatTemplate:%v \n", resp)
-}
+ log.Printf("Ark ChatModel output:%v \n", resp)
+
+ // demonstrate how to use ChatTemplate to generate with image
+ imgPlaceholder := "{img}"
+ ctx = context.Background()
+ chain := compose.NewChain[map[string]any, *schema.Message]()
+ _ = chain.AppendChatTemplate(prompt.FromMessages(schema.FString,
+ &schema.Message{
+ Role: schema.User,
+ UserInputMultiContent: []schema.MessageInputPart{
+ {
+ Type: schema.ChatMessagePartTypeText,
+ Text: "What do you see in this image?",
+ },
+ {
+ Type: schema.ChatMessagePartTypeImageURL,
+ Image: &schema.MessageInputImage{
+ MessagePartCommon: schema.MessagePartCommon{
+ Base64Data: &imgPlaceholder,
+ MIMEType: "image/png",
+ },
+ Detail: schema.ImageURLDetailAuto,
+ },
+ },
+ },
+ }))
+ _ = chain.AppendChatModel(chatModel)
+ r, err := chain.Compile(ctx)
+ if err != nil {
+ log.Fatalf("Compile failed, err=%v", err)
+ }
+
+ resp, err = r.Invoke(ctx, map[string]any{
+ "img": imageStr,
+ })
+ if err != nil {
+ log.Fatalf("Run failed, err=%v", err)
+ }
+ log.Printf("Ark ChatModel output with ChatTemplate:%v \n", resp)
+}
```
### Streaming Generation
@@ -532,69 +523,68 @@ func main() {
package main
import (
- "context"
- "fmt"
- "io"
- "log"
- "os"
-
- "github.com/cloudwego/eino-ext/components/model/ark"
- "github.com/cloudwego/eino/schema"
- arkModel "github.com/volcengine/volcengine-go-sdk/service/arkruntime/model"
+ "context"
+ "fmt"
+ "io"
+ "log"
+ "os"
+
+ "github.com/cloudwego/eino-ext/components/model/ark"
+ "github.com/cloudwego/eino/schema"
+ arkModel "github.com/volcengine/volcengine-go-sdk/service/arkruntime/model"
)
func main() {
- ctx := context.Background()
+ ctx := context.Background()
- // Get ARK_API_KEY and ARK_MODEL_ID: https://www.volcengine.com/docs/82379/1399008
- chatModel, err := ark.NewChatModel(ctx, &ark.ChatModelConfig{
- APIKey: os.Getenv("ARK_API_KEY"),
- Model: os.Getenv("ARK_MODEL_ID"),
- })
- if err != nil {
- log.Printf("NewChatModel failed, err=%v", err)
- return
- }
+ // Get ARK_API_KEY and ARK_MODEL_ID: https://www.volcengine.com/docs/82379/1399008
+ chatModel, err := ark.NewChatModel(ctx, &ark.ChatModelConfig{
+ APIKey: os.Getenv("ARK_API_KEY"),
+ Model: os.Getenv("ARK_MODEL_ID"),
+ })
+ if err != nil {
+ log.Printf("NewChatModel failed, err=%v", err)
+ return
+ }
- streamMsgs, err := chatModel.Stream(ctx, []*schema.Message{
- {
- Role: schema.User,
- Content: "as a machine, how do you answer user's question?",
- },
- }, ark.WithReasoningEffort(arkModel.ReasoningEffortHigh))
+ streamMsgs, err := chatModel.Stream(ctx, []*schema.Message{
+ {
+ Role: schema.User,
+ Content: "as a machine, how do you answer user's question?",
+ },
+ }, ark.WithReasoningEffort(arkModel.ReasoningEffortHigh))
- if err != nil {
- log.Printf("Generate failed, err=%v", err)
- return
- }
+ if err != nil {
+ log.Printf("Generate failed, err=%v", err)
+ return
+ }
- defer streamMsgs.Close() // do not forget to close the stream
+ defer streamMsgs.Close() // do not forget to close the stream
- msgs := make([]*schema.Message, 0)
+ msgs := make([]*schema.Message, 0)
- log.Printf("typewriter output:")
- for {
- msg, err := streamMsgs.Recv()
- if err == io.EOF {
- break
+ log.Printf("typewriter output:")
+ for {
+ msg, err := streamMsgs.Recv()
+ if err == io.EOF {
+ break
+ }
+ msgs = append(msgs, msg)
+ if err != nil {
+ log.Printf("stream.Recv failed, err=%v", err)
+ return
+ }
+ fmt.Print(msg.Content)
}
- msgs = append(msgs, msg)
+
+ msg, err := schema.ConcatMessages(msgs)
if err != nil {
- log.Printf("stream.Recv failed, err=%v", err)
- return
+ log.Printf("ConcatMessages failed, err=%v", err)
+ return
}
- fmt.Print(msg.Content)
- }
-
- msg, err := schema.ConcatMessages(msgs)
- if err != nil {
- log.Printf("ConcatMessages failed, err=%v", err)
- return
- }
- log.Printf("output: %s \n", msg.Content)
+ log.Printf("output: %s \n", msg.Content)
}
-
```
### Tool Calling
@@ -603,83 +593,82 @@ func main() {
package main
import (
- "context"
- "log"
- "os"
+ "context"
+ "log"
+ "os"
- "github.com/cloudwego/eino-ext/components/model/ark"
- "github.com/cloudwego/eino/schema"
+ "github.com/cloudwego/eino-ext/components/model/ark"
+ "github.com/cloudwego/eino/schema"
)
func main() {
- ctx := context.Background()
+ ctx := context.Background()
- // Get ARK_API_KEY and ARK_MODEL_ID: https://www.volcengine.com/docs/82379/1399008
- chatModel, err := ark.NewChatModel(ctx, &ark.ChatModelConfig{
- APIKey: os.Getenv("ARK_API_KEY"),
- Model: os.Getenv("ARK_MODEL_ID"),
- })
- if err != nil {
- log.Printf("NewChatModel failed, err=%v", err)
- return
- }
+ // Get ARK_API_KEY and ARK_MODEL_ID: https://www.volcengine.com/docs/82379/1399008
+ chatModel, err := ark.NewChatModel(ctx, &ark.ChatModelConfig{
+ APIKey: os.Getenv("ARK_API_KEY"),
+ Model: os.Getenv("ARK_MODEL_ID"),
+ })
+ if err != nil {
+ log.Printf("NewChatModel failed, err=%v", err)
+ return
+ }
- err = chatModel.BindTools([]*schema.ToolInfo{
- {
- Name: "user_company",
- Desc: "Query the user's company and position information based on their name and email",
- ParamsOneOf: schema.NewParamsOneOfByParams(
- map[string]*schema.ParameterInfo{
- "name": {
- Type: "string",
- Desc: "The user's name",
- },
- "email": {
- Type: "string",
- Desc: "The user's email",
- },
- }),
- },
- {
- Name: "user_salary",
- Desc: "Query the user's salary information based on their name and email",
- ParamsOneOf: schema.NewParamsOneOfByParams(
- map[string]*schema.ParameterInfo{
- "name": {
- Type: "string",
- Desc: "The user's name",
- },
- "email": {
- Type: "string",
- Desc: "The user's email",
- },
- }),
- },
- })
- if err != nil {
- log.Printf("BindForcedTools failed, err=%v", err)
- return
- }
+ err = chatModel.BindTools([]*schema.ToolInfo{
+ {
+ Name: "user_company",
+ Desc: "Query the user's company and position information based on their name and email",
+ ParamsOneOf: schema.NewParamsOneOfByParams(
+ map[string]*schema.ParameterInfo{
+ "name": {
+ Type: "string",
+ Desc: "The user's name",
+ },
+ "email": {
+ Type: "string",
+ Desc: "The user's email",
+ },
+ }),
+ },
+ {
+ Name: "user_salary",
+ Desc: "Query the user's salary information based on their name and email",
+ ParamsOneOf: schema.NewParamsOneOfByParams(
+ map[string]*schema.ParameterInfo{
+ "name": {
+ Type: "string",
+ Desc: "The user's name",
+ },
+ "email": {
+ Type: "string",
+ Desc: "The user's email",
+ },
+ }),
+ },
+ })
+ if err != nil {
+ log.Printf("BindForcedTools failed, err=%v", err)
+ return
+ }
- resp, err := chatModel.Generate(ctx, []*schema.Message{
- {
- Role: schema.System,
- Content: "You are a real estate agent. Use the user_company and user_salary APIs to provide relevant property information based on the user's salary and job. Email is required",
- },
- {
- Role: schema.User,
- Content: "My name is zhangsan, and my email is zhangsan@bytedance.com. Please recommend some suitable houses for me.",
- },
- })
+ resp, err := chatModel.Generate(ctx, []*schema.Message{
+ {
+ Role: schema.System,
+ Content: "You are a real estate agent. Use the user_company and user_salary APIs to provide relevant property information based on the user's salary and job. Email is required",
+ },
+ {
+ Role: schema.User,
+ Content: "My name is zhangsan, and my email is zhangsan@bytedance.com. Please recommend some suitable houses for me.",
+ },
+ })
- if err != nil {
- log.Printf("Generate failed, err=%v", err)
- return
- }
+ if err != nil {
+ log.Printf("Generate failed, err=%v", err)
+ return
+ }
- log.Printf("output:%v \n", resp)
+ log.Printf("output:%v \n", resp)
}
-
```
### Image Generation
@@ -688,104 +677,103 @@ func main() {
package main
import (
- "context"
- "encoding/json"
- "errors"
- "io"
- "log"
- "os"
+ "context"
+ "encoding/json"
+ "errors"
+ "io"
+ "log"
+ "os"
- "github.com/volcengine/volcengine-go-sdk/service/arkruntime/model"
+ "github.com/volcengine/volcengine-go-sdk/service/arkruntime/model"
- "github.com/cloudwego/eino-ext/components/model/ark"
- "github.com/cloudwego/eino/schema"
+ "github.com/cloudwego/eino-ext/components/model/ark"
+ "github.com/cloudwego/eino/schema"
)
func ptr[T any](v T) *T {
- return &v
+ return &v
}
func main() {
- ctx := context.Background()
+ ctx := context.Background()
- // Get ARK_API_KEY and ARK_MODEL_ID: https://www.volcengine.com/docs/82379/1399008
- imageGenerationModel, err := ark.NewImageGenerationModel(ctx, &ark.ImageGenerationConfig{
- APIKey: os.Getenv("ARK_API_KEY"),
- Model: os.Getenv("ARK_MODEL_ID"),
+ // Get ARK_API_KEY and ARK_MODEL_ID: https://www.volcengine.com/docs/82379/1399008
+ imageGenerationModel, err := ark.NewImageGenerationModel(ctx, &ark.ImageGenerationConfig{
+ APIKey: os.Getenv("ARK_API_KEY"),
+ Model: os.Getenv("ARK_MODEL_ID"),
- // Control the size of image generated by the model.
- Size: "1K",
+ // Control the size of image generated by the model.
+ Size: "1K",
- // Control whether to generate a set of images.
- SequentialImageGeneration: ark.SequentialImageGenerationAuto,
+ // Control whether to generate a set of images.
+ SequentialImageGeneration: ark.SequentialImageGenerationAuto,
- // Control the maximum number of images to generate
- SequentialImageGenerationOption: &model.SequentialImageGenerationOptions{
- MaxImages: ptr(2),
- },
+ // Control the maximum number of images to generate
+ SequentialImageGenerationOption: &model.SequentialImageGenerationOptions{
+ MaxImages: ptr(2),
+ },
- // Control the format of the generated jpeg image.
- ResponseFormat: ark.ImageResponseFormatURL,
+ // Control the format of the generated jpeg image.
+ ResponseFormat: ark.ImageResponseFormatURL,
- // Control whether to add a watermark to the generated image
- DisableWatermark: false,
- })
+ // Control whether to add a watermark to the generated image
+ DisableWatermark: false,
+ })
- if err != nil {
- log.Fatalf("NewChatModel failed, err=%v", err)
- }
-
- inMsgs := []*schema.Message{
- {
- Role: schema.User,
- Content: "generate two images of a cat",
- },
- }
+ if err != nil {
+ log.Fatalf("NewChatModel failed, err=%v", err)
+ }
- // Use ImageGeneration API
- msg, err := imageGenerationModel.Generate(ctx, inMsgs)
- if err != nil {
- log.Fatalf("Generate failed, err=%v", err)
- }
+ inMsgs := []*schema.Message{
+ {
+ Role: schema.User,
+ Content: "generate two images of a cat",
+ },
+ }
- log.Printf("generate output:")
- respBody, _ := json.MarshalIndent(msg, " ", " ")
- log.Printf(" body: %s \n", string(respBody))
+ // Use ImageGeneration API
+ msg, err := imageGenerationModel.Generate(ctx, inMsgs)
+ if err != nil {
+ log.Fatalf("Generate failed, err=%v", err)
+ }
- sr, err := imageGenerationModel.Stream(ctx, inMsgs)
- if err != nil {
- log.Fatalf("Stream failed, err=%v", err)
- }
+ log.Printf("generate output:")
+ respBody, _ := json.MarshalIndent(msg, " ", " ")
+ log.Printf(" body: %s \n", string(respBody))
- log.Printf("stream output:")
- index := 0
- chunks := make([]*schema.Message, 0, 1024)
- for {
- msgChunk, err := sr.Recv()
- if errors.Is(err, io.EOF) {
- break
- }
+ sr, err := imageGenerationModel.Stream(ctx, inMsgs)
if err != nil {
- log.Fatalf("Stream Recv failed, err=%v", err)
+ log.Fatalf("Stream failed, err=%v", err)
}
- chunks = append(chunks, msgChunk)
-
- respBody, _ = json.MarshalIndent(msgChunk, " ", " ")
- log.Printf("stream chunk %d: body: %s\n", index, string(respBody))
- index++
- }
+ log.Printf("stream output:")
+ index := 0
+ chunks := make([]*schema.Message, 0, 1024)
+ for {
+ msgChunk, err := sr.Recv()
+ if errors.Is(err, io.EOF) {
+ break
+ }
+ if err != nil {
+ log.Fatalf("Stream Recv failed, err=%v", err)
+ }
+
+ chunks = append(chunks, msgChunk)
+
+ respBody, _ = json.MarshalIndent(msgChunk, " ", " ")
+ log.Printf("stream chunk %d: body: %s\n", index, string(respBody))
+ index++
+ }
- msg, err = schema.ConcatMessages(chunks)
- if err != nil {
- log.Fatalf("ConcatMessages failed, err=%v", err)
- }
- log.Printf("stream final output:")
- log.Printf(" request_id: %s \n", ark.GetArkRequestID(msg))
- respBody, _ = json.MarshalIndent(msg, " ", " ")
- log.Printf(" body: %s \n", string(respBody))
+ msg, err = schema.ConcatMessages(chunks)
+ if err != nil {
+ log.Fatalf("ConcatMessages failed, err=%v", err)
+ }
+ log.Printf("stream final output:")
+ log.Printf(" request_id: %s \n", ark.GetArkRequestID(msg))
+ respBody, _ = json.MarshalIndent(msg, " ", " ")
+ log.Printf(" body: %s \n", string(respBody))
}
-
```
### ContextAPI Prefix Cache
@@ -794,86 +782,85 @@ func main() {
package main
import (
- "context"
- "encoding/json"
- "io"
- "log"
- "os"
+ "context"
+ "encoding/json"
+ "io"
+ "log"
+ "os"
- "github.com/cloudwego/eino/schema"
+ "github.com/cloudwego/eino/schema"
- "github.com/cloudwego/eino-ext/components/model/ark"
+ "github.com/cloudwego/eino-ext/components/model/ark"
)
func main() {
- ctx := context.Background()
+ ctx := context.Background()
- // Get ARK_API_KEY and ARK_MODEL_ID: https://www.volcengine.com/docs/82379/1399008
- chatModel, err := ark.NewChatModel(ctx, &ark.ChatModelConfig{
- APIKey: os.Getenv("ARK_API_KEY"),
- Model: os.Getenv("ARK_MODEL_ID"),
- })
- if err != nil {
- log.Fatalf("NewChatModel failed, err=%v", err)
- }
+ // Get ARK_API_KEY and ARK_MODEL_ID: https://www.volcengine.com/docs/82379/1399008
+ chatModel, err := ark.NewChatModel(ctx, &ark.ChatModelConfig{
+ APIKey: os.Getenv("ARK_API_KEY"),
+ Model: os.Getenv("ARK_MODEL_ID"),
+ })
+ if err != nil {
+ log.Fatalf("NewChatModel failed, err=%v", err)
+ }
- info, err := chatModel.CreatePrefixCache(ctx, []*schema.Message{
- schema.UserMessage("my name is megumin"),
- }, 3600)
- if err != nil {
- log.Fatalf("CreatePrefix failed, err=%v", err)
- }
+ info, err := chatModel.CreatePrefixCache(ctx, []*schema.Message{
+ schema.UserMessage("my name is megumin"),
+ }, 3600)
+ if err != nil {
+ log.Fatalf("CreatePrefix failed, err=%v", err)
+ }
- inMsgs := []*schema.Message{
- {
- Role: schema.User,
- Content: "what is my name?",
- },
- }
+ inMsgs := []*schema.Message{
+ {
+ Role: schema.User,
+ Content: "what is my name?",
+ },
+ }
- msg, err := chatModel.Generate(ctx, inMsgs, ark.WithCache(&ark.CacheOption{
- APIType: ark.ContextAPI,
- ContextID: &info.ContextID,
- }))
- if err != nil {
- log.Fatalf("Generate failed, err=%v", err)
- }
+ msg, err := chatModel.Generate(ctx, inMsgs, ark.WithCache(&ark.CacheOption{
+ APIType: ark.ContextAPI,
+ ContextID: &info.ContextID,
+ }))
+ if err != nil {
+ log.Fatalf("Generate failed, err=%v", err)
+ }
- log.Printf("\ngenerate output: \n")
- log.Printf(" request_id: %s\n", ark.GetArkRequestID(msg))
- respBody, _ := json.MarshalIndent(msg, " ", " ")
- log.Printf(" body: %s\n", string(respBody))
+ log.Printf("\ngenerate output: \n")
+ log.Printf(" request_id: %s\n", ark.GetArkRequestID(msg))
+ respBody, _ := json.MarshalIndent(msg, " ", " ")
+ log.Printf(" body: %s\n", string(respBody))
- outStreamReader, err := chatModel.Stream(ctx, inMsgs, ark.WithCache(&ark.CacheOption{
- APIType: ark.ContextAPI,
- ContextID: &info.ContextID,
- }))
- if err != nil {
- log.Fatalf("Stream failed, err=%v", err)
- }
+ outStreamReader, err := chatModel.Stream(ctx, inMsgs, ark.WithCache(&ark.CacheOption{
+ APIType: ark.ContextAPI,
+ ContextID: &info.ContextID,
+ }))
+ if err != nil {
+ log.Fatalf("Stream failed, err=%v", err)
+ }
- var msgs []*schema.Message
- for {
- item, e := outStreamReader.Recv()
- if e == io.EOF {
- break
+ var msgs []*schema.Message
+ for {
+ item, e := outStreamReader.Recv()
+ if e == io.EOF {
+ break
+ }
+ if e != nil {
+ log.Fatal(e)
+ }
+
+ msgs = append(msgs, item)
}
- if e != nil {
- log.Fatal(e)
+ msg, err = schema.ConcatMessages(msgs)
+ if err != nil {
+ log.Fatalf("ConcatMessages failed, err=%v", err)
}
-
- msgs = append(msgs, item)
- }
- msg, err = schema.ConcatMessages(msgs)
- if err != nil {
- log.Fatalf("ConcatMessages failed, err=%v", err)
- }
- log.Printf("\nstream output: \n")
- log.Printf(" request_id: %s\n", ark.GetArkRequestID(msg))
- respBody, _ = json.MarshalIndent(msg, " ", " ")
- log.Printf(" body: %s\n", string(respBody))
+ log.Printf("\nstream output: \n")
+ log.Printf(" request_id: %s\n", ark.GetArkRequestID(msg))
+ respBody, _ = json.MarshalIndent(msg, " ", " ")
+ log.Printf(" body: %s\n", string(respBody))
}
-
```
### Response API Prefix Cache
@@ -882,165 +869,485 @@ func main() {
package main
import (
- "context"
- "encoding/json"
- "log"
- "os"
+ "context"
+ "encoding/json"
+ "log"
+ "os"
- "github.com/cloudwego/eino/schema"
+ "github.com/cloudwego/eino/schema"
- "github.com/cloudwego/eino-ext/components/model/ark"
+ "github.com/cloudwego/eino-ext/components/model/ark"
)
func main() {
- ctx := context.Background()
+ ctx := context.Background()
+
+ // Get ARK_API_KEY and ARK_MODEL_ID: https://www.volcengine.com/docs/82379/1399008
+ chatModel, err := ark.NewChatModel(ctx, &ark.ChatModelConfig{
+ APIKey: os.Getenv("ARK_API_KEY"),
+ Model: os.Getenv("ARK_MODEL_ID"),
+ Cache: &ark.CacheConfig{
+ APIType: ptrOf(ark.ResponsesAPI),
+ },
+ })
+ if err != nil {
+ log.Fatalf("NewChatModel failed, err=%v", err)
+ }
+
+ err = chatModel.BindTools([]*schema.ToolInfo{
+ {
+ Name: "article_content_extractor",
+ Desc: "Extract key statements and chapter summaries from the provided article content",
+ ParamsOneOf: schema.NewParamsOneOfByParams(
+ map[string]*schema.ParameterInfo{
+ "content": {
+ Type: schema.String,
+ Desc: "The full article content to analyze and extract key information from",
+ Required: true,
+ },
+ }),
+ },
+ })
+
+ if err != nil {
+ log.Fatalf("BindTools failed, err=%v", err)
+ }
+
+ // create response prefix cache, note: more than 1024 tokens are required, otherwise the prefix cache cannot be created
+ cacheInfo, err := chatModel.CreatePrefixCache(ctx, []*schema.Message{
+ schema.SystemMessage(`Once upon a time, in a quaint little village surrounded by vast green forests and blooming meadows, there lived a spirited young girl known as Little Red Riding Hood. She earned her name from the vibrant red cape that her beloved grandmother had sewn for her, a gift that she cherished deeply. This cape was more than just a piece of clothing; it was a symbol of the bond between her and her grandmother, who lived on the other side of the great woods, near a sparkling brook that bubbled merrily all year round.
+
+ One sunny morning, Little Red Riding Hood's mother called her into the cozy kitchen, where the aroma of freshly baked bread filled the air. "My dear," she said, "your grandmother isn't feeling well today. I want you to take her this basket of treats. There are some delicious cakes, a jar of honey, and her favorite herbal tea. Can you do that for me?"
+
+ Little Red Riding Hood's eyes sparkled with excitement as she nodded eagerly. "Yes, Mama! I'll take good care of them!" Her mother handed her a beautifully woven basket, filled to the brim with goodies, and reminded her, "Remember to stay on the path and don't talk to strangers."
+
+ "I promise, Mama!" she replied confidently, pulling her red hood over her head and setting off on her adventure. The sun shone brightly, and birds chirped merrily as she walked, making her feel like she was in a fairy tale.
+
+ As she journeyed through the woods, the tall trees whispered secrets to one another, and colorful flowers danced in the gentle breeze. Little Red Riding Hood was so enchanted by the beauty around her that she began to hum a tune, her voice harmonizing with the sounds of nature.
+
+ However, unbeknownst to her, lurking in the shadows was a cunning wolf. The wolf was known throughout the forest for his deceptive wit and insatiable hunger. He watched Little Red Riding Hood with keen interest, contemplating his next meal.
+
+ "Good day, little girl!" the wolf called out, stepping onto the path with a friendly yet sly smile.
+
+ Startled, she halted and took a step back. "Hello there! I'm just on my way to visit my grandmother," she replied, clutching the basket tightly.
+
+ "Ah, your grandmother! I know her well," the wolf said, his eyes glinting with mischief. "Why don't you pick some lovely flowers for her? I'm sure she would love them, and I'm sure there are many beautiful ones just off the path."
+
+ Little Red Riding Hood hesitated for a moment but was easily convinced by the wolf's charming suggestion. "That's a wonderful idea! Thank you!" she exclaimed, letting her curiosity pull her away from the safety of the path. As she wandered deeper into the woods, her gaze fixed on the vibrant blooms, the wolf took a shortcut towards her grandmother's house.
+
+ When the wolf arrived at Grandma's quaint cottage, he knocked on the door with a confident swagger. "It's me, Little Red Riding Hood!" he shouted in a high-pitched voice to mimic the girl.
+
+ "Come in, dear!" came the frail voice of the grandmother, who had been resting on her cozy bed, wrapped in warm blankets. The wolf burst through the door, his eyes gleaming with the thrill of his plan.
+
+ With astonishing speed, the wolf gulped down the unsuspecting grandmother whole. Afterward, he dressed in her nightgown, donning her nightcap and climbing into her bed. He lay there, waiting for Little Red Riding Hood to arrive, concealing his wicked smile behind a facade of innocence.
+
+ Meanwhile, Little Red Riding Hood was merrily picking flowers, completely unaware of the impending danger. After gathering a beautiful bouquet of wildflowers, she finally made her way back to the path and excitedly skipped towards her grandmother's cottage.
+
+ Upon arriving, she noticed the door was slightly ajar. "Grandmother, it's me!" she called out, entering the dimly lit home. It was silent, with only the faint sound of an old clock ticking in the background. She stepped into the small living room, a feeling of unease creeping over her.
+
+ "Grandmother, are you here?" she asked, peeking into the bedroom. There, she saw a figure lying under the covers.
+
+ "Grandmother, what big ears you have!" she exclaimed, taking a few cautious steps closer.
+
+ "All the better to hear you with, my dear," the wolf replied in a voice that was deceptively sweet.
+
+ "Grandmother, what big eyes you have!" Little Red Riding Hood continued, now feeling an unsettling chill in the air.
+
+ "All the better to see you with, my dear," the wolf said, his eyes narrowing as he tried to contain his glee.
+
+ "Grandmother, what big teeth you have!" she exclaimed, the terror flooding her senses as she began to realize this was no ordinary visit.
+
+ "All the better to eat you with!" the wolf roared, springing out of the bed with startling speed.
+
+ Just as the wolf lunged towards her, a brave woodsman, who had been passing by the cottage and heard the commotion, burst through the door. His strong presence was a beacon of hope in the dire situation. "Stay back, wolf!" he shouted with authority, brandishing his axe.
+
+ The wolf, taken aback by the sudden intrusion, hesitated for a moment. Before he could react, the woodsman swung his axe with determination, and with a swift motion, he drove the wolf away, rescuing Little Red Riding Hood and her grandmother from certain doom.
+
+ Little Red Riding Hood was shaking with fright, but relief washed over her as the woodsman helped her grandmother out from behind the bed where the wolf had hidden her. The grandmother, though shaken, was immensely grateful to the woodsman for his bravery. "Thank you so much! You saved us!" she cried, embracing him warmly.
+
+ Little Red Riding Hood, still in shock but filled with gratitude, looked up at the woodsman and said, "I promise I will never stray from the path again. Thank you for being our hero!"
+
+ From that day on, the woodland creatures spoke of the brave woodsman who saved Little Red Riding Hood and her grandmother. Little Red Riding Hood learned a valuable lesson about being cautious and listening to her mother's advice. The bond between her and her grandmother grew stronger, and they often reminisced about that day's adventure over cups of tea, surrounded by cookies and laughter.
+
+ To ensure safety, Little Red Riding Hood always took extra precautions when traveling through the woods, carrying a small whistle her grandmother had given her. It would alert anyone nearby if she ever found herself in trouble again.
+
+ And so, in the heart of that small village, life continued, filled with love, laughter, and the occasional adventure, as Little Red Riding Hood and her grandmother thrived, forever grateful for the friendship of the woodsman who had acted as their guardian that fateful day.
+
+ And they all lived happily ever after.
+
+ The end.`),
+ }, 300)
+ if err != nil {
+ log.Fatalf("CreatePrefixCache failed, err=%v", err)
+ }
+
+ // use cache information in subsequent requests
+ cacheOpt := &ark.CacheOption{
+ APIType: ark.ResponsesAPI,
+ HeadPreviousResponseID: &cacheInfo.ResponseID,
+ }
+
+ outMsg, err := chatModel.Generate(ctx, []*schema.Message{
+ schema.UserMessage("What is the main idea expressed above？"),
+ }, ark.WithCache(cacheOpt))
+
+ if err != nil {
+ log.Fatalf("Generate failed, err=%v", err)
+ }
+
+ respID, ok := ark.GetResponseID(outMsg)
+ if !ok {
+ log.Fatalf("not found response id in message")
+ }
+
+ log.Printf("\ngenerate output: \n")
+ log.Printf(" request_id: %s\n", respID)
+ respBody, _ := json.MarshalIndent(outMsg, " ", " ")
+ log.Printf(" body: %s\n", string(respBody))
+}
+func ptrOf[T any](v T) *T {
+ return &v
+
+}
+```
+
+When you don't want to use cache for messages that have already been cached, you can call `InvalidateMessageCaches(messages []*schema.Message) error` to clear the cache markers in the messages. This way, when the ARK SDK constructs the Responses API request, it cannot find the corresponding ResponseID based on the cache marker, and thus will not assign a value to PreviousResponseID in the Responses API.
+
+### ContextAPI Session Cache
+
+```go
+package main
+
+import (
+ "context"
+ "encoding/json"
+ "fmt"
+ "io"
+ "log"
+ "os"
+ "time"
+
+ arkModel "github.com/volcengine/volcengine-go-sdk/service/arkruntime/model"
+
+ "github.com/cloudwego/eino-ext/components/model/ark"
+ "github.com/cloudwego/eino/schema"
+)
+
+func main() {
+ ctx := context.Background()
+
+ // Get ARK_API_KEY and ARK_MODEL_ID: https://www.volcengine.com/docs/82379/1399008
+ chatModel, err := ark.NewChatModel(ctx, &ark.ChatModelConfig{
+ APIKey: os.Getenv("ARK_API_KEY"),
+ Model: os.Getenv("ARK_MODEL_ID"),
+ })
+ if err != nil {
+ log.Fatalf("NewChatModel failed, err=%v", err)
+ }
- // Get ARK_API_KEY and ARK_MODEL_ID: https://www.volcengine.com/docs/82379/1399008
- chatModel, err := ark.NewChatModel(ctx, &ark.ChatModelConfig{
- APIKey: os.Getenv("ARK_API_KEY"),
- Model: os.Getenv("ARK_MODEL_ID"),
- Cache: &ark.CacheConfig{
- APIType: ptrOf(ark.ResponsesAPI),
+ instructions := []*schema.Message{
+ schema.SystemMessage("Your name is superman"),
+ }
+
+ cacheInfo, err := chatModel.CreateSessionCache(ctx, instructions, 86400, nil)
+ if err != nil {
+ log.Fatalf("CreateSessionCache failed, err=%v", err)
+ }
+
+ thinking := &arkModel.Thinking{
+ Type: arkModel.ThinkingTypeDisabled,
+ }
+
+ cacheOpt := &ark.CacheOption{
+ APIType: ark.ContextAPI,
+ ContextID: &cacheInfo.ContextID,
+ SessionCache: &ark.SessionCacheConfig{
+ EnableCache: true,
+ TTL: 86400,
+ },
+ }
+
+ msg, err := chatModel.Generate(ctx, instructions,
+ ark.WithThinking(thinking),
+ ark.WithCache(cacheOpt))
+ if err != nil {
+ log.Fatalf("Generate failed, err=%v", err)
+ }
+
+ <-time.After(500 * time.Millisecond)
+
+ msg, err = chatModel.Generate(ctx, []*schema.Message{
+ {
+ Role: schema.User,
+ Content: "What's your name?",
+ },
},
- })
- if err != nil {
- log.Fatalf("NewChatModel failed, err=%v", err)
- }
+ ark.WithThinking(thinking),
+ ark.WithCache(cacheOpt))
+ if err != nil {
+ log.Fatalf("Generate failed, err=%v", err)
+ }
- err = chatModel.BindTools([]*schema.ToolInfo{
- {
- Name: "article_content_extractor",
- Desc: "Extract key statements and chapter summaries from the provided article content",
- ParamsOneOf: schema.NewParamsOneOfByParams(
- map[string]*schema.ParameterInfo{
- "content": {
- Type: schema.String,
- Desc: "The full article content to analyze and extract key information from",
- Required: true,
- },
- }),
+ fmt.Printf("\ngenerate output: \n")
+ fmt.Printf(" request_id: %s\n", ark.GetArkRequestID(msg))
+ respBody, _ := json.MarshalIndent(msg, " ", " ")
+ fmt.Printf(" body: %s\n", string(respBody))
+
+ outStreamReader, err := chatModel.Stream(ctx, []*schema.Message{
+ {
+ Role: schema.User,
+ Content: "What do I ask you last time?",
+ },
},
- })
+ ark.WithThinking(thinking),
+ ark.WithCache(cacheOpt))
+ if err != nil {
+ log.Fatalf("Stream failed, err=%v", err)
+ }
- if err != nil {
- log.Fatalf("BindTools failed, err=%v", err)
- }
+ fmt.Println("\ntypewriter output:")
+ var msgs []*schema.Message
+ for {
+ item, e := outStreamReader.Recv()
+ if e == io.EOF {
+ break
+ }
+ if e != nil {
+ log.Fatal(e)
+ }
+
+ fmt.Print(item.Content)
+ msgs = append(msgs, item)
+ }
- // create response prefix cache, note: more than 1024 tokens are required, otherwise the prefix cache cannot be created
- cacheInfo, err := chatModel.CreatePrefixCache(ctx, []*schema.Message{
- schema.SystemMessage(`Once upon a time, in a quaint little village surrounded by vast green forests and blooming meadows, there lived a spirited young girl known as Little Red Riding Hood. She earned her name from the vibrant red cape that her beloved grandmother had sewn for her, a gift that she cherished deeply. This cape was more than just a piece of clothing; it was a symbol of the bond between her and her grandmother, who lived on the other side of the great woods, near a sparkling brook that bubbled merrily all year round.
-
- One sunny morning, Little Red Riding Hood's mother called her into the cozy kitchen, where the aroma of freshly baked bread filled the air. “My dear,” she said, “your grandmother isn’t feeling well today. I want you to take her this basket of treats. There are some delicious cakes, a jar of honey, and her favorite herbal tea. Can you do that for me?”
-
- Little Red Riding Hood’s eyes sparkled with excitement as she nodded eagerly. “Yes, Mama! I’ll take good care of them!” Her mother handed her a beautifully woven basket, filled to the brim with goodies, and reminded her, “Remember to stay on the path and don’t talk to strangers.”
-
- “I promise, Mama!” she replied confidently, pulling her red hood over her head and setting off on her adventure. The sun shone brightly, and birds chirped merrily as she walked, making her feel like she was in a fairy tale.
-
- As she journeyed through the woods, the tall trees whispered secrets to one another, and colorful flowers danced in the gentle breeze. Little Red Riding Hood was so enchanted by the beauty around her that she began to hum a tune, her voice harmonizing with the sounds of nature.
-
- However, unbeknownst to her, lurking in the shadows was a cunning wolf. The wolf was known throughout the forest for his deceptive wit and insatiable hunger. He watched Little Red Riding Hood with keen interest, contemplating his next meal.
-
- “Good day, little girl!” the wolf called out, stepping onto the path with a friendly yet sly smile.
-
- Startled, she halted and took a step back. “Hello there! I’m just on my way to visit my grandmother,” she replied, clutching the basket tightly.
-
- “Ah, your grandmother! I know her well,” the wolf said, his eyes glinting with mischief. “Why don’t you pick some lovely flowers for her? I’m sure she would love them, and I’m sure there are many beautiful ones just off the path.”
-
- Little Red Riding Hood hesitated for a moment but was easily convinced by the wolf’s charming suggestion. “That’s a wonderful idea! Thank you!” she exclaimed, letting her curiosity pull her away from the safety of the path. As she wandered deeper into the woods, her gaze fixed on the vibrant blooms, the wolf took a shortcut towards her grandmother’s house.
-
- When the wolf arrived at Grandma’s quaint cottage, he knocked on the door with a confident swagger. “It’s me, Little Red Riding Hood!” he shouted in a high-pitched voice to mimic the girl.
-
- “Come in, dear!” came the frail voice of the grandmother, who had been resting on her cozy bed, wrapped in warm blankets. The wolf burst through the door, his eyes gleaming with the thrill of his plan.
-
- With astonishing speed, the wolf gulped down the unsuspecting grandmother whole. Afterward, he dressed in her nightgown, donning her nightcap and climbing into her bed. He lay there, waiting for Little Red Riding Hood to arrive, concealing his wicked smile behind a facade of innocence.
-
- Meanwhile, Little Red Riding Hood was merrily picking flowers, completely unaware of the impending danger. After gathering a beautiful bouquet of wildflowers, she finally made her way back to the path and excitedly skipped towards her grandmother’s cottage.
-
- Upon arriving, she noticed the door was slightly ajar. “Grandmother, it’s me!” she called out, entering the dimly lit home. It was silent, with only the faint sound of an old clock ticking in the background. She stepped into the small living room, a feeling of unease creeping over her.
-
- “Grandmother, are you here?” she asked, peeking into the bedroom. There, she saw a figure lying under the covers.
-
- “Grandmother, what big ears you have!” she exclaimed, taking a few cautious steps closer.
-
- “All the better to hear you with, my dear,” the wolf replied in a voice that was deceptively sweet.
-
- “Grandmother, what big eyes you have!” Little Red Riding Hood continued, now feeling an unsettling chill in the air.
-
- “All the better to see you with, my dear,” the wolf said, his eyes narrowing as he tried to contain his glee.
-
- “Grandmother, what big teeth you have!” she exclaimed, the terror flooding her senses as she began to realize this was no ordinary visit.
-
- “All the better to eat you with!” the wolf roared, springing out of the bed with startling speed.
-
- Just as the wolf lunged towards her, a brave woodsman, who had been passing by the cottage and heard the commotion, burst through the door. His strong presence was a beacon of hope in the dire situation. “Stay back, wolf!” he shouted with authority, brandishing his axe.
-
- The wolf, taken aback by the sudden intrusion, hesitated for a moment. Before he could react, the woodsman swung his axe with determination, and with a swift motion, he drove the wolf away, rescuing Little Red Riding Hood and her grandmother from certain doom.
-
- Little Red Riding Hood was shaking with fright, but relief washed over her as the woodsman helped her grandmother out from behind the bed where the wolf had hidden her. The grandmother, though shaken, was immensely grateful to the woodsman for his bravery. “Thank you so much! You saved us!” she cried, embracing him warmly.
-
- Little Red Riding Hood, still in shock but filled with gratitude, looked up at the woodsman and said, “I promise I will never stray from the path again. Thank you for being our hero!”
-
- From that day on, the woodland creatures spoke of the brave woodsman who saved Little Red Riding Hood and her grandmother. Little Red Riding Hood learned a valuable lesson about being cautious and listening to her mother’s advice. The bond between her and her grandmother grew stronger, and they often reminisced about that day’s adventure over cups of tea, surrounded by cookies and laughter.
-
- To ensure safety, Little Red Riding Hood always took extra precautions when traveling through the woods, carrying a small whistle her grandmother had given her. It would alert anyone nearby if she ever found herself in trouble again.
-
- And so, in the heart of that small village, life continued, filled with love, laughter, and the occasional adventure, as Little Red Riding Hood and her grandmother thrived, forever grateful for the friendship of the woodsman who had acted as their guardian that fateful day.
-
- And they all lived happily ever after.
-
- The end.`),
- }, 300)
- if err != nil {
- log.Fatalf("CreatePrefixCache failed, err=%v", err)
- }
+ msg, err = schema.ConcatMessages(msgs)
+ if err != nil {
+ log.Fatalf("ConcatMessages failed, err=%v", err)
+ }
+ fmt.Print("\n\nstream output: \n")
+ fmt.Printf(" request_id: %s\n", ark.GetArkRequestID(msg))
+ respBody, _ = json.MarshalIndent(msg, " ", " ")
+ fmt.Printf(" body: %s\n", string(respBody))
+}
+```
- // use cache information in subsequent requests
- cacheOpt := &ark.CacheOption{
- APIType: ark.ResponsesAPI,
- HeadPreviousResponseID: &cacheInfo.ResponseID,
- }
+### ResponseAPI Session Cache
- outMsg, err := chatModel.Generate(ctx, []*schema.Message{
- schema.UserMessage("What is the main idea expressed above？"),
- }, ark.WithCache(cacheOpt))
+```go
+package main
- if err != nil {
- log.Fatalf("Generate failed, err=%v", err)
- }
+import (
+ "context"
+ "fmt"
+ "io"
+ "log"
+ "os"
- respID, ok := ark.GetResponseID(outMsg)
- if !ok {
- log.Fatalf("not found response id in message")
- }
+ arkModel "github.com/volcengine/volcengine-go-sdk/service/arkruntime/model"
- log.Printf("\ngenerate output: \n")
- log.Printf(" request_id: %s\n", respID)
- respBody, _ := json.MarshalIndent(outMsg, " ", " ")
- log.Printf(" body: %s\n", string(respBody))
+ "github.com/cloudwego/eino/schema"
+
+ "github.com/cloudwego/eino-ext/components/model/ark"
+)
+
+func main() {
+ ctx := context.Background()
+
+ // Get ARK_API_KEY and ARK_MODEL_ID: https://www.volcengine.com/docs/82379/1399008
+ chatModel, err := ark.NewChatModel(ctx, &ark.ChatModelConfig{
+ APIKey: os.Getenv("ARK_API_KEY"),
+ Model: os.Getenv("ARK_MODEL_ID"),
+ Cache: &ark.CacheConfig{
+ SessionCache: &ark.SessionCacheConfig{
+ EnableCache: true,
+ TTL: 86400,
+ },
+ },
+ })
+ if err != nil {
+ log.Fatalf("NewChatModel failed, err=%v", err)
+ }
+
+ thinking := &arkModel.Thinking{
+ Type: arkModel.ThinkingTypeDisabled,
+ }
+ cacheOpt := &ark.CacheOption{
+ APIType: ark.ResponsesAPI,
+ SessionCache: &ark.SessionCacheConfig{
+ EnableCache: true,
+ TTL: 86400,
+ },
+ }
+
+ useMsgs := []*schema.Message{
+ schema.UserMessage("Your name is superman"),
+ schema.UserMessage("What's your name?"),
+ schema.UserMessage("What do I ask you last time?"),
+ }
+
+ var input []*schema.Message
+ for _, msg := range useMsgs {
+ input = append(input, msg)
+
+ streamResp, err := chatModel.Stream(ctx, input,
+ ark.WithThinking(thinking),
+ ark.WithCache(cacheOpt))
+ if err != nil {
+ log.Fatalf("Stream failed, err=%v", err)
+ }
+
+ var messages []*schema.Message
+ for {
+ chunk, err := streamResp.Recv()
+ if err == io.EOF {
+ break
+ }
+ if err != nil {
+ log.Fatalf("Recv of streamResp failed, err=%v", err)
+ }
+ messages = append(messages, chunk)
+ }
+
+ resp, err := schema.ConcatMessages(messages)
+ if err != nil {
+ log.Fatalf("ConcatMessages of ark failed, err=%v", err)
+ }
+
+ fmt.Printf("stream output: \n%v\n\n", resp)
+
+ input = append(input, resp)
+ }
}
-func ptrOf[T any](v T) *T {
- return &v
+```
+
+When you don't want to use cache for messages that have already been cached, you can call `InvalidateMessageCaches(messages []*schema.Message) error` to clear the cache markers in the messages. This way, when the ARK SDK constructs the Responses API request, it cannot find the corresponding ResponseID based on the cache marker, and thus will not assign a value to PreviousResponseID in the Responses API.
+
+## **Usage**
+
+### ChatModel
+
+#### **Component Initialization**
+
+The Ark model is initialized through the `NewChatModel` function with the following main configuration parameters:
+
+```go
+import "github.com/cloudwego/eino-ext/components/model/ark"
+
+model, err := ark.NewChatModel(ctx, &ark.ChatModelConfig{
+ // Service configuration
+ BaseURL: "https://ark.cn-beijing.volces.com/api/v3", // Service URL
+ Region: "cn-beijing", // Region
+ HTTPClient: httpClient, // Custom HTTP client
+ Timeout: &timeout, // Timeout
+ RetryTimes: &retries, // Retry times
+
+ // Authentication configuration (choose one)
+ APIKey: "your-api-key", // API Key authentication
+ AccessKey: "your-ak", // AK/SK authentication
+ SecretKey: "your-sk",
+
+ // Model configuration
+ Model: "endpoint-id", // Model endpoint ID
+
+ // Generation parameters
+ MaxTokens: &maxTokens, // Maximum generation length
+ Temperature: &temp, // Temperature
+ TopP: &topP, // Top-P sampling
+ Stop: []string{}, // Stop words
+ FrequencyPenalty: &fp, // Frequency penalty
+ PresencePenalty: &pp, // Presence penalty
+
+ // Advanced parameters
+ LogitBias: map[string]int{}, // Token bias
+ CustomHeader: map[string]string{}, // HTTP custom header
+})
+```
+
+#### **Generate Conversation**
+Conversation generation supports both normal mode and streaming mode:
+
+```go
+func main() {
+ // Normal mode
+ response, err := model.Generate(ctx, messages)
+
+ // Streaming mode
+ stream, err := model.Stream(ctx, messages)
}
+```
+
+Message format example:
+> Note: Whether multimodal images are supported depends on the specific model
+
+```go
+func main() {
+ imgUrl := "https://example.com/image.jpg",
+ messages := []*schema.Message{
+ // System message
+ schema.SystemMessage("You are an assistant"),
+
+ // Multimodal message (with image)
+ {
+ Role: schema.User,
+ UserInputMultiContent: []schema.MessageInputPart{
+ {
+ Type: schema.ChatMessagePartTypeText,
+ Text: "What is this image?",
+ },
+ {
+ Type: schema.ChatMessagePartTypeImageURL,
+ Image: &schema.MessageInputImage{
+ MessagePartCommon: schema.MessagePartCommon{
+ URL: &imgUrl,
+ },
+ Detail: schema.ImageURLDetailAuto,
+ },
+ },
+ },
+ },
+ }
+}
```
-### ContextAPI Session Cache
+#### **Tool Calling**
+
+Supports binding tools:
+
+```go
+// Define tools
+tools := []*schema.ToolInfo{
+ {
+ Name: "search",
+ Desc: "Search for information",
+ ParamsOneOf: schema.NewParamsOneOfByParams(map[string]*schema.ParameterInfo{
+ "query": {
+ Type: schema.String,
+ Desc: "Search keywords",
+ Required: true,
+ },
+ }),
+ },
+}
+
+// Bind tools
+err := model.BindTools(tools)
+```
+
+> For tool-related information, please refer to [Eino: ToolsNode Guide](/docs/eino/core_modules/components/tools_node_guide)
+
+#### **Complete Usage Examples**
+
+##### **Direct Conversation**
```go
package main
import (
"context"
- "encoding/json"
- "fmt"
- "io"
- "log"
- "os"
"time"
- arkModel "github.com/volcengine/volcengine-go-sdk/service/arkruntime/model"
-
"github.com/cloudwego/eino-ext/components/model/ark"
"github.com/cloudwego/eino/schema"
)
@@ -1048,197 +1355,278 @@ import (
func main() {
ctx := context.Background()
- // Get ARK_API_KEY and ARK_MODEL_ID: https://www.volcengine.com/docs/82379/1399008
- chatModel, err := ark.NewChatModel(ctx, &ark.ChatModelConfig{
- APIKey: os.Getenv("ARK_API_KEY"),
- Model: os.Getenv("ARK_MODEL_ID"),
+ timeout := 30 * time.Second
+ // Initialize model
+ model, err := ark.NewChatModel(ctx, &ark.ChatModelConfig{
+ APIKey: "your-api-key",
+ Region: "cn-beijing",
+ Model: "endpoint-id",
+ Timeout: &timeout,
})
if err != nil {
- log.Fatalf("NewChatModel failed, err=%v", err)
+ panic(err)
}
- instructions := []*schema.Message{
- schema.SystemMessage("Your name is superman"),
+ // Prepare messages
+ messages := []*schema.Message{
+ schema.SystemMessage("You are an assistant"),
+ schema.UserMessage("Introduce Volcengine"),
}
- cacheInfo, err := chatModel.CreateSessionCache(ctx, instructions, 86400, nil)
+ // Generate response
+ response, err := model.Generate(ctx, messages)
if err != nil {
- log.Fatalf("CreateSessionCache failed, err=%v", err)
+ panic(err)
}
- thinking := &arkModel.Thinking{
- Type: arkModel.ThinkingTypeDisabled,
- }
+ // Process response
+ println(response.Content)
- cacheOpt := &ark.CacheOption{
- APIType: ark.ContextAPI,
- ContextID: &cacheInfo.ContextID,
- SessionCache: &ark.SessionCacheConfig{
- EnableCache: true,
- TTL: 86400,
- },
+ // Get token usage
+ if usage := response.ResponseMeta.Usage; usage != nil {
+ println("Prompt Tokens:", usage.PromptTokens)
+ println("Completion Tokens:", usage.CompletionTokens)
+ println("Total Tokens:", usage.TotalTokens)
}
+}
+```
- msg, err := chatModel.Generate(ctx, instructions,
- ark.WithThinking(thinking),
- ark.WithCache(cacheOpt))
- if err != nil {
- log.Fatalf("Generate failed, err=%v", err)
- }
+##### **Streaming Conversation**
- <-time.After(500 * time.Millisecond)
+```go
+package main
- msg, err = chatModel.Generate(ctx, []*schema.Message{
- {
- Role: schema.User,
- Content: "What's your name?",
- },
- },
- ark.WithThinking(thinking),
- ark.WithCache(cacheOpt))
+import (
+ "context"
+ "time"
+
+ "github.com/cloudwego/eino-ext/components/model/ark"
+ "github.com/cloudwego/eino/schema"
+)
+
+func main() {
+ ctx := context.Background()
+
+ // Initialize model
+ model, err := ark.NewChatModel(ctx, &ark.ChatModelConfig{
+ APIKey: "your-api-key",
+ Model: "ep-xxx",
+ })
if err != nil {
- log.Fatalf("Generate failed, err=%v", err)
+ panic(err)
}
-
- fmt.Printf("\ngenerate output: \n")
- fmt.Printf(" request_id: %s\n", ark.GetArkRequestID(msg))
- respBody, _ := json.MarshalIndent(msg, " ", " ")
- fmt.Printf(" body: %s\n", string(respBody))
-
- outStreamReader, err := chatModel.Stream(ctx, []*schema.Message{
- {
- Role: schema.User,
- Content: "What do I ask you last time?",
- },
- },
- ark.WithThinking(thinking),
- ark.WithCache(cacheOpt))
+
+ // Prepare messages
+ messages := []*schema.Message{
+ schema.SystemMessage("You are an assistant"),
+ schema.UserMessage("Introduce Eino"),
+ }
+
+ // Get streaming response
+ reader, err := model.Stream(ctx, messages)
if err != nil {
- log.Fatalf("Stream failed, err=%v", err)
+ panic(err)
}
-
- fmt.Println("\ntypewriter output:")
- var msgs []*schema.Message
+ defer reader.Close() // Remember to close
+
+ // Process streaming content
for {
- item, e := outStreamReader.Recv()
- if e == io.EOF {
+ chunk, err := reader.Recv()
+ if err != nil {
break
}
- if e != nil {
- log.Fatal(e)
- }
-
- fmt.Print(item.Content)
- msgs = append(msgs, item)
- }
-
- msg, err = schema.ConcatMessages(msgs)
- if err != nil {
- log.Fatalf("ConcatMessages failed, err=%v", err)
+ print(chunk.Content)
}
- fmt.Print("\n\nstream output: \n")
- fmt.Printf(" request_id: %s\n", ark.GetArkRequestID(msg))
- respBody, _ = json.MarshalIndent(msg, " ", " ")
- fmt.Printf(" body: %s\n", string(respBody))
}
+```
+
+### ImageGeneration Model
+#### Component Initialization
+
+The Seedream (seedream4.0) image generation model is initialized through the `NewImageGenerationModel` function:
+
+```go
+import "github.com/cloudwego/eino-ext/components/model/ark"
+
+// Get ARK_API_KEY and ARK_MODEL_ID: https://www.volcengine.com/docs/82379/1399008
+imageGenerationModel, err := ark.NewImageGenerationModel(ctx, &ark.ImageGenerationConfig{
+ // Service configuration
+ BaseURL: "https://ark.cn-beijing.volces.com/api/v3", // Service URL
+ Region: "cn-beijing", // Region
+ HTTPClient: httpClient, // Custom HTTP client
+ Timeout: &timeout, // Timeout
+ RetryTimes: &retries, // Retry times
+
+ // Model configuration
+ APIKey: os.Getenv("ARK_API_KEY"),
+ Model: os.Getenv("ARK_MODEL_ID"),
+
+ // Generation configuration
+ Size: "1K", // Specify generated image size
+ SequentialImageGeneration: ark.SequentialImageGenerationAuto, // Determine whether to generate a set of images
+ SequentialImageGenerationOption: &model.SequentialImageGenerationOptions{ // Maximum number of images when generating a set
+ MaxImages: ptr(2),
+ },
+ ResponseFormat: ark.ImageResponseFormatURL, // Image data return method, URL or Base64
+ DisableWatermark: false, // Whether to include "AI Generated" watermark
+})
```
-### ResponseAPI Session Cache
+#### Generate Conversation
+
+The image generation model also supports both normal mode and streaming mode:
```go
-package main
+func main() {
+ // Normal mode
+ response, err := model.Generate(ctx, messages)
+
+ // Streaming mode
+ stream, err := model.Stream(ctx, messages)
+}
+```
+
+#### Complete Usage Examples
+##### Direct Conversation
+
+```go
import (
"context"
- "fmt"
+ "encoding/json"
+ "errors"
"io"
"log"
"os"
- arkModel "github.com/volcengine/volcengine-go-sdk/service/arkruntime/model"
-
- "github.com/cloudwego/eino/schema"
+ "github.com/volcengine/volcengine-go-sdk/service/arkruntime/model"
"github.com/cloudwego/eino-ext/components/model/ark"
+ "github.com/cloudwego/eino/schema"
)
+// Pointer helper function
+func ptr[T any](v T) *T {
+ return &v
+}
+
func main() {
+ // Initialize model
ctx := context.Background()
-
- // Get ARK_API_KEY and ARK_MODEL_ID: https://www.volcengine.com/docs/82379/1399008
- chatModel, err := ark.NewChatModel(ctx, &ark.ChatModelConfig{
- APIKey: os.Getenv("ARK_API_KEY"),
- Model: os.Getenv("ARK_MODEL_ID"),
- Cache: &ark.CacheConfig{
- SessionCache: &ark.SessionCacheConfig{
- EnableCache: true,
- TTL: 86400,
- },
- },
+ imageGenerationModel, err := ark.NewImageGenerationModel(ctx, &ark.ImageGenerationConfig{
+ APIKey: os.Getenv("ARK_API_KEY"),
+ Model: os.Getenv("ARK_MODEL_ID"),
+ Size: "1920x1080",
+ SequentialImageGeneration: ark.SequentialImageGenerationDisabled,
+ ResponseFormat: ark.ImageResponseFormatURL,
+ DisableWatermark: false,
})
+
if err != nil {
- log.Fatalf("NewChatModel failed, err=%v", err)
+ log.Fatalf("NewChatModel failed, err=%v", err)
}
- thinking := &arkModel.Thinking{
- Type: arkModel.ThinkingTypeDisabled,
- }
- cacheOpt := &ark.CacheOption{
- APIType: ark.ResponsesAPI,
- SessionCache: &ark.SessionCacheConfig{
- EnableCache: true,
- TTL: 86400,
- },
+ // Prepare messages
+ inMsgs := []*schema.Message{
+ {
+ Role: schema.User,
+ Content: "generate two images of a cat",
+ },
}
- useMsgs := []*schema.Message{
- schema.UserMessage("Your name is superman"),
- schema.UserMessage("What's your name?"),
- schema.UserMessage("What do I ask you last time?"),
+ // Generate image
+ msg, err := imageGenerationModel.Generate(ctx, inMsgs)
+ if err != nil {
+ log.Fatalf("Generate failed, err=%v", err)
}
- var input []*schema.Message
- for _, msg := range useMsgs {
- input = append(input, msg)
+ // Print image information
+ log.Printf("\ngenerate output: \n")
+ respBody, _ := json.MarshalIndent(msg, " ", " ")
+ log.Printf(" body: %s\n", string(respBody))
+}
+```
- streamResp, err := chatModel.Stream(ctx, input,
- ark.WithThinking(thinking),
- ark.WithCache(cacheOpt))
- if err != nil {
- log.Fatalf("Stream failed, err=%v", err)
- }
+##### Streaming Conversation
- var messages []*schema.Message
- for {
- chunk, err := streamResp.Recv()
- if err == io.EOF {
- break
- }
- if err != nil {
- log.Fatalf("Recv of streamResp failed, err=%v", err)
- }
- messages = append(messages, chunk)
- }
+```go
+import (
+ "context"
+ "encoding/json"
+ "errors"
+ "io"
+ "log"
+ "os"
- resp, err := schema.ConcatMessages(messages)
- if err != nil {
- log.Fatalf("ConcatMessages of ark failed, err=%v", err)
- }
+ "github.com/volcengine/volcengine-go-sdk/service/arkruntime/model"
- fmt.Printf("stream output: \n%v\n\n", resp)
+ "github.com/cloudwego/eino-ext/components/model/ark"
+ "github.com/cloudwego/eino/schema"
+)
- input = append(input, resp)
- }
+// Pointer helper function
+func ptr[T any](v T) *T {
+ return &v
}
+func main() {
+ // Initialize model
+ ctx := context.Background()
+ imageGenerationModel, err := ark.NewImageGenerationModel(ctx, &ark.ImageGenerationConfig{
+ APIKey: os.Getenv("ARK_API_KEY"),
+ Model: os.Getenv("ARK_MODEL_ID"),
+ Size: "1K",
+ SequentialImageGeneration: ark.SequentialImageGenerationAuto,
+ SequentialImageGenerationOption: &model.SequentialImageGenerationOptions{
+ MaxImages: ptr(2),
+ },
+ ResponseFormat: ark.ImageResponseFormatURL,
+ DisableWatermark: false,
+ })
+
+ if err != nil {
+ log.Fatalf("NewChatModel failed, err=%v", err)
+ }
+
+ // Prepare messages
+ inMsgs := []*schema.Message{
+ {
+ Role: schema.User,
+ Content: "generate two images of a cat",
+ },
+ }
+
+ // Stream generate images
+ sr, err := imageGenerationModel.Stream(ctx, inMsgs)
+ if err != nil {
+ log.Fatalf("Stream failed, err=%v", err)
+ }
+
+ // Process streaming information
+ log.Printf("stream output: \n")
+ index := 0
+ for {
+ msgChunk, err := sr.Recv()
+ if errors.Is(err, io.EOF) {
+ break
+ }
+ if err != nil {
+ log.Fatalf("Stream Recv failed, err=%v", err)
+ }
+
+ respBody, _ = json.MarshalIndent(msgChunk, " ", " ")
+ log.Printf("stream chunk %d: body: %s\n", index, string(respBody))
+ index++
+ }
+}
```
### [More Examples](https://github.com/cloudwego/eino-ext/tree/main/components/model/ark/examples)
## **Related Documentation**
-- `Eino: ChatModel Guide` at `/docs/eino/core_modules/components/chat_model_guide`
-- `ChatModel - OpenAI` at `/docs/eino/ecosystem_integration/chat_model/chat_model_openai`
-- `ChatModel - Ollama` at `/docs/eino/ecosystem_integration/chat_model/chat_model_ollama`
+- [Eino: ChatModel Guide](/docs/eino/core_modules/components/chat_model_guide)
+- [ChatModel - OpenAI](/docs/eino/ecosystem_integration/chat_model/chat_model_openai)
+- [ChatModel - Ollama](/docs/eino/ecosystem_integration/chat_model/chat_model_ollama)
- [Volcengine Official Site](https://www.volcengine.com/product/doubao)
diff --git a/content/en/docs/eino/ecosystem_integration/chat_model/chat_model_deepseek.md b/content/en/docs/eino/ecosystem_integration/chat_model/chat_model_deepseek.md
index 6ee312e5ac4..4a5c209ca41 100644
--- a/content/en/docs/eino/ecosystem_integration/chat_model/chat_model_deepseek.md
+++ b/content/en/docs/eino/ecosystem_integration/chat_model/chat_model_deepseek.md
@@ -1,9 +1,9 @@
---
Description: ""
-date: "2025-12-11"
+date: "2026-01-20"
lastmod: ""
tags: []
-title: ChatModel - deepseek
+title: ChatModel - DeepSeek
weight: 0
---
@@ -33,62 +33,63 @@ Here's a quick example of how to use the DeepSeek model:
package main
import (
- "context"
- "fmt"
- "log"
- "os"
- "time"
-
- "github.com/cloudwego/eino-ext/components/model/deepseek"
- "github.com/cloudwego/eino/schema"
+ "context"
+ "fmt"
+ "log"
+ "os"
+ "time"
+
+ "github.com/cloudwego/eino-ext/components/model/deepseek"
+ "github.com/cloudwego/eino/schema"
)
func main() {
- ctx := context.Background()
- apiKey := os.Getenv("DEEPSEEK_API_KEY")
- if apiKey == "" {
- log.Fatal("DEEPSEEK_API_KEY environment variable is not set")
- }
- cm, err := deepseek.NewChatModel(ctx, &deepseek.ChatModelConfig{
- APIKey: apiKey,
- Model: os.Getenv("MODEL_NAME"),
- BaseURL: "https://api.deepseek.com/beta",
-
- })
- if err != nil {
- log.Fatal(err)
- }
-
- messages := []*schema.Message{
- {
- Role: schema.System,
- Content: "You are a helpful AI assistant. Be concise in your responses.",
- },
- {
- Role: schema.User,
- Content: "What is the capital of France?",
- },
- }
-
- resp, err := cm.Generate(ctx, messages)
- if err != nil {
- log.Printf("Generate error: %v", err)
- return
- }
-
- reasoning, ok := deepseek.GetReasoningContent(resp)
- if !ok {
- fmt.Printf("Unexpected: non-reasoning")
- } else {
- fmt.Printf("Reasoning Content: %s\n", reasoning)
- }
- fmt.Printf("Assistant: %s\n", resp.Content)
- if resp.ResponseMeta != nil && resp.ResponseMeta.Usage != nil {
- fmt.Printf("Tokens used: %d (prompt) + %d (completion) = %d (total) \n",
- resp.ResponseMeta.Usage.PromptTokens,
- resp.ResponseMeta.Usage.CompletionTokens,
- resp.ResponseMetaUsage.TotalTokens)
- }
+ ctx := context.Background()
+ apiKey := os.Getenv("DEEPSEEK_API_KEY")
+ if apiKey == "" {
+ log.Fatal("DEEPSEEK_API_KEY environment variable is not set")
+ }
+ cm, err := deepseek.NewChatModel(ctx, &deepseek.ChatModelConfig{
+ APIKey: apiKey,
+ Model: os.Getenv("MODEL_NAME"),
+ BaseURL: "https://api.deepseek.com/beta",
+
+
+ })
+ if err != nil {
+ log.Fatal(err)
+ }
+
+ messages := []*schema.Message{
+ {
+ Role: schema.System,
+ Content: "You are a helpful AI assistant. Be concise in your responses.",
+ },
+ {
+ Role: schema.User,
+ Content: "What is the capital of France?",
+ },
+ }
+
+ resp, err := cm.Generate(ctx, messages)
+ if err != nil {
+ log.Printf("Generate error: %v", err)
+ return
+ }
+
+ reasoning, ok := deepseek.GetReasoningContent(resp)
+ if !ok {
+ fmt.Printf("Unexpected: non-reasoning")
+ } else {
+ fmt.Printf("Reasoning Content: %s \n", reasoning)
+ }
+ fmt.Printf("Assistant: %s\n", resp.Content)
+ if resp.ResponseMeta != nil && resp.ResponseMeta.Usage != nil {
+ fmt.Printf("Tokens used: %d (prompt) + %d (completion) = %d (total) \n",
+ resp.ResponseMeta.Usage.PromptTokens,
+ resp.ResponseMeta.Usage.CompletionTokens,
+ resp.ResponseMetaUsage.TotalTokens)
+ }
}
```
@@ -98,79 +99,75 @@ func main() {
You can configure the model using the `deepseek.ChatModelConfig` struct:
```go
-
type ChatModelConfig struct {
// APIKey is your authentication key
// Required
APIKey string `json:"api_key"`
-
+
// Timeout specifies the maximum duration to wait for API responses
// Optional. Default: 5 minutes
Timeout time.Duration `json:"timeout"`
-
+
// HTTPClient specifies the client to send HTTP requests.
// Optional. Default http.DefaultClient
HTTPClient *http.Client `json:"http_client"`
-
+
// BaseURL is your custom deepseek endpoint url
// Optional. Default: https://api.deepseek.com/
BaseURL string `json:"base_url"`
-
+
// Path sets the path for the API request. Defaults to "chat/completions", if not set.
// Example usages would be "/c/chat/" or any http after the baseURL extension
- // Path 用于设置 API 请求的路径。如果未设置，则默认为 "chat/completions"。
- // 用法示例可以是 "/c/chat/" 或 baseURL 之后的任何 http 路径。
Path string `json:"path"`
-
+
// The following fields correspond to DeepSeek's chat API parameters
// Ref: https://api-docs.deepseek.com/api/create-chat-completion
-
+
// Model specifies the ID of the model to use
// Required
Model string `json:"model"`
-
+
// MaxTokens limits the maximum number of tokens that can be generated in the chat completion
// Range: [1, 8192].
// Optional. Default: 4096
MaxTokens int `json:"max_tokens,omitempty"`
-
+
// Temperature specifies what sampling temperature to use
// Generally recommend altering this or TopP but not both.
// Range: [0.0, 2.0]. Higher values make output more random
// Optional. Default: 1.0
Temperature float32 `json:"temperature,omitempty"`
-
+
// TopP controls diversity via nucleus sampling
// Generally recommend altering this or Temperature but not both.
// Range: [0.0, 1.0]. Lower values make output more focused
// Optional. Default: 1.0
TopP float32 `json:"top_p,omitempty"`
-
+
// Stop sequences where the API will stop generating further tokens
// Optional. Example: []string{"\n", "User:"}
Stop []string `json:"stop,omitempty"`
-
+
// PresencePenalty prevents repetition by penalizing tokens based on presence
// Range: [-2.0, 2.0]. Positive values increase likelihood of new topics
// Optional. Default: 0
PresencePenalty float32 `json:"presence_penalty,omitempty"`
-
+
// ResponseFormat specifies the format of the model's response
// Optional. Use for structured outputs
ResponseFormatType ResponseFormatType `json:"response_format_type,omitempty"`
-
+
// FrequencyPenalty prevents repetition by penalizing tokens based on frequency
// Range: [-2.0, 2.0]. Positive values decrease likelihood of repetition
// Optional. Default: 0
FrequencyPenalty float32 `json:"frequency_penalty,omitempty"`
-
+
// LogProbs specifies whether to return log probabilities of the output tokens.
LogProbs bool `json:"log_probs"`
-
+
// TopLogProbs specifies the number of most likely tokens to return at each token position, each with an associated log probability.
TopLogProbs int `json:"top_log_probs"`
}
-
```
## **Examples**
@@ -178,301 +175,291 @@ type ChatModelConfig struct {
### **Text Generation**
```go
-
package main
import (
- "context"
- "fmt"
- "log"
- "os"
- "time"
-
- "github.com/cloudwego/eino-ext/components/model/deepseek"
- "github.com/cloudwego/eino/schema"
+ "context"
+ "fmt"
+ "log"
+ "os"
+ "time"
+
+ "github.com/cloudwego/eino-ext/components/model/deepseek"
+ "github.com/cloudwego/eino/schema"
)
func main() {
- ctx := context.Background()
- apiKey := os.Getenv("DEEPSEEK_API_KEY")
- if apiKey == "" {
- log.Fatal("DEEPSEEK_API_KEY environment variable is not set")
- }
-
- cm, err := deepseek.NewChatModel(ctx, &deepseek.ChatModelConfig{
- APIKey: apiKey,
- Model: os.Getenv("MODEL_NAME"),
- BaseURL: "https://api.deepseek.com/beta",
- Timeout: 30 * time.Second,
- })
- if err != nil {
- log.Fatal(err)
- }
-
- messages := []*schema.Message{
- {
- Role: schema.System,
- Content: "You are a helpful AI assistant. Be concise in your responses.",
- },
- {
- Role: schema.User,
- Content: "What is the capital of France?",
- },
- }
-
- resp, err := cm.Generate(ctx, messages)
- if err != nil {
- log.Printf("Generate error: %v", err)
- return
- }
-
- reasoning, ok := deepseek.GetReasoningContent(resp)
- if !ok {
- fmt.Printf("Unexpected: non-reasoning")
- } else {
- fmt.Printf("Reasoning Content: %s\n", reasoning)
- }
- fmt.Printf("Assistant: %s\n", resp.Content)
- if resp.ResponseMeta != nil && resp.ResponseMeta.Usage != nil {
- fmt.Printf("Tokens used: %d (prompt) + %d (completion) = %d (total)\n",
- resp.ResponseMeta.Usage.PromptTokens,
- resp.ResponseMeta.Usage.CompletionTokens,
- resp.ResponseMeta.Usage.TotalTokens)
- }
+ ctx := context.Background()
+ apiKey := os.Getenv("DEEPSEEK_API_KEY")
+ if apiKey == "" {
+ log.Fatal("DEEPSEEK_API_KEY environment variable is not set")
+ }
-}
+ cm, err := deepseek.NewChatModel(ctx, &deepseek.ChatModelConfig{
+ APIKey: apiKey,
+ Model: os.Getenv("MODEL_NAME"),
+ BaseURL: "https://api.deepseek.com/beta",
+ Timeout: 30 * time.Second,
+ })
+ if err != nil {
+ log.Fatal(err)
+ }
+
+ messages := []*schema.Message{
+ {
+ Role: schema.System,
+ Content: "You are a helpful AI assistant. Be concise in your responses.",
+ },
+ {
+ Role: schema.User,
+ Content: "What is the capital of France?",
+ },
+ }
+
+ resp, err := cm.Generate(ctx, messages)
+ if err != nil {
+ log.Printf("Generate error: %v", err)
+ return
+ }
+
+ reasoning, ok := deepseek.GetReasoningContent(resp)
+ if !ok {
+ fmt.Printf("Unexpected: non-reasoning")
+ } else {
+ fmt.Printf("Reasoning Content: %s\n", reasoning)
+ }
+ fmt.Printf("Assistant: %s \n", resp.Content)
+ if resp.ResponseMeta != nil && resp.ResponseMeta.Usage != nil {
+ fmt.Printf("Tokens used: %d (prompt) + %d (completion) = %d (total) \n",
+ resp.ResponseMeta.Usage.PromptTokens,
+ resp.ResponseMeta.Usage.CompletionTokens,
+ resp.ResponseMeta.Usage.TotalTokens)
+ }
+}
```
### **Text Generation with Prefix**
```go
-
package main
import (
- "context"
- "fmt"
- "log"
- "os"
- "time"
-
- "github.com/cloudwego/eino-ext/components/model/deepseek"
- "github.com/cloudwego/eino/schema"
+ "context"
+ "fmt"
+ "log"
+ "os"
+ "time"
+
+ "github.com/cloudwego/eino-ext/components/model/deepseek"
+ "github.com/cloudwego/eino/schema"
)
func main() {
- ctx := context.Background()
- apiKey := os.Getenv("DEEPSEEK_API_KEY")
- if apiKey == "" {
- log.Fatal("DEEPSEEK_API_KEY environment variable is not set")
- }
- cm, err := deepseek.NewChatModel(ctx, &deepseek.ChatModelConfig{
- APIKey: apiKey,
- Model: os.Getenv("MODEL_NAME"),
- BaseURL: "https://api.deepseek.com/beta",
- Timeout: 30 * time.Second,
- })
- if err != nil {
- log.Fatal(err)
- }
-
- messages := []*schema.Message{
- schema.UserMessage("Please write quick sort code"),
- schema.AssistantMessage("```python\n", nil),
- }
- deepseek.SetPrefix(messages[1])
-
- result, err := cm.Generate(ctx, messages)
- if err != nil {
- log.Printf("Generate error: %v", err)
- }
-
- reasoningContent, ok := deepseek.GetReasoningContent(result)
- if !ok {
- fmt.Printf("No reasoning content")
- } else {
- fmt.Printf("Reasoning: %v\n", reasoningContent)
- }
- fmt.Printf("Content: %v\n", result)
+ ctx := context.Background()
+ apiKey := os.Getenv("DEEPSEEK_API_KEY")
+ if apiKey == "" {
+ log.Fatal("DEEPSEEK_API_KEY environment variable is not set")
+ }
+ cm, err := deepseek.NewChatModel(ctx, &deepseek.ChatModelConfig{
+ APIKey: apiKey,
+ Model: os.Getenv("MODEL_NAME"),
+ BaseURL: "https://api.deepseek.com/beta",
+ Timeout: 30 * time.Second,
+ })
+ if err != nil {
+ log.Fatal(err)
+ }
-}
+ messages := []*schema.Message{
+ schema.UserMessage("Please write quick sort code"),
+ schema.AssistantMessage("```python \n", nil),
+ }
+ deepseek.SetPrefix(messages[1])
+
+ result, err := cm.Generate(ctx, messages)
+ if err != nil {
+ log.Printf("Generate error: %v", err)
+ }
+ reasoningContent, ok := deepseek.GetReasoningContent(result)
+ if !ok {
+ fmt.Printf("No reasoning content")
+ } else {
+ fmt.Printf("Reasoning: %v \n", reasoningContent)
+ }
+ fmt.Printf("Content: %v\n", result)
+
+}
```
### **Streaming Generation**
```go
-
package main
import (
- "context"
- "fmt"
- "io"
- "log"
- "os"
- "time"
-
- "github.com/cloudwego/eino-ext/components/model/deepseek"
- "github.com/cloudwego/eino/schema"
+ "context"
+ "fmt"
+ "io"
+ "log"
+ "os"
+ "time"
+
+ "github.com/cloudwego/eino-ext/components/model/deepseek"
+ "github.com/cloudwego/eino/schema"
)
func main() {
- ctx := context.Background()
- apiKey := os.Getenv("DEEPSEEK_API_KEY")
- if apiKey == "" {
- log.Fatal("DEEPSEEK_API_KEY environment variable is not set")
- }
- cm, err := deepseek.NewChatModel(ctx, &deepseek.ChatModelConfig{
- APIKey: apiKey,
- Model: os.Getenv("MODEL_NAME"),
- BaseURL: "https://api.deepseek.com/beta",
- Timeout: 30 * time.Second,
- })
- if err != nil {
- log.Fatal(err)
- }
-
- messages := []*schema.Message{
- {
- Role: schema.User,
- Content: "Write a short poem about spring, word by word.",
- },
- }
-
- stream, err := cm.Stream(ctx, messages)
- if err != nil {
- log.Printf("Stream error: %v", err)
- return
- }
-
- fmt.Print("Assistant: ")
- for {
- resp, err := stream.Recv()
- if err == io.EOF {
- break
+ ctx := context.Background()
+ apiKey := os.Getenv("DEEPSEEK_API_KEY")
+ if apiKey == "" {
+ log.Fatal("DEEPSEEK_API_KEY environment variable is not set")
}
+ cm, err := deepseek.NewChatModel(ctx, &deepseek.ChatModelConfig{
+ APIKey: apiKey,
+ Model: os.Getenv("MODEL_NAME"),
+ BaseURL: "https://api.deepseek.com/beta",
+ Timeout: 30 * time.Second,
+ })
if err != nil {
- log.Printf("Stream receive error: %v", err)
- return
+ log.Fatal(err)
}
- if reasoning, ok := deepseek.GetReasoningContent(resp); ok {
- fmt.Printf("Reasoning Content: %s\n", reasoning)
+
+ messages := []*schema.Message{
+ {
+ Role: schema.User,
+ Content: "Write a short poem about spring, word by word.",
+ },
}
- if len(resp.Content) > 0 {
- fmt.Printf("Content: %s\n", resp.Content)
+
+ stream, err := cm.Stream(ctx, messages)
+ if err != nil {
+ log.Printf("Stream error: %v", err)
+ return
}
- if resp.ResponseMeta != nil && resp.ResponseMeta.Usage != nil {
- fmt.Printf("Tokens used: %d (prompt) + %d (completion) = %d (total)\n",
- resp.ResponseMeta.Usage.PromptTokens,
- resp.ResponseMeta.Usage.CompletionTokens,
- resp.ResponseMeta.Usage.TotalTokens)
+
+ fmt.Print("Assistant: ")
+ for {
+ resp, err := stream.Recv()
+ if err == io.EOF {
+ break
+ }
+ if err != nil {
+ log.Printf("Stream receive error: %v", err)
+ return
+ }
+ if reasoning, ok := deepseek.GetReasoningContent(resp); ok {
+ fmt.Printf("Reasoning Content: %s\n", reasoning)
+ }
+ if len(resp.Content) > 0 {
+ fmt.Printf("Content: %s\n", resp.Content)
+ }
+ if resp.ResponseMeta != nil && resp.ResponseMeta.Usage != nil {
+ fmt.Printf("Tokens used: %d (prompt) + %d (completion) = %d (total) \n",
+ resp.ResponseMeta.Usage.PromptTokens,
+ resp.ResponseMeta.Usage.CompletionTokens,
+ resp.ResponseMeta.Usage.TotalTokens)
+ }
}
- }
}
-
```
### **Tool Calling**
```go
-
package main
import (
- "context"
- "fmt"
- "io"
- "log"
- "os"
- "time"
-
- "github.com/cloudwego/eino-ext/components/model/deepseek"
- "github.com/cloudwego/eino/schema"
+ "context"
+ "fmt"
+ "io"
+ "log"
+ "os"
+ "time"
+
+ "github.com/cloudwego/eino-ext/components/model/deepseek"
+ "github.com/cloudwego/eino/schema"
)
func main() {
- ctx := context.Background()
- apiKey := os.Getenv("DEEPSEEK_API_KEY")
- if apiKey == "" {
- log.Fatal("DEEPSEEK_API_KEY environment variable is not set")
- }
- cm, err := deepseek.NewChatModel(ctx, &deepseek.ChatModelConfig{
- APIKey: apiKey,
- Model: os.Getenv("MODEL_NAME"),
- BaseURL: "https://api.deepseek.com/beta",
- Timeout: 30 * time.Second,
- })
- if err != nil {
- log.Fatal(err)
- }
-
- _, err = cm.WithTools([]*schema.ToolInfo{
- {
- Name: "user_company",
- Desc: "Retrieve the user's company and position based on their name and email.",
- ParamsOneOf: schema.NewParamsOneOfByParams(
- map[string]*schema.ParameterInfo{
- "name": {Type: "string", Desc: "user's name"},
- "email": {Type: "string", Desc: "user's email"}},
- ),
+ ctx := context.Background()
+ apiKey := os.Getenv("DEEPSEEK_API_KEY")
+ if apiKey == "" {
+ log.Fatal("DEEPSEEK_API_KEY environment variable is not set")
+ }
+ cm, err := deepseek.NewChatModel(ctx, &deepseek.ChatModelConfig{
+ APIKey: apiKey,
+ Model: os.Getenv("MODEL_NAME"),
+ BaseURL: "https://api.deepseek.com/beta",
+ Timeout: 30 * time.Second,
+ })
+ if err != nil {
+ log.Fatal(err)
+ }
+
+ _, err = cm.WithTools([]*schema.ToolInfo{
+ {
+ Name: "user_company",
+ Desc: "Retrieve the user's company and position based on their name and email.",
+ ParamsOneOf: schema.NewParamsOneOfByParams(
+ map[string]*schema.ParameterInfo{
+ "name": {Type: "string", Desc: "user's name"},
+ "email": {Type: "string", Desc: "user's email"}}),
+ }, {
+ Name: "user_salary",
+ Desc: "Retrieve the user's salary based on their name and email.",
+ ParamsOneOf: schema.NewParamsOneOfByParams(
+ map[string]*schema.ParameterInfo{
+ "name": {Type: "string", Desc: "user's name"},
+ "email": {Type: "string", Desc: "user's email"},
+ }),
+ }})
+ if err != nil {
+ log.Fatalf("BindTools of deepseek failed, err=%v", err)
+ }
+ resp, err := cm.Generate(ctx, []*schema.Message{{
+ Role: schema.System,
+ Content: "As a real estate agent, provide relevant property information based on the user's salary and job using the user_company and user_salary APIs. An email address is required.",
}, {
- Name: "user_salary",
- Desc: "Retrieve the user's salary based on their name and email.\n",
- ParamsOneOf: schema.NewParamsOneOfByParams(
- map[string]*schema.ParameterInfo{
- "name": {Type: "string", Desc: "user's name"},
- "email": {Type: "string", Desc: "user's email"},
- },
- ),
+ Role: schema.User,
+ Content: "My name is John and my email is john@abc.com，Please recommend some houses that suit me.",
}})
- if err != nil {
- log.Fatalf("BindTools of deepseek failed, err=%v", err)
- }
- resp, err := cm.Generate(ctx, []*schema.Message{{
- Role: schema.System,
- Content: "As a real estate agent, provide relevant property information based on the user's salary and job using the user_company and user_salary APIs. An email address is required.",
- }, {
- Role: schema.User,
- Content: "My name is John and my email is john@abc.com，Please recommend some houses that suit me.",
- }})
- if err != nil {
- log.Fatalf("Generate of deepseek failed, err=%v", err)
- }
- fmt.Printf("output: \n%v", resp)
-
- streamResp, err := cm.Stream(ctx, []*schema.Message{
- {
- Role: schema.System,
- Content: "As a real estate agent, provide relevant property information based on the user's salary and job using the user_company and user_salary APIs. An email address is required.",
- }, {
- Role: schema.User,
- Content: "My name is John and my email is john@abc.com，Please recommend some houses that suit me.",
- },
- })
- if err != nil {
- log.Fatalf("Stream of deepseek failed, err=%v", err)
- }
- var messages []*schema.Message
- for {
- chunk, err := streamResp.Recv()
- if err == io.EOF {
- break
+ if err != nil {
+ log.Fatalf("Generate of deepseek failed, err=%v", err)
}
+ fmt.Printf("output:%v \n", resp)
+
+ streamResp, err := cm.Stream(ctx, []*schema.Message{
+ {
+ Role: schema.System,
+ Content: "As a real estate agent, provide relevant property information based on the user's salary and job using the user_company and user_salary APIs. An email address is required.",
+ }, {
+ Role: schema.User,
+ Content: "My name is John and my email is john@abc.com，Please recommend some houses that suit me.",
+ },
+ })
if err != nil {
- log.Fatalf("Recv of streamResp failed, err=%v", err)
+ log.Fatalf("Stream of deepseek failed, err=%v", err)
+ }
+ var messages []*schema.Message
+ for {
+ chunk, err := streamResp.Recv()
+ if err == io.EOF {
+ break
+ }
+ if err != nil {
+ log.Fatalf("Recv of streamResp failed, err=%v", err)
+ }
+ messages = append(messages, chunk)
}
- messages = append(messages, chunk)
- }
- resp, err = schema.ConcatMessages(messages)
- if err != nil {
- log.Fatalf("ConcatMessages of deepseek failed, err=%v", err)
- }
- fmt.Printf("stream output: \n%v", resp)
+ resp, err = schema.ConcatMessages(messages)
+ if err != nil {
+ log.Fatalf("ConcatMessages of deepseek failed, err=%v", err)
+ }
+ fmt.Printf("stream output:%v \n", resp)
}
-
```
## **More Information**
diff --git a/content/en/docs/eino/ecosystem_integration/chat_model/chat_model_openai.md b/content/en/docs/eino/ecosystem_integration/chat_model/chat_model_openai.md
index 06b2e13a8cc..aaaf25ca29c 100644
--- a/content/en/docs/eino/ecosystem_integration/chat_model/chat_model_openai.md
+++ b/content/en/docs/eino/ecosystem_integration/chat_model/chat_model_openai.md
@@ -1,12 +1,14 @@
---
Description: ""
-date: "2025-12-11"
+date: "2026-01-20"
lastmod: ""
tags: []
-title: ChatModel - openai
+title: ChatModel - OpenAI
weight: 0
---
+## **OpenAI Model**
+
An OpenAI model implementation for [Eino](https://github.com/cloudwego/eino) that implements the `ToolCallingChatModel` interface. This enables seamless integration with Eino's LLM capabilities for enhanced natural language processing and generation.
## **Features**
@@ -19,13 +21,13 @@ An OpenAI model implementation for [Eino](https://github.com/cloudwego/eino) tha
- Custom response parsing support
- Flexible model configuration
-## **Installation**
+## Installation
-```bash
+```go
go get github.com/cloudwego/eino-ext/components/model/openai@latest
```
-## **Quick Start**
+## Quick Start
Here's a quick example of how to use the OpenAI model:
@@ -33,49 +35,57 @@ Here's a quick example of how to use the OpenAI model:
package main
import (
- "context"
- "fmt"
- "log"
- "os"
+ "context"
+ "fmt"
+ "log"
+ "os"
- "github.com/cloudwego/eino-ext/components/model/openai"
- "github.com/cloudwego/eino/schema"
+ "github.com/cloudwego/eino-ext/components/model/openai"
+ "github.com/cloudwego/eino/schema"
)
func main() {
- ctx := context.Background()
+ ctx := context.Background()
+
+ chatModel, err := openai.NewChatModel(ctx, &openai.ChatModelConfig{
+ // If you want to use Azure OpenAI Service, set these two fields.
+ // BaseURL: "https://{RESOURCE_NAME}.openai.azure.com",
+ // ByAzure: true,
+ // APIVersion: "2024-06-01",
+ APIKey: os.Getenv("OPENAI_API_KEY"),
+ Model: os.Getenv("OPENAI_MODEL"),
+ BaseURL: os.Getenv("OPENAI_BASE_URL"),
+ ByAzure: func() bool {
+ if os.Getenv("OPENAI_BY_AZURE") == "true" {
+ return true
+ }
+ return false
+ }(),
+ ReasoningEffort: openai.ReasoningEffortLevelHigh,
+ })
+ if err != nil {
+ log.Fatalf("NewChatModel failed, err=%v", err)
+ }
+
+ resp, err := chatModel.Generate(ctx, []*schema.Message{
+ {
+ Role: schema.User,
+ Content: "as a machine, how do you answer user's question?",
+ },
+ })
+ if err != nil {
+ log.Fatalf("Generate failed, err=%v", err)
+ }
+ fmt.Printf("output: \n%v", resp)
- chatModel, err := openai.NewChatModel(ctx, &openai.ChatModelConfig{
- // if you want to use Azure OpenAI Service, set these two field.
- // BaseURL: "https://{RESOURCE_NAME}.openai.azure.com",
- // ByAzure: true,
- // APIVersion: "2024-06-01",
- APIKey: os.Getenv("OPENAI_API_KEY"),
- Model: os.Getenv("OPENAI_MODEL"),
- BaseURL: os.Getenv("OPENAI_BASE_URL"),
- ByAzure: func() bool {
- if os.Getenv("OPENAI_BY_AZURE") == "true" { return true }
- return false
- }(),
- ReasoningEffort: openai.ReasoningEffortLevelHigh,
- })
- if err != nil { log.Fatalf("NewChatModel failed, err=%v", err) }
-
- resp, err := chatModel.Generate(ctx, []*schema.Message{{
- Role: schema.User,
- Content: "as a machine, how do you answer user's question?",
- }})
- if err != nil { log.Fatalf("Generate failed, err=%v", err) }
- fmt.Printf("output: \n%v", resp)
}
```
-## **Configuration**
+## Configuration
You can configure the model using the `openai.ChatModelConfig` struct:
```go
-
type ChatModelConfig struct {
// APIKey is your authentication key
// Use OpenAI API key or Azure API key depending on the service
@@ -188,272 +198,735 @@ Audio *Audio `json:"audio,omitempty"`
}
```
-## **Examples**
+## Examples
-### **Text Generation**
+### Text Generation
```go
package main
import (
- "context"
- "fmt"
- "log"
- "os"
+ "context"
+ "fmt"
+ "log"
+ "os"
- "github.com/cloudwego/eino/schema"
+ "github.com/cloudwego/eino/schema"
- "github.com/cloudwego/eino-ext/components/model/openai"
+ "github.com/cloudwego/eino-ext/components/model/openai"
)
func main() {
- ctx := context.Background()
+ ctx := context.Background()
+
+ chatModel, err := openai.NewChatModel(ctx, &openai.ChatModelConfig{
+ // If you want to use Azure OpenAI Service, set these two fields.
+ // BaseURL: "https://{RESOURCE_NAME}.openai.azure.com",
+ // ByAzure: true,
+ // APIVersion: "2024-06-01",
+ APIKey: os.Getenv("OPENAI_API_KEY"),
+ Model: os.Getenv("OPENAI_MODEL"),
+ BaseURL: os.Getenv("OPENAI_BASE_URL"),
+ ByAzure: func() bool {
+ if os.Getenv("OPENAI_BY_AZURE") == "true" {
+ return true
+ }
+ return false
+ }(),
+ ReasoningEffort: openai.ReasoningEffortLevelHigh,
+ })
+ if err != nil {
+ log.Fatalf("NewChatModel failed, err=%v", err)
+ }
+
+ resp, err := chatModel.Generate(ctx, []*schema.Message{
+ {
+ Role: schema.User,
+ Content: "as a machine, how do you answer user's question?",
+ },
+ })
+ if err != nil {
+ log.Fatalf("Generate failed, err=%v", err)
+ }
+ fmt.Printf("output: \n%v", resp)
- chatModel, err := openai.NewChatModel(ctx, &openai.ChatModelConfig{
- // if you want to use Azure OpenAI Service, set these fields
- // BaseURL: "https://{RESOURCE_NAME}.openai.azure.com",
- // ByAzure: true,
- // APIVersion: "2024-06-01",
- APIKey: os.Getenv("OPENAI_API_KEY"),
- Model: os.Getenv("OPENAI_MODEL"),
- BaseURL: os.Getenv("OPENAI_BASE_URL"),
- ByAzure: func() bool {
- if os.Getenv("OPENAI_BY_AZURE") == "true" {
- return true
- }
- return false
- }(),
- ReasoningEffort: openai.ReasoningEffortLevelHigh,
- })
- if err != nil {
- log.Fatalf("NewChatModel failed, err=%v", err)
- }
-
- resp, err := chatModel.Generate(ctx, []*schema.Message{
- {
- Role: schema.User,
- Content: "as a machine, how do you answer user's question?",
- },
- })
- if err != nil {
- log.Fatalf("Generate failed, err=%v", err)
- }
- fmt.Printf("output: \n%v", resp)
}
```
-### **Multimodal Support (Image Understanding)**
+### Multimodal Support (Image Understanding)
```go
package main
import (
- "context"
- "fmt"
- "log"
- "os"
+ "context"
+ "fmt"
+ "log"
+ "os"
- "github.com/cloudwego/eino/schema"
- "github.com/cloudwego/eino-ext/components/model/openai"
+ "github.com/cloudwego/eino/schema"
+
+ "github.com/cloudwego/eino-ext/components/model/openai"
)
func main() {
- ctx := context.Background()
- chatModel, err := openai.NewChatModel(ctx, &openai.ChatModelConfig{
- APIKey: os.Getenv("OPENAI_API_KEY"),
- Model: os.Getenv("OPENAI_MODEL"),
- BaseURL: os.Getenv("OPENAI_BASE_URL"),
- ByAzure: func() bool { return os.Getenv("OPENAI_BY_AZURE") == "true" }(),
- })
- if err != nil { log.Fatalf("NewChatModel failed, err=%v", err) }
-
- multi := &schema.Message{
- Role: schema.User,
- UserInputMultiContent: []schema.MessageInputPart{
- {Type: schema.ChatMessagePartTypeText, Text: "this picture is a landscape photo, what's the picture's content"},
- {Type: schema.ChatMessagePartTypeImageURL, Image: &schema.MessageInputImage{MessagePartCommon: schema.MessagePartCommon{URL: of("https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcT11qEDxU4X_MVKYQVU5qiAVFidA58f8GG0bQ&s")}, Detail: schema.ImageURLDetailAuto}},
- },
- }
-
- resp, err := chatModel.Generate(ctx, []*schema.Message{ multi })
- if err != nil { log.Fatalf("Generate failed, err=%v", err) }
- fmt.Printf("output: \n%v", resp)
+ ctx := context.Background()
+
+ chatModel, err := openai.NewChatModel(ctx, &openai.ChatModelConfig{
+ APIKey: os.Getenv("OPENAI_API_KEY"),
+ Model: os.Getenv("OPENAI_MODEL"),
+ BaseURL: os.Getenv("OPENAI_BASE_URL"),
+ ByAzure: func() bool {
+ if os.Getenv("OPENAI_BY_AZURE") == "true" {
+ return true
+ }
+ return false
+ }(),
+ })
+ if err != nil {
+ log.Fatalf("NewChatModel failed, err=%v", err)
+
+ }
+
+ multiModalMsg := &schema.Message{
+ UserInputMultiContent: []schema.MessageInputPart{
+ {
+ Type: schema.ChatMessagePartTypeText,
+ Text: "this picture is a landscape photo, what's the picture's content",
+ },
+ {
+ Type: schema.ChatMessagePartTypeImageURL,
+ Image: &schema.MessageInputImage{
+ MessagePartCommon: schema.MessagePartCommon{
+ URL: of("https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcT11qEDxU4X_MVKYQVU5qiAVFidA58f8GG0bQ&s"),
+ },
+ Detail: schema.ImageURLDetailAuto,
+ },
+ },
+ },
+ }
+
+ resp, err := chatModel.Generate(ctx, []*schema.Message{
+ multiModalMsg,
+ })
+ if err != nil {
+ log.Fatalf("Generate failed, err=%v", err)
+ }
+
+ fmt.Printf("output: \n%v", resp)
}
-func of[T any](a T) *T { return &a }
+func of[T any](a T) *T {
+ return &a
+}
```
-### **Streaming Generation**
+### Streaming Generation
```go
package main
import (
- "context"
- "fmt"
- "io"
- "log"
- "os"
+ "context"
+ "fmt"
+ "io"
+ "log"
+ "os"
- "github.com/cloudwego/eino-ext/components/model/openai"
- "github.com/cloudwego/eino/schema"
+ "github.com/cloudwego/eino/schema"
+
+ "github.com/cloudwego/eino-ext/components/model/openai"
)
func main() {
- ctx := context.Background()
- chatModel, err := openai.NewChatModel(ctx, &openai.ChatModelConfig{
- APIKey: os.Getenv("OPENAI_API_KEY"),
- Model: os.Getenv("OPENAI_MODEL"),
- BaseURL: os.Getenv("OPENAI_BASE_URL"),
- ByAzure: func() bool { return os.Getenv("OPENAI_BY_AZURE") == "true" }(),
- })
- if err != nil { log.Fatalf("NewChatModel of openai failed, err=%v", err) }
+ ctx := context.Background()
+ chatModel, err := openai.NewChatModel(ctx, &openai.ChatModelConfig{
+ APIKey: os.Getenv("OPENAI_API_KEY"),
+ Model: os.Getenv("OPENAI_MODEL"),
+ BaseURL: os.Getenv("OPENAI_BASE_URL"),
+ ByAzure: func() bool {
+ if os.Getenv("OPENAI_BY_AZURE") == "true" {
+ return true
+ }
+ return false
+ }(),
+ })
+ if err != nil {
+ log.Fatalf("NewChatModel of openai failed, err=%v", err)
+ }
+
+ streamMsgs, err := chatModel.Stream(ctx, []*schema.Message{
+ {
+ Role: schema.User,
+ Content: "as a machine, how do you answer user's question?",
+ },
+ })
+
+ if err != nil {
+ log.Fatalf("Stream of openai failed, err=%v", err)
+ }
+
+ defer streamMsgs.Close()
+
+ fmt.Printf("typewriter output:")
+ for {
+ msg, err := streamMsgs.Recv()
+ if err == io.EOF {
+ break
+ }
+ if err != nil {
+ log.Fatalf("Recv of streamMsgs failed, err=%v", err)
+ }
+ fmt.Print(msg.Content)
+ }
+
+ fmt.Print("\n")
+}
+```
- sr, err := chatModel.Stream(ctx, []*schema.Message{{ Role: schema.User, Content: "as a machine, how do you answer user's question?" }})
- if err != nil { log.Fatalf("Stream of openai failed, err=%v", err) }
- defer sr.Close()
+### Tool Calling
- fmt.Printf("typewriter output:")
- for {
- msg, err := sr.Recv()
- if err == io.EOF { break }
- if err != nil { log.Fatalf("Recv failed, err=%v", err) }
- fmt.Print(msg.Content)
- }
- fmt.Print("\n")
+```go
+package main
+
+import (
+ "context"
+ "fmt"
+ "io"
+ "log"
+ "os"
+
+ "github.com/cloudwego/eino/schema"
+
+ "github.com/cloudwego/eino-ext/components/model/openai"
+)
+
+func main() {
+ ctx := context.Background()
+ chatModel, err := openai.NewChatModel(ctx, &openai.ChatModelConfig{
+ APIKey: os.Getenv("OPENAI_API_KEY"),
+ Model: os.Getenv("OPENAI_MODEL"),
+ BaseURL: os.Getenv("OPENAI_BASE_URL"),
+ ByAzure: func() bool {
+ if os.Getenv("OPENAI_BY_AZURE") == "true" {
+ return true
+ }
+ return false
+ }(),
+ })
+ if err != nil {
+ log.Fatalf("NewChatModel of openai failed, err=%v", err)
+ }
+ err = chatModel.BindForcedTools([]*schema.ToolInfo{
+ {
+ Name: "user_company",
+ Desc: "Retrieve the user's company and position based on their name and email.",
+ ParamsOneOf: schema.NewParamsOneOfByParams(
+ map[string]*schema.ParameterInfo{
+ "name": {Type: "string", Desc: "user's name"},
+ "email": {Type: "string", Desc: "user's email"}}),
+ }, {
+ Name: "user_salary",
+ Desc: "Retrieve the user's salary based on their name and email.\n",
+ ParamsOneOf: schema.NewParamsOneOfByParams(
+ map[string]*schema.ParameterInfo{
+ "name": {Type: "string", Desc: "user's name"},
+ "email": {Type: "string", Desc: "user's email"},
+ }),
+ }})
+ if err != nil {
+ log.Fatalf("BindForcedTools of openai failed, err=%v", err)
+ }
+ resp, err := chatModel.Generate(ctx, []*schema.Message{{
+ Role: schema.System,
+ Content: "As a real estate agent, provide relevant property information based on the user's salary and job using the user_company and user_salary APIs. An email address is required.",
+ }, {
+ Role: schema.User,
+ Content: "My name is John and my email is john@abc.com，Please recommend some houses that suit me.",
+ }})
+ if err != nil {
+ log.Fatalf("Generate of openai failed, err=%v", err)
+ }
+ fmt.Printf("output: \n%v", resp)
+
+ streamResp, err := chatModel.Stream(ctx, []*schema.Message{
+ {
+ Role: schema.System,
+ Content: "As a real estate agent, provide relevant property information based on the user's salary and job using the user_company and user_salary APIs. An email address is required.",
+ }, {
+ Role: schema.User,
+ Content: "My name is John and my email is john@abc.com，Please recommend some houses that suit me.",
+ },
+ })
+ if err != nil {
+ log.Fatalf("Stream of openai failed, err=%v", err)
+ }
+ var messages []*schema.Message
+ for {
+ chunk, err := streamResp.Recv()
+ if err == io.EOF {
+ break
+ }
+ if err != nil {
+ log.Fatalf("Recv of streamResp failed, err=%v", err)
+ }
+ messages = append(messages, chunk)
+ }
+ resp, err = schema.ConcatMessages(messages)
+ if err != nil {
+ log.Fatalf("ConcatMessages of openai failed, err=%v", err)
+ }
+ fmt.Printf("stream output: \n%v", resp)
}
```
-### **Audio Generation**
+### Audio Generation
```go
package main
import (
- "context"
- "log"
+ "context"
+
+ "log"
+ "os"
+
+ "github.com/bytedance/sonic"
+ "github.com/cloudwego/eino-ext/components/model/openai"
+ "github.com/cloudwego/eino/schema"
+)
+
+func main() {
+ ctx := context.Background()
+
+ chatModel, err := openai.NewChatModel(ctx, &openai.ChatModelConfig{
+ // If you want to use Azure OpenAI Service, set these two fields.
+ // BaseURL: "https://{RESOURCE_NAME}.openai.azure.com",
+ // ByAzure: true,
+ // APIVersion: "2024-06-01",
+ APIKey: os.Getenv("OPENAI_API_KEY"),
+ Model: os.Getenv("OPENAI_MODEL"),
+ BaseURL: os.Getenv("OPENAI_BASE_URL"),
+ ByAzure: func() bool {
+ if os.Getenv("OPENAI_BY_AZURE") == "true" {
+ return true
+ }
+ return false
+ }(),
+ ReasoningEffort: openai.ReasoningEffortLevelHigh,
+ Modalities: []openai.Modality{openai.AudioModality, openai.TextModality},
+ Audio: &openai.Audio{
+ Format: openai.AudioFormatMp3,
+ Voice: openai.AudioVoiceAlloy,
+ },
+ })
+ if err != nil {
+ log.Fatalf("NewChatModel failed, err=%v", err)
+ }
+
+ resp, err := chatModel.Generate(ctx, []*schema.Message{
+ {
+ Role: schema.User,
+ UserInputMultiContent: []schema.MessageInputPart{
+ {Type: schema.ChatMessagePartTypeText, Text: "help me convert the following text to speech"},
+ {Type: schema.ChatMessagePartTypeText, Text: "Hello, what can I help you with?"},
+ },
+ },
+ })
+ if err != nil {
+ log.Fatalf("Generate failed, err=%v", err)
+ }
+
+ respBody, _ := sonic.MarshalIndent(resp, " ", " ")
+ log.Printf(" body: %s\n", string(respBody))
+
+}
+```
+
+### Structured Output
+
+```go
+package main
+
+import (
+ "context"
+ "encoding/json"
+ "fmt"
+ "log"
+ "os"
+
+ "github.com/eino-contrib/jsonschema"
+ orderedmap "github.com/wk8/go-ordered-map/v2"
+
+ "github.com/cloudwego/eino/schema"
+
+ "github.com/cloudwego/eino-ext/components/model/openai"
+)
+
+func main() {
+ type Person struct {
+ Name string `json:"name"`
+ Height int `json:"height"`
+ Weight int `json:"weight"`
+ }
+
+ js := &jsonschema.Schema{
+ Type: string(schema.Object),
+ Properties: orderedmap.New[string, *jsonschema.Schema](
+ orderedmap.WithInitialData[string, *jsonschema.Schema](
+ orderedmap.Pair[string, *jsonschema.Schema]{
+ Key: "name",
+ Value: &jsonschema.Schema{
+ Type: string(schema.String),
+ },
+ },
+ orderedmap.Pair[string, *jsonschema.Schema]{
+ Key: "height",
+ Value: &jsonschema.Schema{
+ Type: string(schema.Integer),
+ },
+ },
+ orderedmap.Pair[string, *jsonschema.Schema]{
+ Key: "weight",
+ Value: &jsonschema.Schema{
+ Type: string(schema.Integer),
+ },
+ },
+ ),
+ ),
+ }
+
+ ctx := context.Background()
+ chatModel, err := openai.NewChatModel(ctx, &openai.ChatModelConfig{
+ APIKey: os.Getenv("OPENAI_API_KEY"),
+ Model: os.Getenv("OPENAI_MODEL"),
+ BaseURL: os.Getenv("OPENAI_BASE_URL"),
+ ByAzure: func() bool {
+ if os.Getenv("OPENAI_BY_AZURE") == "true" {
+ return true
+ }
+ return false
+ }(),
+ ResponseFormat: &openai.ChatCompletionResponseFormat{
+ Type: openai.ChatCompletionResponseFormatTypeJSONSchema,
+ JSONSchema: &openai.ChatCompletionResponseFormatJSONSchema{
+ Name: "person",
+ Description: "data that describes a person",
+ Strict: false,
+ JSONSchema: js,
+ },
+ },
+ })
+ if err != nil {
+ log.Fatalf("NewChatModel failed, err=%v", err)
+ }
+
+ resp, err := chatModel.Generate(ctx, []*schema.Message{
+ {
+ Role: schema.System,
+ Content: "Parse the user input into the specified json struct",
+ },
+ {
+ Role: schema.User,
+ Content: "John is one meter seventy tall and weighs sixty kilograms",
+ },
+ })
+
+ if err != nil {
+ log.Fatalf("Generate of openai failed, err=%v", err)
+ }
+
+ result := &Person{}
+ err = json.Unmarshal([]byte(resp.Content), result)
+ if err != nil {
+ log.Fatalf("Unmarshal of openai failed, err=%v", err)
+ }
+ fmt.Printf("%+v", *result)
+}
+```
+
+## **Usage**
+
+### **Component Initialization**
+
+The OpenAI model is initialized through the `NewChatModel` function with the following main configuration parameters:
+
+```go
+import "github.com/cloudwego/eino-ext/components/model/openai"
+
+model, err := openai.NewChatModel(ctx, &openai.ChatModelConfig{
+ // Azure OpenAI Service configuration (optional)
+ ByAzure: false, // Whether to use Azure OpenAI
+ BaseURL: "your-url", // Azure API base URL
+ APIVersion: "2023-05-15", // Azure API version
+
+ // Basic configuration
+ APIKey: "your-key", // API key
+ Timeout: 30 * time.Second, // Timeout
+
+ // Model parameters
+ Model: "gpt-4", // Model name
+ MaxTokens: &maxTokens,// Maximum generation length
+ Temperature: &temp, // Temperature
+ TopP: &topP, // Top-P sampling
+ Stop: []string{},// Stop words
+ PresencePenalty: &pp, // Presence penalty
+ FrequencyPenalty: &fp, // Frequency penalty
+
+ // Advanced parameters
+ ResponseFormat: &format, // Response format
+ Seed: &seed, // Random seed
+ LogitBias: map[string]int{}, // Token bias
+ User: &user, // User identifier
+
+ ReasoningEffort:openai.ReasoningEffortLevelHigh, // Reasoning level, default "medium"
+
+ Modalities: make([]openai.Modality, 0), // Model response modality types: ["text","audio"] default text
+
+ Audio: &openai.Audio{ // Audio output parameters, required when modality includes audio
+ Format: openai.AudioFormatMp3,
+ Voice: openai.AudioVoiceAlloy,
+ },
+
+ ExtraFields： map[string]any{}, // Extra fields, will add or override request fields, used for experimental validation
+
+})
+```
+
+> - For detailed parameter meanings, refer to: [https://platform.openai.com/docs/api-reference/chat/create](https://platform.openai.com/docs/api-reference/chat/create)
+> - For Azure-related services, refer to: [https://learn.microsoft.com/en-us/azure/ai-services/openai/](https://learn.microsoft.com/en-us/azure/ai-services/openai/)
+
+### **Generate Conversation**
+
+Conversation generation supports both normal mode and streaming mode:
+
+```go
+// Invoke mode
+response, err := model.Generate(ctx, messages)
+
+// Streaming mode
+stream, err := model.Stream(ctx, messages)
+```
+
+Message format example:
+
+```go
+import (
"os"
+ "encoding/base64"
- "github.com/bytedance/sonic"
- "github.com/cloudwego/eino-ext/components/model/openai"
"github.com/cloudwego/eino/schema"
)
-func main() {
- ctx := context.Background()
- chatModel, err := openai.NewChatModel(ctx, &openai.ChatModelConfig{
- APIKey: os.Getenv("OPENAI_API_KEY"),
- Model: os.Getenv("OPENAI_MODEL"),
- BaseURL: os.Getenv("OPENAI_BASE_URL"),
- ByAzure: func() bool { return os.Getenv("OPENAI_BY_AZURE") == "true" }(),
- ReasoningEffort: openai.ReasoningEffortLevelHigh,
- Modalities: []openai.Modality{openai.AudioModality, openai.TextModality},
- Audio: &openai.Audio{ Format: openai.AudioFormatMp3, Voice: openai.AudioVoiceAlloy },
- })
- if err != nil { log.Fatalf("NewChatModel failed, err=%v", err) }
+// Base64 format image data
+image, err := os.ReadFile("./examples/image/eino.png")
+ if err != nil {
+ log.Fatalf("os.ReadFile failed, err=%v\n", err)
+ }
+
+imageStr := base64.StdEncoding.EncodeToString(image)
- resp, err := chatModel.Generate(ctx, []*schema.Message{{
+messages := []*schema.Message{
+ // System message
+ schema.SystemMessage("You are an assistant"),
+
+ // Multimodal message (with image)
+ {
Role: schema.User,
UserInputMultiContent: []schema.MessageInputPart{
- {Type: schema.ChatMessagePartTypeText, Text: "help me convert the following text to speech"},
- {Type: schema.ChatMessagePartTypeText, Text: "Hello, what can I help you with?"},
+ {
+ Type: schema.ChatMessagePartTypeImageURL,
+ Image: &schema.MessageInputImage{
+ MessagePartCommon: schema.MessagePartCommon{
+ Base64Data: &imageStr,
+ MIMEType: "image/png", // required when use Base64Data
+ },
+ Detail: schema.ImageURLDetailAuto,
+ },
+ },
+ {
+ Type: schema.ChatMessagePartTypeText,
+ Text: "What is this image?",
+ },
},
- }})
- if err != nil { log.Fatalf("Generate failed, err=%v", err) }
-
- body, _ := sonic.MarshalIndent(resp, " ", " ")
- log.Printf(" body: %s\n", string(body))
+ },
}
```
### **Tool Calling**
+Supports binding tools and forced tool calling:
+
+```go
+import "github.com/cloudwego/eino/schema"
+
+// Define tools
+tools := []*schema.ToolInfo{
+ {
+ Name: "search",
+ Desc: "Search for information",
+ ParamsOneOf: schema.NewParamsOneOfByParams(map[string]*schema.ParameterInfo{
+ "query": {
+ Type: schema.String,
+ Desc: "Search keywords",
+ Required: true,
+ },
+ }),
+ },
+}
+// Bind optional tools
+err := model.BindTools(tools)
+
+// Bind forced tools
+err := model.BindForcedTools(tools)
+```
+
+> For tool-related information, refer to [Eino: ToolsNode Guide](/docs/eino/core_modules/components/tools_node_guide)
+
+### **Complete Usage Examples**
+
+#### **Direct Conversation**
+
```go
package main
import (
"context"
- "fmt"
- "io"
- "log"
- "os"
-
+ "time"
+
"github.com/cloudwego/eino-ext/components/model/openai"
"github.com/cloudwego/eino/schema"
)
func main() {
ctx := context.Background()
- chatModel, err := openai.NewChatModel(ctx, &openai.ChatModelConfig{
- APIKey: os.Getenv("OPENAI_API_KEY"),
- Model: os.Getenv("OPENAI_MODEL"),
- BaseURL: os.Getenv("OPENAI_BASE_URL"),
- ByAzure: func() bool { return os.Getenv("OPENAI_BY_AZURE") == "true" }(),
- })
- if err != nil { log.Fatalf("NewChatModel of openai failed, err=%v", err) }
-
- err = chatModel.BindForcedTools([]*schema.ToolInfo{
- { Name: "user_company", Desc: "Retrieve the user's company and position based on their name and email.", ParamsOneOf: schema.NewParamsOneOfByParams(map[string]*schema.ParameterInfo{ "name": {Type: "string", Desc: "user's name"}, "email": {Type: "string", Desc: "user's email"} }) },
- { Name: "user_salary", Desc: "Retrieve the user's salary based on their name and email.", ParamsOneOf: schema.NewParamsOneOfByParams(map[string]*schema.ParameterInfo{ "name": {Type: "string", Desc: "user's name"}, "email": {Type: "string", Desc: "user's email"} }) },
+
+ // Initialize model
+ model, err := openai.NewChatModel(ctx, &openai.ChatModelConfig{
+ APIKey: "your-api-key", // required
+ Timeout: 30 * time.Second,
+ Model: "gpt-4", // required
+
+ // If the model supports audio generation and you need to generate audio, configure as follows
+ // Modalities: []openai.Modality{openai.AudioModality, openai.TextModality},
+ //Audio: &openai.Audio{
+ // Format: openai.AudioFormatMp3,
+ // Voice: openai.AudioVoiceAlloy,
+ //},
+},
+
})
- if err != nil { log.Fatalf("BindForcedTools of openai failed, err=%v", err) }
-
- resp, err := chatModel.Generate(ctx, []*schema.Message{{ Role: schema.System, Content: "As a real estate agent, provide relevant property information based on the user's salary and job using the user_company and user_salary APIs. An email address is required." }, { Role: schema.User, Content: "My name is John and my email is john@abc.com，Please recommend some houses that suit me." }})
- if err != nil { log.Fatalf("Generate of openai failed, err=%v", err) }
- fmt.Printf("output: \n%v", resp)
-
- sr, err := chatModel.Stream(ctx, []*schema.Message{{ Role: schema.System, Content: "As a real estate agent, provide relevant property information based on the user's salary and job using the user_company and user_salary APIs. An email address is required." }, { Role: schema.User, Content: "My name is John and my email is john@abc.com，Please recommend some houses that suit me." }})
- if err != nil { log.Fatalf("Stream of openai failed, err=%v", err) }
- var messages []*schema.Message
- for { chunk, err := sr.Recv(); if err == io.EOF { break } ; if err != nil { log.Fatalf("Recv failed, err=%v", err) } ; messages = append(messages, chunk) }
- resp, err = schema.ConcatMessages(messages)
- if err != nil { log.Fatalf("ConcatMessages of openai failed, err=%v", err) }
- fmt.Printf("stream output: \n%v", resp)
+ if err != nil {
+ panic(err)
+ }
+
+ // Base64 format image data
+ image, err := os.ReadFile("./examples/image/cat.png")
+ if err != nil {
+ log.Fatalf("os.ReadFile failed, err=%v\n", err)
+ }
+
+ imageStr := base64.StdEncoding.EncodeToString(image)
+
+ // Request messages
+ messages := []*schema.Message{
+ schema.SystemMessage("You are an image generation assistant that can generate images similar in style to the user's provided image"),
+ {
+ Role: schema.User,
+ UserInputMultiContent: []schema.MessageInputPart{
+ {
+ Type: schema.ChatMessagePartTypeImage,
+ Image: &schema.MessageInputImage{
+ MessagePartCommon: schema.MessagePartCommon{
+ Base64Data: &imageStr,
+ MIMEType: "image/png", // required when use Base64Data
+ },
+ Detail: schema.ImageURLDetailAuto,
+ },
+ {
+ Type: schema.ChatMessagePartTypeText,
+ Text: "Generate an image of a cat",
+ },
+ },
+ },
+ }
+
+ // Generate response
+ response, err := model.Generate(ctx, messages)
+ if err != nil {
+ panic(err)
+ }
+
+ // Process response
+ /*
+ The generated multimodal content is stored in the response.AssistantGentMultiContent field
+ In this example, the final generated message looks like:
+ AssistantMessage = schema.Message{
+ Role: schema.Assistant,
+ AssistantGenMultiContent : []schema.MessageOutputPart{
+ {Type: schema.ChatMessagePartTypeImageURL,
+ Image: &schema.MessageOutputImage{
+ MessagePartCommon: schema.MessagePartCommon{
+ Base64Data: &DataStr,
+ MIMEType: "image/png",
+ },
+ },
+ },
+ },
+ }
+ */
+
+ fmt.Printf("Assistant: %s\n", resp)
}
```
-### **Structured Output**
+#### **Streaming Conversation**
```go
package main
import (
"context"
- "encoding/json"
- "fmt"
- "log"
- "os"
-
+ "time"
+
"github.com/cloudwego/eino-ext/components/model/openai"
"github.com/cloudwego/eino/schema"
- "github.com/eino-contrib/jsonschema"
- orderedmap "github.com/wk8/go-ordered-map/v2"
)
func main() {
- type Person struct { Name string `json:"name"`; Height int `json:"height"`; Weight int `json:"weight"` }
-
- js := &jsonschema.Schema{ Type: string(schema.Object), Properties: orderedmap.New[string, *jsonschema.Schema]( orderedmap.WithInitialData[string, *jsonschema.Schema]( orderedmap.Pair[string, *jsonschema.Schema]{ Key: "name", Value: &jsonschema.Schema{ Type: string(schema.String) } }, orderedmap.Pair[string, *jsonschema.Schema]{ Key: "height", Value: &jsonschema.Schema{ Type: string(schema.Integer) } }, orderedmap.Pair[string, *jsonschema.Schema]{ Key: "weight", Value: &jsonschema.Schema{ Type: string(schema.Integer) } }, ), ), }
-
ctx := context.Background()
- cm, err := openai.NewChatModel(ctx, &openai.ChatModelConfig{
- APIKey: os.Getenv("OPENAI_API_KEY"),
- Model: os.Getenv("OPENAI_MODEL"),
- BaseURL: os.Getenv("OPENAI_BASE_URL"),
- ByAzure: func() bool { return os.Getenv("OPENAI_BY_AZURE") == "true" }(),
- ResponseFormat: &openai.ChatCompletionResponseFormat{
- Type: openai.ChatCompletionResponseFormatTypeJSONSchema,
- JSONSchema: &openai.ChatCompletionResponseFormatJSONSchema{ Name: "person", Description: "data that describes a person", Strict: false, JSONSchema: js },
- },
+
+ // Initialize model
+ model, err := openai.NewChatModel(ctx, &openai.ChatModelConfig{
+ APIKey: "your-api-key",
+ Timeout: 30 * time.Second,
+ Model: "gpt-4",
})
- if err != nil { log.Fatalf("NewChatModel failed, err=%v", err) }
-
- resp, err := cm.Generate(ctx, []*schema.Message{{ Role: schema.System, Content: "Parse the user input into the specified json struct" }, { Role: schema.User, Content: "John is one meter seventy tall and weighs sixty kilograms" }})
- if err != nil { log.Fatalf("Generate failed, err=%v", err) }
-
- result := &Person{}
- if err := json.Unmarshal([]byte(resp.Content), result); err != nil { log.Fatalf("Unmarshal failed, err=%v", err) }
- fmt.Printf("%+v", *result)
+ if err != nil {
+ panic(err)
+ }
+
+ // Prepare messages
+ messages := []*schema.Message{
+ schema.SystemMessage("You are an assistant"),
+ schema.UserMessage("Write a story"),
+ }
+
+ // Get streaming response
+ reader, err := model.Stream(ctx, messages)
+ if err != nil {
+ panic(err)
+ }
+ defer reader.Close() // Remember to close
+
+ // Process streaming content
+ for {
+ chunk, err := reader.Recv()
+ if err != nil {
+ break
+ }
+ print(chunk.Content)
+ }
}
```
@@ -461,7 +934,7 @@ func main() {
## **Related Documentation**
-- `Eino: ChatModel Guide` at `/docs/eino/core_modules/components/chat_model_guide`
-- `Eino: ToolsNode Guide` at `/docs/eino/core_modules/components/tools_node_guide`
-- `ChatModel - ARK` at `/docs/eino/ecosystem_integration/chat_model/chat_model_ark`
-- `ChatModel - Ollama` at `/docs/eino/ecosystem_integration/chat_model/chat_model_ollama`
+- [Eino: ChatModel Guide](/docs/eino/core_modules/components/chat_model_guide)
+- [Eino: ToolsNode Guide](/docs/eino/core_modules/components/tools_node_guide)
+- [ChatModel - ARK](/docs/eino/ecosystem_integration/chat_model/chat_model_ark)
+- [ChatModel - Ollama](/docs/eino/ecosystem_integration/chat_model/chat_model_ollama)
diff --git a/content/en/docs/eino/ecosystem_integration/embedding/_index.md b/content/en/docs/eino/ecosystem_integration/embedding/_index.md
index a655c18ab7c..480fd62fb08 100644
--- a/content/en/docs/eino/ecosystem_integration/embedding/_index.md
+++ b/content/en/docs/eino/ecosystem_integration/embedding/_index.md
@@ -1,6 +1,6 @@
---
Description: ""
-date: "2025-01-06"
+date: "2026-01-20"
lastmod: ""
tags: []
title: Embedding
diff --git a/content/en/docs/eino/ecosystem_integration/embedding/embedding_ark.md b/content/en/docs/eino/ecosystem_integration/embedding/embedding_ark.md
index 1ca938291d6..37b66f88508 100644
--- a/content/en/docs/eino/ecosystem_integration/embedding/embedding_ark.md
+++ b/content/en/docs/eino/ecosystem_integration/embedding/embedding_ark.md
@@ -1,6 +1,6 @@
---
Description: ""
-date: "2025-01-20"
+date: "2026-01-20"
lastmod: ""
tags: []
title: Embedding - ARK
diff --git a/content/en/docs/eino/ecosystem_integration/embedding/embedding_openai.md b/content/en/docs/eino/ecosystem_integration/embedding/embedding_openai.md
index b132706816c..77cb54dedae 100644
--- a/content/en/docs/eino/ecosystem_integration/embedding/embedding_openai.md
+++ b/content/en/docs/eino/ecosystem_integration/embedding/embedding_openai.md
@@ -1,6 +1,6 @@
---
Description: ""
-date: "2025-01-20"
+date: "2026-01-20"
lastmod: ""
tags: []
title: Embedding - OpenAI
diff --git a/content/en/docs/eino/ecosystem_integration/indexer/_index.md b/content/en/docs/eino/ecosystem_integration/indexer/_index.md
index c84033f8cef..31aff746561 100644
--- a/content/en/docs/eino/ecosystem_integration/indexer/_index.md
+++ b/content/en/docs/eino/ecosystem_integration/indexer/_index.md
@@ -1,6 +1,6 @@
---
Description: ""
-date: "2025-01-20"
+date: "2026-01-20"
lastmod: ""
tags: []
title: Indexer
diff --git a/content/en/docs/eino/ecosystem_integration/indexer/indexer_elasticsearch7.md b/content/en/docs/eino/ecosystem_integration/indexer/indexer_elasticsearch7.md
new file mode 100644
index 00000000000..f4562a29f27
--- /dev/null
+++ b/content/en/docs/eino/ecosystem_integration/indexer/indexer_elasticsearch7.md
@@ -0,0 +1,155 @@
+---
+Description: ""
+date: "2026-01-20"
+lastmod: ""
+tags: []
+title: 'Indexer - Elasticsearch 7 '
+weight: 0
+---
+
+> **Cloud Search Service Introduction**
+>
+> Cloud Search Service is a fully managed, one-stop information retrieval and analysis platform that provides ElasticSearch and OpenSearch engines, supporting full-text search, vector search, hybrid search, and spatiotemporal search capabilities.
+
+An Elasticsearch 7.x indexer implementation for [Eino](https://github.com/cloudwego/eino), implementing the `Indexer` interface. This component seamlessly integrates with Eino's document indexing system, providing powerful vector storage and retrieval capabilities.
+
+## Features
+
+- Implements `github.com/cloudwego/eino/components/indexer.Indexer`
+- Easy integration with Eino indexing system
+- Configurable Elasticsearch parameters
+- Supports vector similarity search
+- Supports batch indexing operations
+- Supports custom field mapping
+- Flexible document vectorization support
+
+## Installation
+
+```bash
+go get github.com/cloudwego/eino-ext/components/indexer/es7@latest
+```
+
+## Quick Start
+
+Here is a simple example of using the indexer:
+
+```go
+import (
+ "context"
+ "fmt"
+ "log"
+ "os"
+
+ "github.com/cloudwego/eino/components/embedding"
+ "github.com/cloudwego/eino/schema"
+ elasticsearch "github.com/elastic/go-elasticsearch/v7"
+
+ "github.com/cloudwego/eino-ext/components/embedding/ark"
+ "github.com/cloudwego/eino-ext/components/indexer/es7"
+)
+
+const (
+ indexName = "eino_example"
+ fieldContent = "content"
+ fieldContentVector = "content_vector"
+ fieldExtraLocation = "location"
+ docExtraLocation = "location"
+)
+
+func main() {
+ ctx := context.Background()
+
+ // ES supports multiple connection methods
+ username := os.Getenv("ES_USERNAME")
+ password := os.Getenv("ES_PASSWORD")
+
+ client, _ := elasticsearch.NewClient(elasticsearch.Config{
+ Addresses: []string{"http://localhost:9200"},
+ Username: username,
+ Password: password,
+ })
+
+ // Create embedding component using Volcengine ARK
+ emb, _ := ark.NewEmbedder(ctx, &ark.EmbeddingConfig{
+ APIKey: os.Getenv("ARK_API_KEY"),
+ Region: os.Getenv("ARK_REGION"),
+ Model: os.Getenv("ARK_MODEL"),
+ })
+
+ // Load documents
+ docs := []*schema.Document{
+ {
+ ID: "1",
+ Content: "Eiffel Tower: Located in Paris, France.",
+ MetaData: map[string]any{
+ docExtraLocation: "France",
+ },
+ },
+ {
+ ID: "2",
+ Content: "The Great Wall: Located in China.",
+ MetaData: map[string]any{
+ docExtraLocation: "China",
+ },
+ },
+ }
+
+ // Create ES indexer component
+ indexer, _ := es7.NewIndexer(ctx, &es7.IndexerConfig{
+ Client: client,
+ Index: indexName,
+ BatchSize: 10,
+ DocumentToFields: func(ctx context.Context, doc *schema.Document) (field2Value map[string]es7.FieldValue, err error) {
+ return map[string]es7.FieldValue{
+ fieldContent: {
+ Value: doc.Content,
+ EmbedKey: fieldContentVector, // Vectorize document content and save to "content_vector" field
+ },
+ fieldExtraLocation: {
+ Value: doc.MetaData[docExtraLocation],
+ },
+ }, nil
+ },
+ Embedding: emb,
+ })
+
+ ids, err := indexer.Store(ctx, docs)
+ if err != nil {
+ fmt.Printf("index error: %v\n", err)
+ return
+ }
+ fmt.Println("indexed ids:", ids)
+}
+```
+
+## Configuration
+
+The indexer can be configured using the `IndexerConfig` struct:
+
+```go
+type IndexerConfig struct {
+ Client *elasticsearch.Client // Required: Elasticsearch client instance
+ Index string // Required: Index name for storing documents
+ BatchSize int // Optional: Maximum text embedding batch size (default: 5)
+
+ // Required: Function to map Document fields to Elasticsearch fields
+ DocumentToFields func(ctx context.Context, doc *schema.Document) (map[string]FieldValue, error)
+
+ // Optional: Required only when vectorization is needed
+ Embedding embedding.Embedder
+}
+
+// FieldValue defines how a field should be stored and vectorized
+type FieldValue struct {
+ Value any // Original value to store
+ EmbedKey string // If set, Value will be vectorized and saved
+ Stringify func(val any) (string, error) // Optional: Custom string conversion
+}
+```
+
+## Getting Help
+
+If you have any questions or feature suggestions, feel free to join the oncall group.
+
+- [Eino Documentation](https://www.cloudwego.io/docs/eino/)
+- [Elasticsearch Go Client Documentation](https://github.com/elastic/go-elasticsearch)
diff --git a/content/en/docs/eino/ecosystem_integration/indexer/indexer_elasticsearch9.md b/content/en/docs/eino/ecosystem_integration/indexer/indexer_elasticsearch9.md
new file mode 100644
index 00000000000..4fd82d23e3b
--- /dev/null
+++ b/content/en/docs/eino/ecosystem_integration/indexer/indexer_elasticsearch9.md
@@ -0,0 +1,173 @@
+---
+Description: ""
+date: "2026-01-20"
+lastmod: ""
+tags: []
+title: Indexer - Elasticsearch 9
+weight: 0
+---
+
+> **Cloud Search Service Introduction**
+>
+> Cloud Search Service is a fully managed, one-stop information retrieval and analysis platform that provides ElasticSearch and OpenSearch engines, supporting full-text search, vector search, hybrid search, and spatiotemporal search capabilities.
+
+An Elasticsearch 9.x indexer implementation for [Eino](https://github.com/cloudwego/eino), implementing the `Indexer` interface. This enables seamless integration with Eino's vector storage and retrieval system, enhancing semantic search capabilities.
+
+## Features
+
+- Implements `github.com/cloudwego/eino/components/indexer.Indexer`
+- Easy integration with Eino's indexing system
+- Configurable Elasticsearch parameters
+- Supports vector similarity search
+- Batch indexing operations
+- Custom field mapping support
+- Flexible document vectorization
+
+## Installation
+
+```bash
+go get github.com/cloudwego/eino-ext/components/indexer/es9@latest
+```
+
+## Quick Start
+
+Here is a quick example of using the indexer, for more details please read components/indexer/es9/examples/indexer/add_documents.go:
+
+```go
+import (
+ "context"
+ "fmt"
+ "log"
+ "os"
+
+ "github.com/cloudwego/eino/components/embedding"
+ "github.com/cloudwego/eino/schema"
+ "github.com/elastic/go-elasticsearch/v9"
+
+ "github.com/cloudwego/eino-ext/components/embedding/ark"
+ "github.com/cloudwego/eino-ext/components/indexer/es9"
+)
+
+const (
+ indexName = "eino_example"
+ fieldContent = "content"
+ fieldContentVector = "content_vector"
+ fieldExtraLocation = "location"
+ docExtraLocation = "location"
+)
+
+func main() {
+ ctx := context.Background()
+
+ // ES supports multiple connection methods
+ username := os.Getenv("ES_USERNAME")
+ password := os.Getenv("ES_PASSWORD")
+ httpCACertPath := os.Getenv("ES_HTTP_CA_CERT_PATH")
+
+ var cert []byte
+ var err error
+ if httpCACertPath != "" {
+ cert, err = os.ReadFile(httpCACertPath)
+ if err != nil {
+ log.Fatalf("read file failed, err=%v", err)
+ }
+ }
+
+ client, _ := elasticsearch.NewClient(elasticsearch.Config{
+ Addresses: []string{"https://localhost:9200"},
+ Username: username,
+ Password: password,
+ CACert: cert,
+ })
+
+ // 2. Create embedding component (using Ark)
+ // Replace "ARK_API_KEY", "ARK_REGION", "ARK_MODEL" with actual configuration
+ emb, _ := ark.NewEmbedder(ctx, &ark.EmbeddingConfig{
+ APIKey: os.Getenv("ARK_API_KEY"),
+ Region: os.Getenv("ARK_REGION"),
+ Model: os.Getenv("ARK_MODEL"),
+ })
+
+ // 3. Prepare documents
+ // Documents typically contain ID and Content
+ // Can also include additional Metadata for filtering or other purposes
+ docs := []*schema.Document{
+ {
+ ID: "1",
+ Content: "Eiffel Tower: Located in Paris, France.",
+ MetaData: map[string]any{
+ docExtraLocation: "France",
+ },
+ },
+ {
+ ID: "2",
+ Content: "The Great Wall: Located in China.",
+ MetaData: map[string]any{
+ docExtraLocation: "China",
+ },
+ },
+ }
+
+ // 4. Create ES indexer component
+ indexer, _ := es9.NewIndexer(ctx, &es9.IndexerConfig{
+ Client: client,
+ Index: indexName,
+ BatchSize: 10,
+ // DocumentToFields specifies how to map document fields to ES fields
+ DocumentToFields: func(ctx context.Context, doc *schema.Document) (field2Value map[string]es9.FieldValue, err error) {
+ return map[string]es9.FieldValue{
+ fieldContent: {
+ Value: doc.Content,
+ EmbedKey: fieldContentVector, // Vectorize document content and save to "content_vector" field
+ },
+ fieldExtraLocation: {
+ // Additional metadata field
+ Value: doc.MetaData[docExtraLocation],
+ },
+ }, nil
+ },
+ // Provide embedding component for vectorization
+ Embedding: emb,
+ })
+
+ // 5. Index documents
+ ids, err := indexer.Store(ctx, docs)
+ if err != nil {
+ fmt.Printf("index error: %v\n", err)
+ return
+ }
+ fmt.Println("indexed ids:", ids)
+}
+```
+
+## Configuration
+
+The indexer can be configured using the `IndexerConfig` struct:
+
+```go
+type IndexerConfig struct {
+ Client *elasticsearch.Client // Required: Elasticsearch client instance
+ Index string // Required: Index name for storing documents
+ BatchSize int // Optional: Maximum text count for embedding (default: 5)
+
+ // Required: Function to map Document fields to Elasticsearch fields
+ DocumentToFields func(ctx context.Context, doc *schema.Document) (map[string]FieldValue, error)
+
+ // Optional: Required only when vectorization is needed
+ Embedding embedding.Embedder
+}
+
+// FieldValue defines how a field should be stored and vectorized
+type FieldValue struct {
+ Value any // Original value to store
+ EmbedKey string // If set, Value will be vectorized and saved
+ Stringify func(val any) (string, error) // Optional: Custom string conversion
+}
+```
+
+## Getting Help
+
+If you have any questions or feature suggestions, feel free to join the oncall group.
+
+- [Eino Documentation](https://www.cloudwego.io/docs/eino/)
+- [Elasticsearch Go Client Documentation](https://github.com/elastic/go-elasticsearch)
diff --git a/content/en/docs/eino/ecosystem_integration/indexer/indexer_es8.md b/content/en/docs/eino/ecosystem_integration/indexer/indexer_es8.md
index f138e921e76..88a5c23b245 100644
--- a/content/en/docs/eino/ecosystem_integration/indexer/indexer_es8.md
+++ b/content/en/docs/eino/ecosystem_integration/indexer/indexer_es8.md
@@ -1,118 +1,168 @@
---
Description: ""
-date: "2025-12-11"
+date: "2026-01-20"
lastmod: ""
tags: []
-title: Indexer - ES8
+title: Indexer - ElasticSearch 8
weight: 0
---
-## **ES8 Indexer**
+> **Cloud Search Service Introduction**
+>
+> Cloud Search Service is a fully managed, one-stop information retrieval and analysis platform that provides ElasticSearch and OpenSearch engines, supporting full-text search, vector search, hybrid search, and spatio-temporal search capabilities.
-This is an Elasticsearch 8.x indexer implementation for [Eino](https://github.com/cloudwego/eino) that implements the `Indexer` interface. It integrates with Eino’s vector storage and retrieval system for semantic search.
+This is an Elasticsearch 8.x indexer implementation for [Eino](https://github.com/cloudwego/eino) that implements the `Indexer` interface. It integrates with Eino's vector storage and retrieval system for semantic search.
-## **Features**
+## Features
- Implements `github.com/cloudwego/eino/components/indexer.Indexer`
-- Easy integration with Eino indexing
+- Easy integration with Eino indexing system
- Configurable Elasticsearch parameters
- Supports vector similarity search
- Batch indexing operations
-- Custom field mapping
+- Custom field mapping support
- Flexible document embedding
-## **Installation**
+## Installation
```bash
go get github.com/cloudwego/eino-ext/components/indexer/es8@latest
```
-## **Quick Start**
+## Quick Start
-Example usage (see `components/indexer/es8/examples/indexer/add_documents.go` for details):
+Here is a quick example of using the indexer. For more details, please read components/indexer/es8/examples/indexer/add_documents.go:
```go
import (
- "github.com/cloudwego/eino/components/embedding"
- "github.com/cloudwego/eino/schema"
- "github.com/elastic/go-elasticsearch/v8"
- "github.com/cloudwego/eino-ext/components/indexer/es8"
+ "context"
+ "os"
+
+ "github.com/cloudwego/eino/components/embedding"
+ "github.com/cloudwego/eino/schema"
+ elasticsearch "github.com/elastic/go-elasticsearch/v8"
+
+ "github.com/cloudwego/eino-ext/components/embedding/ark"
+ "github.com/cloudwego/eino-ext/components/indexer/es8"
)
const (
- indexName = "eino_example"
- fieldContent = "content"
- fieldContentVector = "content_vector"
- fieldExtraLocation = "location"
- docExtraLocation = "location"
+ indexName = "eino_example"
+ fieldContent = "content"
+ fieldContentVector = "content_vector"
+ fieldExtraLocation = "location"
+ docExtraLocation = "location"
)
func main() {
- ctx := context.Background()
-
- username := os.Getenv("ES_USERNAME")
- password := os.Getenv("ES_PASSWORD")
- httpCACertPath := os.Getenv("ES_HTTP_CA_CERT_PATH")
-
- cert, err := os.ReadFile(httpCACertPath)
- if err != nil { log.Fatalf("read file failed, err=%v", err) }
-
- client, err := elasticsearch.NewClient(elasticsearch.Config{
- Addresses: []string{"https://localhost:9200"},
- Username: username,
- Password: password,
- CACert: cert,
- })
- if err != nil { log.Panicf("connect es8 failed, err=%v", err) }
-
- emb := createYourEmbedding()
- docs := loadYourDocs()
-
- indexer, err := es8.NewIndexer(ctx, &es8.IndexerConfig{
- Client: client,
- Index: indexName,
- BatchSize: 10,
- DocumentToFields: func(ctx context.Context, doc *schema.Document) (map[string]es8.FieldValue, error) {
- return map[string]es8.FieldValue{
- fieldContent: { Value: doc.Content, EmbedKey: fieldContentVector },
- fieldExtraLocation: { Value: doc.MetaData[docExtraLocation] },
- }, nil
- },
- Embedding: emb,
- })
- if err != nil { log.Panicf("create indexer failed, err=%v", err) }
-
- ids, err := indexer.Store(ctx, docs)
- if err != nil { log.Panicf("create docs failed, err=%v", err) }
- fmt.Println(ids)
+ ctx := context.Background()
+ // es supports multiple ways to connect
+ username := os.Getenv("ES_USERNAME")
+ password := os.Getenv("ES_PASSWORD")
+
+ // 1. Create ES client
+ httpCACertPath := os.Getenv("ES_HTTP_CA_CERT_PATH")
+ if httpCACertPath != "" {
+ cert, err := os.ReadFile(httpCACertPath)
+ if err != nil {
+ log.Fatalf("read file failed, err=%v", err)
+ }
+ }
+
+ client, _ := elasticsearch.NewClient(elasticsearch.Config{
+ Addresses: []string{"https://localhost:9200"},
+ Username: username,
+ Password: password,
+ CACert: cert,
+ })
+
+ // 2. Create embedding component
+ // Using Volcengine Ark, replace environment variables with real configuration
+ emb, _ := ark.NewEmbedder(ctx, &ark.EmbeddingConfig{
+ APIKey: os.Getenv("ARK_API_KEY"),
+ Region: os.Getenv("ARK_REGION"),
+ Model: os.Getenv("ARK_MODEL"),
+ })
+
+ // 3. Prepare documents
+ // Documents typically contain ID and Content. You can also add extra metadata for filtering and other purposes.
+ docs := []*schema.Document{
+ {
+ ID: "1",
+ Content: "Eiffel Tower: Located in Paris, France.",
+ MetaData: map[string]any{
+ docExtraLocation: "France",
+ },
+ },
+ {
+ ID: "2",
+ Content: "The Great Wall: Located in China.",
+ MetaData: map[string]any{
+ docExtraLocation: "China",
+ },
+ },
+ }
+
+ // 4. Create ES indexer component
+ indexer, _ := es8.NewIndexer(ctx, &es8.IndexerConfig{
+ Client: client,
+ Index: indexName,
+ BatchSize: 10,
+ // DocumentToFields specifies how to map document fields to ES fields
+ DocumentToFields: func(ctx context.Context, doc *schema.Document) (field2Value map[string]es8.FieldValue, err error) {
+ return map[string]es8.FieldValue{
+ fieldContent: {
+ Value: doc.Content,
+ EmbedKey: fieldContentVector, // Embed document content and save to "content_vector" field
+ },
+ fieldExtraLocation: {
+ // Extra metadata field
+ Value: doc.MetaData[docExtraLocation],
+ },
+ }, nil
+ },
+ // Provide embedding component for vectorization
+ Embedding: emb,
+ })
+
+ // 5. Index documents
+ ids, err := indexer.Store(ctx, docs)
+ if err != nil {
+ fmt.Printf("index error: %v\n", err)
+ return
+ }
+ fmt.Println("indexed ids:", ids)
}
```
-## **Configuration**
+## Configuration
-Configure via `IndexerConfig`:
+Configure the indexer using the `IndexerConfig` struct:
```go
type IndexerConfig struct {
- Client *elasticsearch.Client // required: Elasticsearch client instance
- Index string // required: index name to store documents
- BatchSize int // optional: batch size (default: 5)
-
- // required: map document fields to ES fields
+ Client *elasticsearch.Client // Required: Elasticsearch client instance
+ Index string // Required: Index name to store documents
+ BatchSize int // Optional: Maximum number of texts for embedding (default: 5)
+
+ // Required: Function to map Document fields to Elasticsearch fields
DocumentToFields func(ctx context.Context, doc *schema.Document) (map[string]FieldValue, error)
-
- // optional: only needed when embedding is required
+
+ // Optional: Only required when vectorization is needed
Embedding embedding.Embedder
}
+// FieldValue defines how a field should be stored and vectorized
type FieldValue struct {
- Value any // raw value to store
- EmbedKey string // if set, Value will be embedded and saved under this field
- Stringify func(val any) (string, error) // optional: custom string conversion
+ Value any // Raw value to store
+ EmbedKey string // If set, Value will be vectorized and saved
+ Stringify func(val any) (string, error) // Optional: Custom string conversion
}
```
-## **More Details**
+## Getting Help
+
+If you have any questions or feature suggestions, feel free to join the oncall group.
- - [Eino docs](https://github.com/cloudwego/eino)
- - [Elasticsearch Go client](https://github.com/elastic/go-elasticsearch)
+- [Eino Documentation](https://www.cloudwego.io/docs/eino/)
+- [Elasticsearch Go Client Documentation](https://github.com/elastic/go-elasticsearch)
diff --git a/content/en/docs/eino/ecosystem_integration/indexer/indexer_milvus.md b/content/en/docs/eino/ecosystem_integration/indexer/indexer_milvus.md
index 2c130d57cf6..94403ed9651 100644
--- a/content/en/docs/eino/ecosystem_integration/indexer/indexer_milvus.md
+++ b/content/en/docs/eino/ecosystem_integration/indexer/indexer_milvus.md
@@ -1,15 +1,19 @@
---
Description: ""
-date: "2025-12-11"
+date: "2026-01-20"
lastmod: ""
tags: []
-title: Indexer - Milvus
+title: Indexer - Milvus v1 (Legacy)
weight: 0
---
+> **Module Note:** This module (`EINO-ext/milvus`) is based on `milvus-sdk-go`. Since the underlying SDK has been deprecated and only supports up to Milvus 2.4, this module is retained only for backward compatibility.
+>
+> **Recommendation:** New users should use [Indexer - Milvus 2 (v2.5+)](/docs/eino/ecosystem_integration/indexer/indexer_milvusv2) for continued support.
+
## **Milvus Storage**
-Vector storage based on Milvus 2.x that provides an `Indexer` implementation for [Eino](https://github.com/cloudwego/eino). Integrates with Eino’s vector storage and retrieval for semantic search.
+Vector storage based on Milvus 2.x that provides an `Indexer` implementation for [Eino](https://github.com/cloudwego/eino). This component integrates seamlessly with Eino's vector storage and retrieval system for semantic search.
## **Quick Start**
@@ -25,43 +29,81 @@ go get github.com/cloudwego/eino-ext/components/indexer/milvus
package main
import (
- "context"
- "log"
- "os"
-
- "github.com/cloudwego/eino-ext/components/embedding/ark"
- "github.com/cloudwego/eino/schema"
- "github.com/milvus-io/milvus-sdk-go/v2/client"
-
- "github.com/cloudwego/eino-ext/components/indexer/milvus"
+ "context"
+ "log"
+ "os"
+
+ "github.com/cloudwego/eino-ext/components/embedding/ark"
+ "github.com/cloudwego/eino/schema"
+ "github.com/milvus-io/milvus-sdk-go/v2/client"
+
+ "github.com/cloudwego/eino-ext/components/indexer/milvus"
)
func main() {
- addr := os.Getenv("MILVUS_ADDR")
- username := os.Getenv("MILVUS_USERNAME")
- password := os.Getenv("MILVUS_PASSWORD")
- arkApiKey := os.Getenv("ARK_API_KEY")
- arkModel := os.Getenv("ARK_MODEL")
-
- ctx := context.Background()
- cli, err := client.NewClient(ctx, client.Config{ Address: addr, Username: username, Password: password })
- if err != nil { log.Fatalf("Failed to create client: %v", err) }
- defer cli.Close()
-
- emb, err := ark.NewEmbedder(ctx, &ark.EmbeddingConfig{ APIKey: arkApiKey, Model: arkModel })
- if err != nil { log.Fatalf("Failed to create embedding: %v", err) }
-
- indexer, err := milvus.NewIndexer(ctx, &milvus.IndexerConfig{ Client: cli, Embedding: emb })
- if err != nil { log.Fatalf("Failed to create indexer: %v", err) }
- log.Printf("Indexer created success")
-
- docs := []*schema.Document{
- { ID: "milvus-1", Content: "milvus is an open-source vector database", MetaData: map[string]any{ "h1": "milvus", "h2": "open-source", "h3": "vector database" } },
- { ID: "milvus-2", Content: "milvus is a distributed vector database" },
- }
- ids, err := indexer.Store(ctx, docs)
- if err != nil { log.Fatalf("Failed to store: %v", err) }
- log.Printf("Store success, ids: %v", ids)
+ // Get the environment variables
+ addr := os.Getenv("MILVUS_ADDR")
+ username := os.Getenv("MILVUS_USERNAME")
+ password := os.Getenv("MILVUS_PASSWORD")
+ arkApiKey := os.Getenv("ARK_API_KEY")
+ arkModel := os.Getenv("ARK_MODEL")
+
+ // Create a client
+ ctx := context.Background()
+ cli, err := client.NewClient(ctx, client.Config{
+ Address: addr,
+ Username: username,
+ Password: password,
+ })
+ if err != nil {
+ log.Fatalf("Failed to create client: %v", err)
+ return
+ }
+ defer cli.Close()
+
+ // Create an embedding model
+ emb, err := ark.NewEmbedder(ctx, &ark.EmbeddingConfig{
+ APIKey: arkApiKey,
+ Model: arkModel,
+ })
+ if err != nil {
+ log.Fatalf("Failed to create embedding: %v", err)
+ return
+ }
+
+ // Create an indexer
+ indexer, err := milvus.NewIndexer(ctx, &milvus.IndexerConfig{
+ Client: cli,
+ Embedding: emb,
+ })
+ if err != nil {
+ log.Fatalf("Failed to create indexer: %v", err)
+ return
+ }
+ log.Printf("Indexer created success")
+
+ // Store documents
+ docs := []*schema.Document{
+ {
+ ID: "milvus-1",
+ Content: "milvus is an open-source vector database",
+ MetaData: map[string]any{
+ "h1": "milvus",
+ "h2": "open-source",
+ "h3": "vector database",
+ },
+ },
+ {
+ ID: "milvus-2",
+ Content: "milvus is a distributed vector database",
+ },
+ }
+ ids, err := indexer.Store(ctx, docs)
+ if err != nil {
+ log.Fatalf("Failed to store: %v", err)
+ return
+ }
+ log.Printf("Store success, ids: %v", ids)
}
```
@@ -69,33 +111,65 @@ func main() {
```go
type IndexerConfig struct {
- Client client.Client // required
-
- // Default collection config
- Collection string
- Description string
- PartitionNum int64
- Fields []*entity.Field
- SharedNum int32
- ConsistencyLevel ConsistencyLevel
- EnableDynamicSchema bool
-
- // Convert schema.Document to row data
- DocumentConverter func(ctx context.Context, docs []*schema.Document, vectors [][]float64) ([]interface{}, error)
-
- // Vector index config
- MetricType MetricType
-
- // Embedding method to vectorize content (required)
- Embedding embedding.Embedder
+ // Client is the milvus client to call
+ // Required
+ Client client.Client
+
+ // Default collection configuration
+ // Collection is the collection name in milvus database
+ // Optional, default value is "eino_collection"
+ // If you want to use this configuration, you must add Field configuration, otherwise it will not work properly
+ Collection string
+ // Description is the description of the collection
+ // Optional, default value is "the collection for eino"
+ Description string
+ // PartitionNum is the number of collection partitions
+ // Optional, default value is 1 (disabled)
+ // If partition number is greater than 1, it means partitioning is enabled, and there must be a partition key in Fields
+ PartitionNum int64
+ // Fields are the collection fields
+ // Optional, default value is the default fields
+ Fields []*entity.Field
+ // SharedNum is the milvus parameter required for creating a collection
+ // Optional, default value is 1
+ SharedNum int32
+ // ConsistencyLevel is the milvus collection consistency policy
+ // Optional, default level is ClBounded (bounded consistency level, default tolerance is 5 seconds)
+ ConsistencyLevel ConsistencyLevel
+ // EnableDynamicSchema indicates whether the collection enables dynamic schema
+ // Optional, default value is false
+ // Enabling dynamic schema may affect milvus performance
+ EnableDynamicSchema bool
+
+ // DocumentConverter is the function to convert schema.Document to row data
+ // Optional, default value is defaultDocumentConverter
+ DocumentConverter func(ctx context.Context, docs []*schema.Document, vectors [][]float64) ([]interface{}, error)
+
+ // Index configuration for vector column
+ // MetricType is the metric type for vectors
+ // Optional, default type is HAMMING
+ MetricType MetricType
+
+ // Embedding is the vectorization method required to embed values from schema.Document content
+ // Required
+ Embedding embedding.Embedder
}
```
-## **Default Schema**
+## **Default Data Model**
+
+## Getting Help
+
+If you have any questions or feature suggestions, feel free to join the oncall group.
+
+### External References
+
+- [Milvus Documentation](https://milvus.io/docs)
+- [Milvus Index Types](https://milvus.io/docs/index.md)
+- [Milvus Metric Types](https://milvus.io/docs/metric.md)
+- [milvus-sdk-go Reference](https://milvus.io/api-reference/go/v2.4.x/About.md)
+
+### Related Documentation
-| field | type | column type | index type | desc | note |
-|----------|-----------------|---------------|--------------------------------|-------------|--------------|
-| id | string | varchar | | unique id | max len: 255 |
-| content | string | varchar | | content | max len: 1024|
-| vector | []byte | binary array | HAMMING(default) / JACCARD | content vec | dim: 81920 |
-| metadata | map[string]any | json | | metadata | |
+- [Eino: Indexer Guide](/docs/eino/core_modules/components/indexer_guide)
+- [Eino: Retriever Guide](/docs/eino/core_modules/components/retriever_guide)
diff --git a/content/en/docs/eino/ecosystem_integration/indexer/indexer_milvusv2.md b/content/en/docs/eino/ecosystem_integration/indexer/indexer_milvusv2.md
new file mode 100644
index 00000000000..68f526fd5c9
--- /dev/null
+++ b/content/en/docs/eino/ecosystem_integration/indexer/indexer_milvusv2.md
@@ -0,0 +1,395 @@
+---
+Description: ""
+date: "2026-01-20"
+lastmod: ""
+tags: []
+title: Indexer - Milvus v2 (Recommended)
+weight: 0
+---
+
+> **Milvus Vector Database Introduction**
+>
+> Milvus Vector Retrieval Service is a fully managed database service built on open-source Milvus, providing efficient unstructured data retrieval capabilities suitable for diverse AI scenarios. Customers no longer need to worry about underlying hardware resources, reducing usage costs and improving overall efficiency.
+>
+> Since the company's **internal** Milvus service uses the standard SDK, the **EINO-ext community version** is applicable.
+
+This package provides a Milvus 2.x (V2 SDK) indexer implementation for the EINO framework, supporting document storage and vector indexing.
+
+> **Note**: This package requires **Milvus 2.5+** to support server-side functions (such as BM25), basic functionality is compatible with lower versions.
+
+## Features
+
+- **Milvus V2 SDK**: Uses the latest `milvus-io/milvus/client/v2` SDK
+- **Flexible Index Types**: Supports multiple index builders including Auto, HNSW, IVF series, SCANN, DiskANN, GPU indexes, and RaBitQ (Milvus 2.6+)
+- **Hybrid Search Ready**: Native support for hybrid storage of sparse vectors (BM25/SPLADE) and dense vectors
+- **Server-side Vector Generation**: Automatic sparse vector generation using Milvus Functions (BM25)
+- **Automated Management**: Automatic handling of collection schema creation, index building, and loading
+- **Field Analysis**: Configurable text analyzers (supporting Chinese Jieba, English, Standard, etc.)
+- **Custom Document Conversion**: Flexible mapping from Eino documents to Milvus columns
+
+## Installation
+
+```bash
+go get github.com/cloudwego/eino-ext/components/indexer/milvus2
+```
+
+## Quick Start
+
+```go
+package main
+
+import (
+ "context"
+ "log"
+ "os"
+
+ "github.com/cloudwego/eino-ext/components/embedding/ark"
+ "github.com/cloudwego/eino/schema"
+ "github.com/milvus-io/milvus/client/v2/milvusclient"
+
+ milvus2 "github.com/cloudwego/eino-ext/components/indexer/milvus2"
+)
+
+func main() {
+ // Get environment variables
+ addr := os.Getenv("MILVUS_ADDR")
+ username := os.Getenv("MILVUS_USERNAME")
+ password := os.Getenv("MILVUS_PASSWORD")
+ arkApiKey := os.Getenv("ARK_API_KEY")
+ arkModel := os.Getenv("ARK_MODEL")
+
+ ctx := context.Background()
+
+ // Create embedding model
+ emb, err := ark.NewEmbedder(ctx, &ark.EmbeddingConfig{
+ APIKey: arkApiKey,
+ Model: arkModel,
+ })
+ if err != nil {
+ log.Fatalf("Failed to create embedding: %v", err)
+ return
+ }
+
+ // Create indexer
+ indexer, err := milvus2.NewIndexer(ctx, &milvus2.IndexerConfig{
+ ClientConfig: &milvusclient.ClientConfig{
+ Address: addr,
+ Username: username,
+ Password: password,
+ },
+ Collection: "my_collection",
+
+ Vector: &milvus2.VectorConfig{
+ Dimension: 1024, // Match embedding model dimension
+ MetricType: milvus2.COSINE,
+ IndexBuilder: milvus2.NewHNSWIndexBuilder().WithM(16).WithEfConstruction(200),
+ },
+ Embedding: emb,
+ })
+ if err != nil {
+ log.Fatalf("Failed to create indexer: %v", err)
+ return
+ }
+ log.Printf("Indexer created successfully")
+
+ // Store documents
+ docs := []*schema.Document{
+ {
+ ID: "doc1",
+ Content: "Milvus is an open-source vector database",
+ MetaData: map[string]any{
+ "category": "database",
+ "year": 2021,
+ },
+ },
+ {
+ ID: "doc2",
+ Content: "EINO is a framework for building AI applications",
+ },
+ }
+ ids, err := indexer.Store(ctx, docs)
+ if err != nil {
+ log.Fatalf("Failed to store: %v", err)
+ return
+ }
+ log.Printf("Store success, ids: %v", ids)
+}
+```
+
+## Configuration Options
+
+| Field | Type | Default | Description |
Client | *milvusclient.Client | - | Pre-configured Milvus client (optional) |
ClientConfig | *milvusclient.ClientConfig | - | Client configuration (required when Client is empty) |
Collection | string | "eino_collection" | Collection name |
Vector | *VectorConfig | - | Dense vector configuration (Dimension, MetricType, field name) |
Sparse | *SparseVectorConfig | - | Sparse vector configuration (MetricType, field name) |
IndexBuilder | IndexBuilder | AutoIndexBuilder | Index type builder |
Embedding | embedding.Embedder | - | Embedder for vectorization (optional). If empty, documents must contain vectors (BYOV). |
ConsistencyLevel | ConsistencyLevel | ConsistencyLevelDefault | Consistency level (ConsistencyLevelDefaultuses Milvus default: Bounded; if not explicitly set, maintains collection-level setting) |
PartitionName | string | - | Default partition for inserting data |
EnableDynamicSchema | bool | false | Enable dynamic field support |
Functions | []*entity.Function | - | Schema function definitions (e.g., BM25) for server-side processing |
FieldParams | map[string]map[string]string | - | Field parameter configuration (e.g., enable_analyzer) |
| Field | Type | Default | Description |
Dimension | int64 | - | Vector dimension (required) |
MetricType | MetricType | L2 | Similarity metric type (L2, IP, COSINE, etc.) |
VectorField | string | "vector" | Dense vector field name |
| Field | Type | Default | Description |
VectorField | string | "sparse_vector" | Sparse vector field name |
MetricType | MetricType | BM25 | Similarity metric type |
Method | SparseMethod | SparseMethodAuto | Generation method (SparseMethodAutoor SparseMethodPrecomputed) |
| Builder | Description | Key Parameters |
NewAutoIndexBuilder() | Milvus automatically selects optimal index | - |
NewHNSWIndexBuilder() | Graph-based high-performance index | M, EfConstruction |
NewIVFFlatIndexBuilder() | Clustering-based search | NList |
NewIVFPQIndexBuilder() | Product quantization, memory efficient | NList, M, NBits |
NewIVFSQ8IndexBuilder() | Scalar quantization | NList |
NewIVFRabitQIndexBuilder() | IVF + RaBitQ binary quantization (Milvus 2.6+) | NList |
NewFlatIndexBuilder() | Brute-force exact search | - |
NewDiskANNIndexBuilder() | Disk index for large datasets | - |
NewSCANNIndexBuilder() | Fast search with high recall | NList, WithRawDataEnabled |
NewBinFlatIndexBuilder() | Brute-force search for binary vectors | - |
NewBinIVFFlatIndexBuilder() | Clustering search for binary vectors | NList |
NewGPUBruteForceIndexBuilder() | GPU-accelerated brute-force search | - |
NewGPUIVFFlatIndexBuilder() | GPU-accelerated IVF_FLAT | - |
NewGPUIVFPQIndexBuilder() | GPU-accelerated IVF_PQ | - |
NewGPUCagraIndexBuilder() | GPU-accelerated graph index (CAGRA) | IntermediateGraphDegree, GraphDegree |
| Builder | Description | Key Parameters |
NewSparseInvertedIndexBuilder() | Inverted index for sparse vectors | DropRatioBuild |
NewSparseWANDIndexBuilder() | WAND algorithm for sparse vectors | DropRatioBuild |
| Metric Type | Description |
L2 | Euclidean distance |
IP | Inner product |
COSINE | Cosine similarity |
| Metric Type | Description |
BM25 | Okapi BM25 (SparseMethodAutorequired) |
IP | Inner product (for precomputed sparse vectors) |
| Metric Type | Description |
HAMMING | Hamming distance |
JACCARD | Jaccard distance |
TANIMOTO | Tanimoto distance |
SUBSTRUCTURE | Substructure search |
SUPERSTRUCTURE | Superstructure search |
| Field | Type | Default | Description |
Client | *milvusclient.Client | - | Pre-configured Milvus client (optional) |
ClientConfig | *milvusclient.ClientConfig | - | Client configuration (required when Client is nil) |
Collection | string | "eino_collection" | Collection name |
TopK | int | 5 | Number of results to return |
VectorField | string | "vector" | Dense vector field name |
SparseVectorField | string | "sparse_vector" | Sparse vector field name |
OutputFields | []string | All fields | Fields to return in results |
SearchMode | SearchMode | - | Search strategy (required) |
Embedding | embedding.Embedder | - | Embedder for query vectorization (required) |
DocumentConverter | func | Default converter | Custom result-to-document conversion |
ConsistencyLevel | ConsistencyLevel | ConsistencyLevelDefault | Consistency level (ConsistencyLevelDefaultuses collection's level; no per-request override applied) |
Partitions | []string | - | Partitions to search |
| Metric Type | Description |
L2 | Euclidean distance |
IP | Inner product |
COSINE | Cosine similarity |
| Metric Type | Description |
BM25 | Okapi BM25 (required for BM25 search) |
IP | Inner product (for pre-computed sparse vectors) |
| Metric Type | Description |
HAMMING | Hamming distance |
JACCARD | Jaccard distance |
TANIMOTO | Tanimoto distance |
SUBSTRUCTURE | Substructure search |
SUPERSTRUCTURE | Superstructure search |
| Search Mode | Minimum OpenSearch Version | Description |
ExactMatch | 1.0+ | Standard Query DSL |
RawString | 1.0+ | Standard Query DSL |
DenseVectorSimilarity | 1.0+ | Uses script_scoreand painless vector functions |
Approximate(KNN) | 1.0+ | Basic KNN supported since 1.0. Efficient filtering (Post-filtering) requires 2.4+ (Lucene HNSW) or 2.9+ (Faiss). |
Approximate(Hybrid) | 2.10+ | Generates boolquery. Requires 2.10+ normalization-processorfor advanced score normalization (Convex Combination). Basic boolqueries work in earlier versions (1.0+). |
Approximate(RRF) | 2.19+ | Requires score-ranker-processor(2.19+) and neural-searchplugin. |
NeuralSparse(Query Text) | 2.11+ | Requires neural-searchplugin and deployed model. |
NeuralSparse(TokenWeights) | 2.11+ | Requires neural-searchplugin. |
| Search Mode | Minimum OpenSearch Version | Description |
ExactMatch | 1.0+ | Standard Query DSL |
RawString | 1.0+ | Standard Query DSL |
DenseVectorSimilarity | 1.0+ | Uses script_scoreand painless vector functions |
Approximate(KNN) | 1.0+ | Basic KNN supported since 1.0. Efficient filtering (Post-filtering) requires 2.4+ (Lucene HNSW) or 2.9+ (Faiss). |
Approximate(Hybrid) | 2.10+ | Generates boolquery. Requires 2.10+ normalization-processorfor advanced score normalization (Convex Combination). Basic boolqueries work in earlier versions (1.0+). |
Approximate(RRF) | 2.19+ | Requires score-ranker-processor(2.19+) and neural-searchplugin. |
NeuralSparse(Query Text) | 2.11+ | Requires neural-searchplugin and deployed model. |
NeuralSparse(TokenWeights) | 2.11+ | Requires neural-searchplugin. |
-Also see: [ChatTemplate - MCP](/docs/eino/ecosystem_integration/chat_template/chat_template_mcp)
+Also see:
+
+[ChatTemplate - MCP](/docs/eino/ecosystem_integration/chat_template/chat_template_mcp)
## Usage
### QuickStart
-First create an MCP client. Eino leverages the open-source SDK `mark3labs/mcp-go`:
+First create an MCP client. Eino leverages the open-source SDK [mark3labs/mcp-go](https://github.com/mark3labs/mcp-go):
```go
import "github.com/mark3labs/mcp-go/client"
-// stdio
-cli, _ := client.NewStdioMCPClient(cmd, envs, args...)
-// sse
-cli, _ := client.NewSSEMCPClient(baseURL)
+// stdio client
+cli, err := client.NewStdioMCPClient(myCommand, myEnvs, myArgs...)
+
+// sse client
+cli, err := client.NewSSEMCPClient(myBaseURL)
// sse client needs to manually start asynchronous communication
// while stdio does not require it.
-_ = cli.Start(ctx)
+err = cli.Start(ctx)
```
-Considering client reuse, the wrapper assumes the client has finished `Initialize` with the Server; you need to perform client initialization yourself:
+mcp-go also supports other methods to create a Client (such as InProcess). For more information, see: [https://mcp-go.dev/transports](https://mcp-go.dev/transports)
+
+Considering client reuse, the wrapper assumes the client has finished [Initialize](https://spec.modelcontextprotocol.io/specification/2024-11-05/basic/lifecycle/) with the Server; you need to perform client initialization yourself:
```go
import "github.com/mark3labs/mcp-go/mcp"
-init := mcp.InitializeRequest{}
-init.Params.ProtocolVersion = mcp.LATEST_PROTOCOL_VERSION
-init.Params.ClientInfo = mcp.Implementation{ Name: "example-client", Version: "1.0.0" }
-_, _ = cli.Initialize(ctx, init)
+initRequest := mcp.InitializeRequest{}
+initRequest.Params.ProtocolVersion = mcp.LATEST_PROTOCOL_VERSION
+initRequest.Params.ClientInfo = mcp.Implementation{
+ Name: "example-client",
+ Version: "1.0.0",
+}
+_, err = cli.Initialize(ctx, initRequest)
```
Then create Eino tools using the Client:
@@ -51,16 +59,18 @@ Then create Eino tools using the Client:
```go
import "github.com/cloudwego/eino-ext/components/tool/mcp"
-tools, _ := mcp.GetTools(ctx, &mcp.Config{ Cli: cli })
+tools, err := mcp.GetTools(ctx, &mcp.Config{Cli: cli})
```
Tools can be called directly:
```
-for i, mcpTool := range tools {
+for i, mcpTool := range mcpTools {
fmt.Println(i, ":")
info, err := mcpTool.Info(ctx)
- if err != nil { log.Fatal(err) }
+ if err != nil {
+ log.Fatal(err)
+ }
fmt.Println("Name:", info.Name)
fmt.Println("Desc:", info.Desc)
fmt.Println()
@@ -72,15 +82,15 @@ You can also use tools within any Eino Agent; for example with [ReAct Agent](/do
```
import (
"github.com/cloudwego/eino/flow/agent/react"
- "github.com/cloudwego/eino-ext/components/tool/mcp"
+ "github.com/cloudwego/eino-ext/components/tool/mcp"
)
llm, err := /*create a chat model*/
tools, err := mcp.GetTools(ctx, &mcp.Config{Cli: cli})
agent, err := react.NewAgent(ctx, &react.AgentConfig{
- Model: llm,
- ToolsConfig: compose.ToolsNodeConfig{Tools: tools},
+ Model: llm,
+ ToolsConfig: compose.ToolsNodeConfig{Tools: tools},
})
```
@@ -89,15 +99,14 @@ agent, err := react.NewAgent(ctx, &react.AgentConfig{
`GetTools` supports filtering by tool names to avoid unintended tools:
```go
-tools, _ := mcp.GetTools(ctx, &mcp.Config{ Cli: cli, ToolNameList: []string{"name"} })
-```
-
-Or use tools within any Eino Agent; for example with [ReAct Agent](/docs/eino/core_modules/flow_integration_components/react_agent_manual):
+import "github.com/cloudwego/eino-ext/components/tool/mcp"
-```go
-agent, _ := react.NewAgent(ctx, &react.AgentConfig{ Model: llm, ToolsConfig: compose.ToolsNodeConfig{ Tools: tools } })
+tools, err := mcp.GetTools(ctx, &mcp.Config{
+ Cli: cli,
+ ToolNameList: []string{"name"},
+})
```
## More Information
-Practice example: https://github.com/cloudwego/eino-ext/blob/main/components/tool/mcp/examples/mcp.go
+Practice example: [https://github.com/cloudwego/eino-ext/blob/main/components/tool/mcp/examples/mcp.go](https://github.com/cloudwego/eino-ext/blob/main/components/tool/mcp/examples/mcp.go)
diff --git a/content/en/docs/eino/overview/_index.md b/content/en/docs/eino/overview/_index.md
index 66b5a1064cc..191c0a55acd 100644
--- a/content/en/docs/eino/overview/_index.md
+++ b/content/en/docs/eino/overview/_index.md
@@ -1,6 +1,6 @@
---
Description: ""
-date: "2025-12-09"
+date: "2026-01-20"
lastmod: ""
tags: []
title: 'Eino: Overview'
@@ -215,7 +215,7 @@ The Eino framework consists of several parts:
- [Eino DevOps](https://github.com/cloudwego/eino-ext/tree/main/devops): Visual development and debugging.
- [EinoExamples](https://github.com/cloudwego/eino-examples): Example applications and best practices.
-See also: [Eino Architecture Overview](/docs/eino/overview/)
+See also: [Eino Architecture Overview](/docs/eino/overview/eino_arch/)
## Documentation
@@ -228,7 +228,7 @@ Complete API reference: [https://pkg.go.dev/github.com/cloudwego/eino](https://p
## Dependencies
- Go 1.18 or higher
-- Eino depends on [kin-openapi](https://github.com/getkin/kin-openapi) for OpenAPI JSONSchema. To remain compatible with Go 1.18, we pin kin-openapi at `v0.118.0`.
+- Eino depends on [kin-openapi](https://github.com/getkin/kin-openapi) for OpenAPI JSONSchema. To remain compatible with Go 1.18, we pin kin-openapi at `v0.118.0`. This dependency has been removed after V0.6.0.
## Security
diff --git a/content/en/docs/eino/overview/graph_or_agent.md b/content/en/docs/eino/overview/graph_or_agent.md
index 404386ac3cf..6f28d8aee9b 100644
--- a/content/en/docs/eino/overview/graph_or_agent.md
+++ b/content/en/docs/eino/overview/graph_or_agent.md
@@ -1,6 +1,6 @@
---
Description: ""
-date: "2025-12-09"
+date: "2026-01-20"
lastmod: ""
tags: []
title: Agent or Graph? AI Application Path Analysis
@@ -9,12 +9,16 @@ weight: 5
## Introduction: Two Coexisting AI Interaction Paradigms
-Many applications integrate different forms of AI capabilities, as shown below:
+Many application interfaces integrate different forms of AI capabilities, as shown below:
+This seemingly simple screenshot represents two forms of "AI applications":
+- The "Agent" represented by the "chat box". **Agents use LLM (Large Language Model) as the decision center, autonomously plan and can conduct multi-turn interactions**, naturally suited for handling open-ended, continuous tasks, manifesting as a "dialogue" form.
+- The "Graph" represented by "buttons" or "APIs". For example, the "Recording Summary" button above, the Graph behind it is roughly "Recording" -> "LLM understands and summarizes" -> "Save recording" - this kind of fixed process. **The core of Graph lies in the determinism of its process and the closure of tasks**, completing specific goals through predefined nodes and edges, manifesting as a "function" form. For example, video generation is an "API" form AI application:
+
```mermaid
flowchart TD
@@ -29,14 +33,14 @@ flowchart TD
D{"Task Characteristics"}
A("Agent")
G("Graph")
- A1("LLM decision center")
- A2("Multi-turn interaction")
- G1("Preset topology")
- G2("Deterministic output")
+ A1("LLM Decision Center")
+ A2("Multi-turn Interaction")
+ G1("Preset Topology")
+ G2("Deterministic Output")
S --> D
- D -->|"Open or uncertain"| A
- D -->|"Closed and deterministic"| G
+ D -->|"Open or Uncertain"| A
+ D -->|"Closed and Deterministic"| G
A --> A1
A --> A2
G --> G1
@@ -47,41 +51,43 @@ flowchart TD
class A,G,A1,A2,G1,G2 process_style
```
-This article explores the differences and connections between Agent and Graph and proposes the best integration point: encapsulate Graphs as Agent Tools. It also offers recommended usage for Eino developers: https://github.com/cloudwego/eino.
+This article explores in detail the differences and connections between Agent and Graph, two forms of AI applications, proposes that "the best integration point is to encapsulate Graph as Agent's Tool", and provides recommended usage patterns for [Eino](https://github.com/cloudwego/eino) developers.
## Core Concepts Clarification
### Basic Definitions
-- Graph: a developer-predefined flowchart with a clear topology. Nodes can be code functions, API calls, or LLMs; inputs and outputs are typically structured. The core trait is determinism — given the same input, the execution path and final result are predictable.
-- Agent: an entity centered on an LLM that can autonomously plan, decide, and execute tasks. It completes goals through dynamic interaction with the environment (Tools, users, other Agents) and exhibits uncertainty in behavior. The core trait is autonomy.
-- Tool: any external capability an Agent can call, typically a function or API encapsulating a specific capability. Tools themselves can be sync or async, stateful or stateless; they execute but do not make autonomous decisions.
-- Orchestration: the process of organizing and coordinating multiple compute units (nodes, Agents) to work together. In this article, it refers to predefined static flows via Graphs.
+- **Graph**: A flowchart **predefined** by developers with a clear topology. Its nodes can be code functions, API calls, or LLMs, and inputs and outputs are typically structured. **The core characteristic is "determinism"**, meaning given the same input, the execution path and final output are predictable.
+- **Agent**: An entity centered on LLM that can **autonomously plan, decide, and execute** tasks. It completes goals through **dynamic interaction** with the environment (Tools, users, other Agents), and its behavior is uncertain. **The core characteristic is "autonomy"**.
+- **Tool**: Any external capability that an Agent can call, typically a **function or API that encapsulates specific functionality**. Tools themselves can be synchronous or asynchronous, stateful or stateless. They are only responsible for execution and do not have autonomous decision-making capabilities.
+- **Orchestration**: The process of **organizing and coordinating multiple compute units (nodes, Agents) to work together**. In this article, it specifically refers to predefining static processes through Graphs.
### Deep Comparison
| Dimension | Agent | Graph |
| Core driver | LLM autonomy | Preset flow |
| Input | Unstructured (text/images) | Structured data |
| Deliverable | Process + result | Focused final result |
| State | Long-lived across runs | Typically stateless per run |
| Mode | Often async | Often sync |
| Core Driver | LLM Autonomous Decision | Developer Preset Process |
| Input | Unstructured natural language, images, etc. | Structured data |
| Deliverable | Process and result equally important | Focused on final result |
| State Management | Long-term, cross-execution | Single execution, stateless |
| Runtime Mode | Tends toward asynchronous | Tends toward synchronous |
| Dimension | Graph | Tool |
| Input | Structured | Structured |
| Deliverable | Focused final result | Focused final result |
| State | Stateless per run | Stateless (from LLM’s view) |
| Mode | Synchronous as a whole | Synchronous from LLM call semantics |
| Input | Structured data | Structured data |
| Deliverable | Focused on final result | Focused on final result |
| State Management | Single execution, stateless | Single execution, stateless |
| Runtime Mode | Synchronous as a whole | Tool is synchronous from LLM's perspective |
| AgenticModel | ChatModel | |
| 定位 | 以 “目标驱动的自主执行” 为核心的模型能力抽象，是 ChatModel 的增强抽象 | 一次性问答引擎 |
| 核心实体 | Message | |
| 能力 | ||
| 相关组件 |
| AgenticModel | ChatModel | |
| Temperature | 支持 | 支持 |
| Model | 支持 | 支持 |
| TopP | 支持 | 支持 |
| Tools | 支持 | 支持 |
| ToolChoice | 支持 | 支持 |
| MaxTokens | 支持 | 支持 |
| AllowedToolNames | 不支持 | 支持 |
| Stop | 部分组件实现支持 | 支持 |
| AllowedTools | 支持 | 不支持 |
| 协作方式 | 描述 | ||||
| Transfer | 直接将任务转让给另外一个 Agent，本 Agent 则执行结束后退出，不关心转让 Agent 的任务执行状态 | ||||
| ToolCall(AgentAsTool) | 将 Agent 当成 ToolCall 调用，等待 Agent 的响应，并可获取被调用Agent 的输出结果，进行下一轮处理 |
| 上下文策略 | 描述 | ||||
| 上游 Agent 全对话 | 获取本 Agent 的上游 Agent 的完整对话记录 | ||||
| 全新任务描述 | 忽略掉上游 Agent 的完整对话记录，给出一个全新的任务总结，作为子 Agent 的 AgentInput 输入 |
| 决策自主性 | 描述 |
| 自主决策 | 在 Agent 内部，基于其可选的下游 Agent， 如需协助时，自主选择下游 Agent 进行协助。 一般来说，Agent 内部是基于 LLM 进行决策，不过即使是基于预设逻辑进行选择，从 Agent 外部看依然视为自主决策 |
| 预设决策 | 事先预设好一个Agent 执行任务后的下一个 Agent。 Agent 的执行顺序是事先确定、可预测的 |
| 类型 | 描述 | 运行模式 | 协作方式 | 上下文策略 | 决策自主性 |
| SubAgents | 将用户提供的 agent 作为 父Agent，用户提供的 subAgents 列表作为 子Agents，组合而成可自主决策的 Agent，其中的 Name 和 Description 作为该 Agent 的名称标识和描述。 | | Transfer | 上游 Agent 全对话 | 自主决策 |
| Sequential | 将用户提供的 SubAgents 列表，组合成按照顺序依次执行的 Sequential Agent，其中的 Name 和 Description 作为 Sequential Agent 的名称标识和描述。Sequential Agent 执行时，将 SubAgents 列表，按照顺序依次执行，直至将所有 Agent 执行一遍后结束。 | | Transfer | 上游 Agent 全对话 | 预设决策 |
| Example | RunPath |
| |
| |
| |
| |
| |
| |
|
+
三个 Agent 均使用 ChatModelAgent 实现：
@@ -405,7 +405,7 @@ func AgentWithDeterministicTransferTo(_ context.Context, config *DeterministicTr
在 Supervisor 模式中，子 Agent 执行完毕后固定回到 Supervisor，由 Supervisor 生成下一步任务目标。此时可以使用 AgentWithDeterministicTransferTo：
-
+
```go
// github.com/cloudwego/eino/adk/prebuilt/supervisor.go
@@ -443,7 +443,7 @@ WorkflowAgent 支持以代码中预设好的流程运行 Agents。Eino ADK 提
SequentialAgent 会按照你提供的顺序，依次执行一系列 Agent：
-
+
```go
type SequentialAgentConfig struct {
@@ -459,7 +459,7 @@ func NewSequentialAgent(ctx context.Context, config *SequentialAgentConfig) (Age
LoopAgent 基于 SequentialAgent 实现，在 SequentialAgent 运行完成后，再次从头运行：
-
+
```go
type LoopAgentConfig struct {
@@ -477,7 +477,7 @@ func NewLoopAgent(ctx context.Context, config *LoopAgentConfig) (Agent, error)
ParallelAgent 会并发运行若干 Agent：
-
+
```go
type ParallelAgentConfig struct {
@@ -498,3 +498,21 @@ func NewAgentTool(_ context.Context, agent Agent, options ...AgentToolOption) to
```
转换为 Tool 后，Agent 可以被支持 function calling 的 ChatModel 调用，也可以被所有基于 LLM 驱动的 Agent 调用，调用方式取决于 Agent 实现。
+
+消息历史隔离：作为 Tool 的 Agent，不会继承上级 Agent 的消息历史（History）。
+
+SessionValues 共享：但是，会共享上级 Agent 的 SessionValues，即读写同一个 KV map。
+
+内部事件透出：作为 Tool 的 Agent 也是 Agent，会产生 AgentEvent。这些内部的 AgentEvent，默认情况下，不会通过 `Runner` 返回的 `AsyncIterator` 透出。在部分业务场景中，如果需要像用户透出内部 AgentTool 的 AgentEvent，需要在 AgentTool 的上级 `ChatModelAgent` 的 `ToolsConfig` 中增加配置，开启内部事件透出：
+
+```go
+// from adk/chatmodel.go
+
+**type **ToolsConfig **struct **{
+ // other configurations...
+
+ _// EmitInternalEvents indicates whether internal events from agentTool should be emitted_
+_ // to the parent generator via a tool option injection at run-time._
+_ _EmitInternalEvents bool
+}
+```
diff --git a/content/zh/docs/eino/core_modules/eino_adk/agent_hitl.md b/content/zh/docs/eino/core_modules/eino_adk/agent_hitl.md
index 33d2ada48c6..d2208738159 100644
--- a/content/zh/docs/eino/core_modules/eino_adk/agent_hitl.md
+++ b/content/zh/docs/eino/core_modules/eino_adk/agent_hitl.md
@@ -1,6 +1,6 @@
---
Description: ""
-date: "2025-12-11"
+date: "2026-01-20"
lastmod: ""
tags: []
title: Eino human-in-the-loop框架：技术架构指南
@@ -290,6 +290,16 @@ answer: The ticket for Martin to Beijing on 2025-12-01 has been successfully boo
to ask!
```
+### 更多样例
+
+- 审查与编辑模式：允许在执行前进行人工审查和原地编辑工具调用参数。[https://github.com/cloudwego/eino-examples/tree/main/adk/human-in-the-loop/2_review-and-edit](https://github.com/cloudwego/eino-examples/tree/main/adk/human-in-the-loop/2_review-and-edit)
+- 反馈循环模式：迭代优化模式，其中 agent 生成内容，人类提供定性反馈以进行改进。[https://github.com/cloudwego/eino-examples/tree/main/adk/human-in-the-loop/3_feedback-loop](https://github.com/cloudwego/eino-examples/tree/main/adk/human-in-the-loop/3_feedback-loop)
+- 追问模式：主动模式，其中 agent 识别出不充分的工具输出并请求澄清或下一步行动。[https://github.com/cloudwego/eino-examples/tree/main/adk/human-in-the-loop/4_follow-up](https://github.com/cloudwego/eino-examples/tree/main/adk/human-in-the-loop/4_follow-up)
+- 在 supervisor 架构内中断：[https://github.com/cloudwego/eino-examples/tree/main/adk/human-in-the-loop/5_supervisor](https://github.com/cloudwego/eino-examples/tree/main/adk/human-in-the-loop/5_supervisor)
+- 在 plan-execute-replan 架构内中断：[https://github.com/cloudwego/eino-examples/tree/main/adk/human-in-the-loop/6_plan-execute-replan](https://github.com/cloudwego/eino-examples/tree/main/adk/human-in-the-loop/6_plan-execute-replan)
+- 在 deep-agents 架构内中断：[https://github.com/cloudwego/eino-examples/tree/main/adk/human-in-the-loop/7_deep-agents](https://github.com/cloudwego/eino-examples/tree/main/adk/human-in-the-loop/7_deep-agents)
+- 在 supervisor 的一个子 agent 是 plan-execute-replan 的情况下中断：[https://github.com/cloudwego/eino-examples/tree/main/adk/human-in-the-loop/8_supervisor-plan-execute](https://github.com/cloudwego/eino-examples/tree/main/adk/human-in-the-loop/8_supervisor-plan-execute)
+
## 架构概述
以下流程图说明了高层次的中断/恢复流程：
@@ -1173,25 +1183,3 @@ func (a *myLegacyAgent) Resume(ctx context.Context, info *adk.ResumeInfo) *adk.A
- **灵活的状态管理**: 对于静态图中断，你可以选择通过 `.Info` 字段进行现代、直接的状态修改，或者继续使用旧的 `WithStateModifier` 选项。
这种向后兼容性模型确保了现有用户的平滑过渡，同时为采用新的 human-in-the-loop 功能提供了清晰的路径。
-
-## 实现示例
-
-有关 human-in-the-loop 模式的完整、可工作的示例，请参阅 [eino-examples repository](https://github.com/cloudwego/eino-examples/pull/125)。该仓库包含四个作为独立示例实现的典型模式：
-
-### 1. 审批模式
-
-在关键工具调用之前的简单、显式批准。非常适合不可逆操作，如数据库修改或金融交易。
-
-### 2. 审查与编辑模式
-
-高级模式，允许在执行前进行人工审查和原地编辑工具调用参数。非常适合纠正误解。
-
-### 3. 反馈循环模式
-
-迭代优化模式，其中 agent 生成内容，人类提供定性反馈以进行改进。
-
-### 4. 追问模式
-
-主动模式，其中 agent 识别出不充分的工具输出并请求澄清或下一步行动。
-
-这些示例演示了中断/恢复机制的实际用法，并附有可重用的工具包装器和详细文档。
diff --git a/content/zh/docs/eino/core_modules/eino_adk/agent_implementation/chat_model.md b/content/zh/docs/eino/core_modules/eino_adk/agent_implementation/chat_model.md
index 443e675cffa..3fdd2b396ad 100644
--- a/content/zh/docs/eino/core_modules/eino_adk/agent_implementation/chat_model.md
+++ b/content/zh/docs/eino/core_modules/eino_adk/agent_implementation/chat_model.md
@@ -1,6 +1,6 @@
---
Description: ""
-date: "2025-12-09"
+date: "2026-01-20"
lastmod: ""
tags: []
title: 'Eino ADK: ChatModelAgent'
@@ -41,6 +41,10 @@ type ToolsConfig struct {
// Names of the tools that will make agent return directly when the tool is called.
// When multiple tools are called and more than one tool is in the return directly list, only the first one will be returned.
ReturnDirectly map[string]bool
+
+ // EmitInternalEvents indicates whether internal events from agentTool should be emitted
+ // to the parent generator via a tool option injection at run-time.
+ EmitInternalEvents bool
}
```
@@ -48,6 +52,9 @@ ToolsConfig 复用了 Eino Graph ToolsNodeConfig，详细参考：[Eino: ToolsNo
## ChatModelAgent 配置字段
+> 💡
+> 注意：GenModelInput 默认情况下，会通过 adk.GetSessionValues() 并以 F-String 的格式渲染 Instruction，如需关闭此行为，可定制 GenModelInput 方法。
+
```go
type ChatModelAgentConfig struct {
// Name of the agent. Better be unique across all agents.
@@ -83,6 +90,12 @@ type ChatModelAgentConfig struct {
// The agent will terminate with an error if this limit is exceeded.
// Optional. Defaults to 20.
MaxIterations int
+
+ // ModelRetryConfig configures retry behavior for the ChatModel.
+ // When set, the agent will automatically retry failed ChatModel calls
+ // based on the configured policy.
+ // Optional. If nil, no retry will be performed.
+ ModelRetryConfig *ModelRetryConfig
}
type ToolsConfig struct {
@@ -91,6 +104,10 @@ type ToolsConfig struct {
// Names of the tools that will make agent return directly when the tool is called.
// When multiple tools are called and more than one tool is in the return directly list, only the first one will be returned.
ReturnDirectly map[string]bool
+
+ // EmitInternalEvents indicates whether internal events from agentTool should be emitted
+ // to the parent generator via a tool option injection at run-time.
+ EmitInternalEvents bool
}
type GenModelInput func(ctx context.Context, instruction string, input *AgentInput) ([]Message, error)
@@ -103,9 +120,14 @@ type GenModelInput func(ctx context.Context, instruction string, input *AgentInp
- `ToolsConfig`：工具配置
- ToolsConfig 复用了 Eino Graph ToolsNodeConfig，详细参考：[Eino: ToolsNode&Tool 使用说明](/zh/docs/eino/core_modules/components/tools_node_guide)。
- ReturnDirectly：当 ChatModelAgent 调用配置在 ReturnDirectly 中的 Tool 后，将携带结果立刻退出，不会按照 react 模式返回 ChatModel。如果命中了多个 Tool，只有首个 Tool 会返回。Map key 为 Tool 名称。
+ - EmitInternalEvents：当通过 adk.AgentTool() 将一个 Agent 通过 ToolCall 的形式当成 SubAgent 时，默认情况下，这个 SubAgent 不会发送 AgentEvent，只将最终结果作为 ToolResult 返回。
- `GenModelInput`：Agent 被调用时会使用该方法将 `Instruction` 和 `AgentInput` 转换为调用 ChatModel 的 Messages。Agent 提供了默认的 GenModelInput 方法：
1. 将 `Instruction` 作为 `System Message` 加到 `AgentInput.Messages` 前
2. 将 `SessionValues` 为 variables 渲染到步骤 1 的 message list 中
+
+> 💡
+> 默认的 `GenModelInput` 使用 pyfmt 渲染，message list 中的文本会被作为 pyfmt 模板，这意味着文本中的 '{' 与 '}' 都会被视为关键字，如果希望直接输入这两个字符，需要进行转义 '{{'、'}}'
+
- `OutputKey`：配置后，ChatModelAgent 运行产生的最后一条 Message 将会以 `OutputKey` 为 key 设置到 `SessionValues` 中
- `MaxIterations`：react 模式下 ChatModel 最大生成次数，超过时 Agent 会报错退出，默认值为 20
- `Exit`：Exit 是一个特殊的 Tool，当模型调用这个工具并执行后，ChatModelAgent 将直接退出，效果与 `ToolsConfig.ReturnDirectly` 类似。ADK 提供了一个默认 ExitTool 实现供用户使用：
@@ -137,6 +159,47 @@ func (et ExitTool) InvokableRun(ctx context.Context, argumentsInJSON string, _ .
}
```
+- `ModelRetryConfig`: 配置后，ChatModel 请求过程中发生的各种错误（包括直接返回错误、流式响应过程中发生错误等），都会按照配置的策略选择是否以及何时进行重试。如果是流式响应过程中发生错误，则这一次流式响应依然会第一时间通过 AgentEvent 的形式返回出去。如果这次流式响应过程中的错误，按照配置的策略，会进行重试，则消费 AgentEvent 中的 message stream，会得到 `WillRetryError`。用户可以处理这个 error，做对应的上屏展示等处理，示例如下：
+
+```go
+iterator := agent.Run(ctx, input)
+for {
+ event, ok := iterator.Next()
+ if !ok {
+ break
+ }
+
+ if event.Err != nil {
+ handleFinalError(event.Err)
+ break
+ }
+
+ // Process streaming output
+ if event.Output != nil && event.Output.MessageOutput.IsStreaming {
+ stream := event.Output.MessageOutput.MessageStream
+ for {
+ msg, err := stream.Recv()
+ if err == io.EOF {
+ break // Stream completed successfully
+ }
+ if err != nil {
+ // Check if this error will be retried (more streams coming)
+ var willRetry *adk.WillRetryError
+ if errors.As(err, &willRetry) {
+ log.Printf("Attempt %d failed, retrying...", willRetry.RetryAttempt)
+ break // Wait for next event with new stream
+ }
+ // Original error - won't retry, agent will stop and the next AgentEvent probably will be an error
+ log.Printf("Final error (no retry): %v", err)
+ break
+ }
+ // Display chunk to user
+ displayChunk(msg)
+ }
+ }
+}
+```
+
## ChatModelAgent Transfer
`ChatModelAgent` 支持将其他 Agent 的元信息转为自身的 Tool ，经由 ChatModel 判断实现动态 Transfer：
@@ -468,6 +531,8 @@ func NewBookRecommendAgent() adk.Agent {
ToolsNodeConfig: compose.ToolsNodeConfig{
Tools: []tool.BaseTool{NewBookRecommender(), NewAskForClarificationTool()},
},
+ // Tool 内部通过 AgentTool() 调用 SubAgent 时，是否将这个 SubAgent 的 AgentEvent 输出
+ EmitInternalEvents: true,
},
})
// xxx
diff --git a/content/zh/docs/eino/core_modules/eino_adk/agent_implementation/deepagents.md b/content/zh/docs/eino/core_modules/eino_adk/agent_implementation/deepagents.md
index 4e9ae338d63..070a2003cfa 100644
--- a/content/zh/docs/eino/core_modules/eino_adk/agent_implementation/deepagents.md
+++ b/content/zh/docs/eino/core_modules/eino_adk/agent_implementation/deepagents.md
@@ -1,6 +1,6 @@
---
Description: ""
-date: "2025-12-01"
+date: "2026-01-20"
lastmod: ""
tags: []
title: 'Eino ADK MultiAgent: DeepAgents'
diff --git a/content/zh/docs/eino/core_modules/eino_adk/agent_implementation/supervisor.md b/content/zh/docs/eino/core_modules/eino_adk/agent_implementation/supervisor.md
index dcd4f993b19..798630e28e5 100644
--- a/content/zh/docs/eino/core_modules/eino_adk/agent_implementation/supervisor.md
+++ b/content/zh/docs/eino/core_modules/eino_adk/agent_implementation/supervisor.md
@@ -1,6 +1,6 @@
---
Description: ""
-date: "2025-12-03"
+date: "2026-01-20"
lastmod: ""
tags: []
title: 'Eino ADK MultiAgent: Supervisor Agent'
diff --git a/content/zh/docs/eino/core_modules/eino_adk/agent_implementation/workflow.md b/content/zh/docs/eino/core_modules/eino_adk/agent_implementation/workflow.md
index 258102bcd13..fe881e924f0 100644
--- a/content/zh/docs/eino/core_modules/eino_adk/agent_implementation/workflow.md
+++ b/content/zh/docs/eino/core_modules/eino_adk/agent_implementation/workflow.md
@@ -1,6 +1,6 @@
---
Description: ""
-date: "2025-12-03"
+date: "2026-01-20"
lastmod: ""
tags: []
title: 'Eino ADK: Workflow Agents'
diff --git a/content/zh/docs/eino/core_modules/eino_adk/agent_interface.md b/content/zh/docs/eino/core_modules/eino_adk/agent_interface.md
index ee3908519be..cccf8e7432a 100644
--- a/content/zh/docs/eino/core_modules/eino_adk/agent_interface.md
+++ b/content/zh/docs/eino/core_modules/eino_adk/agent_interface.md
@@ -1,6 +1,6 @@
---
Description: ""
-date: "2025-12-09"
+date: "2026-01-20"
lastmod: ""
tags: []
title: 'Eino ADK: Agent 抽象'
diff --git a/content/zh/docs/eino/core_modules/eino_adk/agent_preview.md b/content/zh/docs/eino/core_modules/eino_adk/agent_preview.md
index 79d905bae37..dc86485ed42 100644
--- a/content/zh/docs/eino/core_modules/eino_adk/agent_preview.md
+++ b/content/zh/docs/eino/core_modules/eino_adk/agent_preview.md
@@ -1,6 +1,6 @@
---
Description: ""
-date: "2025-12-11"
+date: "2026-01-20"
lastmod: ""
tags: []
title: 'Eino ADK: 概述'
@@ -145,18 +145,22 @@ AgentRunner 是 Agent 的执行器，为 Agent 运行所需要的拓展功能加
只有通过 Runner 执行 agent 时，才可以使用 ADK 的如下功能：
- Interrupt & Resume
-- 切面机制（当前版本尚未支持）
+- 切面机制（1226 测试版本支持，正式发布前不保证 api 兼容，详见 [[WIP] Eino ADK: 切面](https://bytedance.larkoffice.com/wiki/JM9mwNJuGij69UkrvIpcHwXdnhf)。另基于 adk 切面的 fornax trace 能力：[ADK Middleware - Fornax Trace [WIP]](https://bytedance.larkoffice.com/docx/TemBdR67zoESssx1SiOczXiJnQd)）
+
+ ```go
+ go get github.com/cloudwego/eino@v0.8.0-alpha.1
+ ```
- Context 环境的预处理
-```go
-type RunnerConfig struct {
- Agent Agent
- EnableStreaming bool
+ ```go
+ type RunnerConfig struct {
+ Agent Agent
+ EnableStreaming bool
- CheckPointStore compose.CheckPointStore
-}
+ CheckPointStore compose.CheckPointStore
+ }
-func NewRunner(_ context.Context, conf RunnerConfig) *Runner {
- // omit code
-}
-```
+ func NewRunner(_ context.Context, conf RunnerConfig) *Runner {
+ // omit code
+ }
+ ```
diff --git a/content/zh/docs/eino/core_modules/eino_adk/agent_quickstart.md b/content/zh/docs/eino/core_modules/eino_adk/agent_quickstart.md
index b38e60bdbb6..d9939b14d15 100644
--- a/content/zh/docs/eino/core_modules/eino_adk/agent_quickstart.md
+++ b/content/zh/docs/eino/core_modules/eino_adk/agent_quickstart.md
@@ -1,6 +1,6 @@
---
Description: ""
-date: "2025-12-11"
+date: "2026-01-20"
lastmod: ""
tags: []
title: 'Eino ADK: Quickstart'
diff --git a/content/zh/docs/eino/core_modules/flow_integration_components/multi_agent_hosting.md b/content/zh/docs/eino/core_modules/flow_integration_components/multi_agent_hosting.md
index 356878b8a06..d559dd25d9d 100644
--- a/content/zh/docs/eino/core_modules/flow_integration_components/multi_agent_hosting.md
+++ b/content/zh/docs/eino/core_modules/flow_integration_components/multi_agent_hosting.md
@@ -1,6 +1,6 @@
---
Description: ""
-date: "2025-12-09"
+date: "2026-01-20"
lastmod: ""
tags: []
title: 'Eino Tutorial: Host Multi-Agent'
diff --git a/content/zh/docs/eino/core_modules/flow_integration_components/react_agent_manual.md b/content/zh/docs/eino/core_modules/flow_integration_components/react_agent_manual.md
index 12f98f5c1c4..1d67ac886e1 100644
--- a/content/zh/docs/eino/core_modules/flow_integration_components/react_agent_manual.md
+++ b/content/zh/docs/eino/core_modules/flow_integration_components/react_agent_manual.md
@@ -1,6 +1,6 @@
---
Description: ""
-date: "2025-12-09"
+date: "2026-01-20"
lastmod: ""
tags: []
title: 'Eino: ReAct Agent 使用手册'
diff --git a/content/zh/docs/eino/ecosystem_integration/_index.md b/content/zh/docs/eino/ecosystem_integration/_index.md
index 8b558e5ef61..de87ef9e28e 100644
--- a/content/zh/docs/eino/ecosystem_integration/_index.md
+++ b/content/zh/docs/eino/ecosystem_integration/_index.md
@@ -1,6 +1,6 @@
---
Description: ""
-date: "2025-07-21"
+date: "2026-01-20"
lastmod: ""
tags: []
title: 'Eino: 组件集成'
@@ -42,10 +42,24 @@ weight: 5
### Indexer
- volc vikingdb: [Indexer - volc VikingDB](/zh/docs/eino/ecosystem_integration/indexer/indexer_volc_vikingdb)
+- Milvus 2.5+: [Indexer - Milvus 2 (v2.5+)](/zh/docs/eino/ecosystem_integration/indexer/indexer_milvusv2)
+- Milvus 2.4: [Indexer - Milvus](/zh/docs/eino/ecosystem_integration/indexer/indexer_milvus)
+- OpenSearch 3: [Indexer - OpenSearch 3](/zh/docs/eino/ecosystem_integration/indexer/indexer_opensearch3)
+- OpenSearch 2: [Indexer - OpenSearch 2](/zh/docs/eino/ecosystem_integration/indexer/indexer_opensearch2)
+- ElasticSearch 9: [Indexer - Elasticsearch 9](/zh/docs/eino/ecosystem_integration/indexer/indexer_elasticsearch9)
+- Elasticsearch 8: [Indexer - ES8](/zh/docs/eino/ecosystem_integration/indexer/indexer_es8)
+- ElasticSearch 7: [Indexer - Elasticsearch 7 ](/zh/docs/eino/ecosystem_integration/indexer/indexer_elasticsearch7)
### Retriever
- volc vikingdb: [Retriever - volc VikingDB](/zh/docs/eino/ecosystem_integration/retriever/retriever_volc_vikingdb)
+- Milvus 2.5+: [Retriever - Milvus 2 (v2.5+) ](/zh/docs/eino/ecosystem_integration/retriever/retriever_milvusv2)
+- Milvus 2.4: [Retriever - Milvus](/zh/docs/eino/ecosystem_integration/retriever/retriever_milvus)
+- OpenSearch 3: [Retriever - OpenSearch 3](/zh/docs/eino/ecosystem_integration/retriever/retriever_opensearch3)
+- OpenSearch 2: [Retriever - OpenSearch 2](/zh/docs/eino/ecosystem_integration/retriever/retriever_opensearch2)
+- ElasticSearch 9: [Retriever - Elasticsearch 9](/zh/docs/eino/ecosystem_integration/retriever/retriever_elasticsearch9)
+- ElasticSearch 8: [Retriever - ES8](/zh/docs/eino/ecosystem_integration/retriever/retriever_es8)
+- ElasticSearch 7: [Retriever - ES 7](/zh/docs/eino/ecosystem_integration/retriever/retriever_elasticsearch7)
### Tools
diff --git a/content/zh/docs/eino/ecosystem_integration/callbacks/callback_cozeloop.md b/content/zh/docs/eino/ecosystem_integration/callbacks/callback_cozeloop.md
index 30241c5e7e0..381a2ffe9b8 100644
--- a/content/zh/docs/eino/ecosystem_integration/callbacks/callback_cozeloop.md
+++ b/content/zh/docs/eino/ecosystem_integration/callbacks/callback_cozeloop.md
@@ -1,6 +1,6 @@
---
Description: ""
-date: "2025-12-11"
+date: "2026-01-20"
lastmod: ""
tags: []
title: Callback - CozeLoop
diff --git a/content/zh/docs/eino/ecosystem_integration/chat_model/_index.md b/content/zh/docs/eino/ecosystem_integration/chat_model/_index.md
index e4ea8e1730d..4818a55eba8 100644
--- a/content/zh/docs/eino/ecosystem_integration/chat_model/_index.md
+++ b/content/zh/docs/eino/ecosystem_integration/chat_model/_index.md
@@ -1,6 +1,6 @@
---
Description: ""
-date: "2025-07-21"
+date: "2026-01-20"
lastmod: ""
tags: []
title: ChatModel
diff --git a/content/zh/docs/eino/ecosystem_integration/chat_model/chat_model_ark.md b/content/zh/docs/eino/ecosystem_integration/chat_model/chat_model_ark.md
index edfd8034f7a..f6b9bda3309 100644
--- a/content/zh/docs/eino/ecosystem_integration/chat_model/chat_model_ark.md
+++ b/content/zh/docs/eino/ecosystem_integration/chat_model/chat_model_ark.md
@@ -1,6 +1,6 @@
---
Description: ""
-date: "2025-12-03"
+date: "2026-01-20"
lastmod: ""
tags: []
title: ChatModel - ARK
@@ -1011,6 +1011,8 @@ func ptrOf[T any](v T) *T {
}
```
+当针对已经被缓存的消息，不想使用缓存时，可调用 `InvalidateMessageCaches(messages []*schema.Message) error` 清楚消息中的缓存标记。这样 ARK SDK 在构造 Responses API 的请求时，便无法根据缓存标记找到对应的 ResponseID，从而不再给 Responses API 中的 PreviousResponseID 赋值
+
### ContextAPI Session 缓存
```go
@@ -1218,6 +1220,8 @@ func main() {
}
```
+当针对已经被缓存的消息，不想使用缓存时，可调用 `InvalidateMessageCaches(messages []*schema.Message) error` 清楚消息中的缓存标记。这样 ARK SDK 在构造 Responses API 的请求时，便无法根据缓存标记找到对应的 ResponseID，从而不再给 Responses API 中的 PreviousResponseID 赋值
+
## **使用方式**
### ChatModel
diff --git a/content/zh/docs/eino/ecosystem_integration/chat_model/chat_model_deepseek.md b/content/zh/docs/eino/ecosystem_integration/chat_model/chat_model_deepseek.md
index 7f127b92280..d408ecddfe2 100644
--- a/content/zh/docs/eino/ecosystem_integration/chat_model/chat_model_deepseek.md
+++ b/content/zh/docs/eino/ecosystem_integration/chat_model/chat_model_deepseek.md
@@ -1,6 +1,6 @@
---
Description: ""
-date: "2025-12-11"
+date: "2026-01-20"
lastmod: ""
tags: []
title: ChatModel - DeepSeek
diff --git a/content/zh/docs/eino/ecosystem_integration/chat_model/chat_model_openai.md b/content/zh/docs/eino/ecosystem_integration/chat_model/chat_model_openai.md
index 8d06aecd191..aad68498cb5 100644
--- a/content/zh/docs/eino/ecosystem_integration/chat_model/chat_model_openai.md
+++ b/content/zh/docs/eino/ecosystem_integration/chat_model/chat_model_openai.md
@@ -289,7 +289,6 @@ func main() {
}
multiModalMsg := &schema.Message{
- Role: schema.User,
UserInputMultiContent: []schema.MessageInputPart{
{
Type: schema.ChatMessagePartTypeText,
diff --git a/content/zh/docs/eino/ecosystem_integration/document/loader_amazon_s3.md b/content/zh/docs/eino/ecosystem_integration/document/loader_amazon_s3.md
index 7817230955b..f3c0e790adb 100644
--- a/content/zh/docs/eino/ecosystem_integration/document/loader_amazon_s3.md
+++ b/content/zh/docs/eino/ecosystem_integration/document/loader_amazon_s3.md
@@ -1,6 +1,6 @@
---
Description: ""
-date: "2025-01-20"
+date: "2026-01-20"
lastmod: ""
tags: []
title: Loader - amazon s3
diff --git a/content/zh/docs/eino/ecosystem_integration/embedding/_index.md b/content/zh/docs/eino/ecosystem_integration/embedding/_index.md
index cf069862dba..b183f6abcdf 100644
--- a/content/zh/docs/eino/ecosystem_integration/embedding/_index.md
+++ b/content/zh/docs/eino/ecosystem_integration/embedding/_index.md
@@ -1,6 +1,6 @@
---
Description: ""
-date: "2025-01-06"
+date: "2026-01-20"
lastmod: ""
tags: []
title: Embedding
diff --git a/content/zh/docs/eino/ecosystem_integration/embedding/embedding_ark.md b/content/zh/docs/eino/ecosystem_integration/embedding/embedding_ark.md
index 22396aaa8d4..6e15e39bf78 100644
--- a/content/zh/docs/eino/ecosystem_integration/embedding/embedding_ark.md
+++ b/content/zh/docs/eino/ecosystem_integration/embedding/embedding_ark.md
@@ -1,6 +1,6 @@
---
Description: ""
-date: "2025-01-20"
+date: "2026-01-20"
lastmod: ""
tags: []
title: Embedding - ARK
diff --git a/content/zh/docs/eino/ecosystem_integration/embedding/embedding_dashscope.md b/content/zh/docs/eino/ecosystem_integration/embedding/embedding_dashscope.md
index a92f2b6c1e5..76ac8d76419 100644
--- a/content/zh/docs/eino/ecosystem_integration/embedding/embedding_dashscope.md
+++ b/content/zh/docs/eino/ecosystem_integration/embedding/embedding_dashscope.md
@@ -1,6 +1,6 @@
---
Description: ""
-date: "2025-12-11"
+date: "2026-01-20"
lastmod: ""
tags: []
title: Embedding - dashscope
diff --git a/content/zh/docs/eino/ecosystem_integration/embedding/embedding_ollama.md b/content/zh/docs/eino/ecosystem_integration/embedding/embedding_ollama.md
index 3d041a3e2bb..b472675f030 100644
--- a/content/zh/docs/eino/ecosystem_integration/embedding/embedding_ollama.md
+++ b/content/zh/docs/eino/ecosystem_integration/embedding/embedding_ollama.md
@@ -1,6 +1,6 @@
---
Description: ""
-date: "2025-12-11"
+date: "2026-01-20"
lastmod: ""
tags: []
title: Embedding - Ollama
diff --git a/content/zh/docs/eino/ecosystem_integration/embedding/embedding_openai.md b/content/zh/docs/eino/ecosystem_integration/embedding/embedding_openai.md
index e8883842f13..dab1796ed34 100644
--- a/content/zh/docs/eino/ecosystem_integration/embedding/embedding_openai.md
+++ b/content/zh/docs/eino/ecosystem_integration/embedding/embedding_openai.md
@@ -1,6 +1,6 @@
---
Description: ""
-date: "2025-01-20"
+date: "2026-01-20"
lastmod: ""
tags: []
title: Embedding - OpenAI
diff --git a/content/zh/docs/eino/ecosystem_integration/embedding/embedding_qianfan.md b/content/zh/docs/eino/ecosystem_integration/embedding/embedding_qianfan.md
index 166acdce774..dde774b3d11 100644
--- a/content/zh/docs/eino/ecosystem_integration/embedding/embedding_qianfan.md
+++ b/content/zh/docs/eino/ecosystem_integration/embedding/embedding_qianfan.md
@@ -1,6 +1,6 @@
---
Description: ""
-date: "2025-12-11"
+date: "2026-01-20"
lastmod: ""
tags: []
title: Embedding - Qianfan
diff --git a/content/zh/docs/eino/ecosystem_integration/embedding/embedding_tencentcloud.md b/content/zh/docs/eino/ecosystem_integration/embedding/embedding_tencentcloud.md
index 33ea6777742..171c9f9ba9c 100644
--- a/content/zh/docs/eino/ecosystem_integration/embedding/embedding_tencentcloud.md
+++ b/content/zh/docs/eino/ecosystem_integration/embedding/embedding_tencentcloud.md
@@ -1,6 +1,6 @@
---
Description: ""
-date: "2025-12-11"
+date: "2026-01-20"
lastmod: ""
tags: []
title: Embedding - TencentCloud
diff --git a/content/zh/docs/eino/ecosystem_integration/indexer/_index.md b/content/zh/docs/eino/ecosystem_integration/indexer/_index.md
index b37e9ddda24..41fa5c5ec08 100644
--- a/content/zh/docs/eino/ecosystem_integration/indexer/_index.md
+++ b/content/zh/docs/eino/ecosystem_integration/indexer/_index.md
@@ -1,6 +1,6 @@
---
Description: ""
-date: "2025-01-20"
+date: "2026-01-20"
lastmod: ""
tags: []
title: Indexer
diff --git a/content/zh/docs/eino/ecosystem_integration/indexer/indexer_elasticsearch7.md b/content/zh/docs/eino/ecosystem_integration/indexer/indexer_elasticsearch7.md
new file mode 100644
index 00000000000..9311ff525b9
--- /dev/null
+++ b/content/zh/docs/eino/ecosystem_integration/indexer/indexer_elasticsearch7.md
@@ -0,0 +1,155 @@
+---
+Description: ""
+date: "2026-01-20"
+lastmod: ""
+tags: []
+title: 'Indexer - Elasticsearch 7 '
+weight: 0
+---
+
+> **云搜索服务介绍**
+>
+> 云搜索服务（Cloud Search）是一个全托管、一站式信息检索与分析平台，提供了 ElasticSearch 和 OpenSearch 引擎，支持全文检索、向量检索、混合搜索及时空检索等多种核心能力。
+
+[Eino](https://github.com/cloudwego/eino) 的 Elasticsearch 7.x 索引器实现，实现了 `Indexer` 接口。该组件可以与 Eino 的文档索引系统无缝集成，提供强大的向量存储和检索能力。
+
+## 功能特性
+
+- 实现了 `github.com/cloudwego/eino/components/indexer.Indexer`
+- 易于集成 Eino 索引系统
+- 可配置 Elasticsearch 参数
+- 支持向量相似度搜索
+- 支持批量索引操作
+- 支持自定义字段映射
+- 灵活的文档向量化支持
+
+## 安装
+
+```bash
+go get github.com/cloudwego/eino-ext/components/indexer/es7@latest
+```
+
+## 快速开始
+
+以下是一个使用索引器的简单示例：
+
+```go
+import (
+ "context"
+ "fmt"
+ "log"
+ "os"
+
+ "github.com/cloudwego/eino/components/embedding"
+ "github.com/cloudwego/eino/schema"
+ elasticsearch "github.com/elastic/go-elasticsearch/v7"
+
+ "github.com/cloudwego/eino-ext/components/embedding/ark"
+ "github.com/cloudwego/eino-ext/components/indexer/es7"
+)
+
+const (
+ indexName = "eino_example"
+ fieldContent = "content"
+ fieldContentVector = "content_vector"
+ fieldExtraLocation = "location"
+ docExtraLocation = "location"
+)
+
+func main() {
+ ctx := context.Background()
+
+ // ES 支持多种连接方式
+ username := os.Getenv("ES_USERNAME")
+ password := os.Getenv("ES_PASSWORD")
+
+ client, _ := elasticsearch.NewClient(elasticsearch.Config{
+ Addresses: []string{"http://localhost:9200"},
+ Username: username,
+ Password: password,
+ })
+
+ // 使用 Volcengine ARK 创建 embedding 组件
+ emb, _ := ark.NewEmbedder(ctx, &ark.EmbeddingConfig{
+ APIKey: os.Getenv("ARK_API_KEY"),
+ Region: os.Getenv("ARK_REGION"),
+ Model: os.Getenv("ARK_MODEL"),
+ })
+
+ // 加载文档
+ docs := []*schema.Document{
+ {
+ ID: "1",
+ Content: "Eiffel Tower: Located in Paris, France.",
+ MetaData: map[string]any{
+ docExtraLocation: "France",
+ },
+ },
+ {
+ ID: "2",
+ Content: "The Great Wall: Located in China.",
+ MetaData: map[string]any{
+ docExtraLocation: "China",
+ },
+ },
+ }
+
+ // 创建 ES 索引器组件
+ indexer, _ := es7.NewIndexer(ctx, &es7.IndexerConfig{
+ Client: client,
+ Index: indexName,
+ BatchSize: 10,
+ DocumentToFields: func(ctx context.Context, doc *schema.Document) (field2Value map[string]es7.FieldValue, err error) {
+ return map[string]es7.FieldValue{
+ fieldContent: {
+ Value: doc.Content,
+ EmbedKey: fieldContentVector, // 对文档内容进行向量化并保存到 "content_vector" 字段
+ },
+ fieldExtraLocation: {
+ Value: doc.MetaData[docExtraLocation],
+ },
+ }, nil
+ },
+ Embedding: emb,
+ })
+
+ ids, err := indexer.Store(ctx, docs)
+ if err != nil {
+ fmt.Printf("index error: %v\n", err)
+ return
+ }
+ fmt.Println("indexed ids:", ids)
+}
+```
+
+## 配置
+
+可以使用 `IndexerConfig` 结构体配置索引器：
+
+```go
+type IndexerConfig struct {
+ Client *elasticsearch.Client // 必填：Elasticsearch 客户端实例
+ Index string // 必填：存储文档的索引名称
+ BatchSize int // 选填：最大文本嵌入批次大小（默认：5）
+
+ // 必填：将 Document 字段映射到 Elasticsearch 字段的函数
+ DocumentToFields func(ctx context.Context, doc *schema.Document) (map[string]FieldValue, error)
+
+ // 选填：仅在需要向量化时必填
+ Embedding embedding.Embedder
+}
+
+// FieldValue 定义了字段应如何存储和向量化
+type FieldValue struct {
+ Value any // 要存储的原始值
+ EmbedKey string // 如果设置，Value 将被向量化并保存
+ Stringify func(val any) (string, error) // 选填：自定义字符串转换
+}
+```
+
+## 获取帮助
+
+如果有任何问题 或者任何功能建议，欢迎进这个群 oncall。
+
+- [Eino 文档](https://www.cloudwego.io/zh/docs/eino/)
+- [Elasticsearch Go 客户端文档](https://github.com/elastic/go-elasticsearch)
diff --git a/content/zh/docs/eino/ecosystem_integration/indexer/indexer_elasticsearch9.md b/content/zh/docs/eino/ecosystem_integration/indexer/indexer_elasticsearch9.md
new file mode 100644
index 00000000000..0890058b186
--- /dev/null
+++ b/content/zh/docs/eino/ecosystem_integration/indexer/indexer_elasticsearch9.md
@@ -0,0 +1,173 @@
+---
+Description: ""
+date: "2026-01-20"
+lastmod: ""
+tags: []
+title: Indexer - Elasticsearch 9
+weight: 0
+---
+
+> **云搜索服务介绍**
+>
+> 云搜索服务（Cloud Search）是一个全托管、一站式信息检索与分析平台，提供了 ElasticSearch 和 OpenSearch 引擎，支持全文检索、向量检索、混合搜索及时空检索等多种核心能力。
+
+为 [Eino](https://github.com/cloudwego/eino) 实现的 Elasticsearch 9.x 索引器，实现了 `Indexer` 接口。这使得可以与 Eino 的向量存储和检索系统无缝集成，从而增强语义搜索能力。
+
+## 功能特性
+
+- 实现 `github.com/cloudwego/eino/components/indexer.Indexer`
+- 易于集成 Eino 的索引系统
+- 可配置 Elasticsearch 参数
+- 支持向量相似度搜索
+- 批量索引操作
+- 自定义字段映射支持
+- 灵活的文档向量化
+
+## 安装
+
+```bash
+go get github.com/cloudwego/eino-ext/components/indexer/es9@latest
+```
+
+## 快速开始
+
+这里是使用索引器的快速示例，更多细节请阅读 components/indexer/es9/examples/indexer/add_documents.go：
+
+```go
+import (
+ "context"
+ "fmt"
+ "log"
+ "os"
+
+ "github.com/cloudwego/eino/components/embedding"
+ "github.com/cloudwego/eino/schema"
+ "github.com/elastic/go-elasticsearch/v9"
+
+ "github.com/cloudwego/eino-ext/components/embedding/ark"
+ "github.com/cloudwego/eino-ext/components/indexer/es9"
+)
+
+const (
+ indexName = "eino_example"
+ fieldContent = "content"
+ fieldContentVector = "content_vector"
+ fieldExtraLocation = "location"
+ docExtraLocation = "location"
+)
+
+func main() {
+ ctx := context.Background()
+
+ // ES 支持多种连接方式
+ username := os.Getenv("ES_USERNAME")
+ password := os.Getenv("ES_PASSWORD")
+ httpCACertPath := os.Getenv("ES_HTTP_CA_CERT_PATH")
+
+ var cert []byte
+ var err error
+ if httpCACertPath != "" {
+ cert, err = os.ReadFile(httpCACertPath)
+ if err != nil {
+ log.Fatalf("read file failed, err=%v", err)
+ }
+ }
+
+ client, _ := elasticsearch.NewClient(elasticsearch.Config{
+ Addresses: []string{"https://localhost:9200"},
+ Username: username,
+ Password: password,
+ CACert: cert,
+ })
+
+ // 2. 创建 embedding 组件 (使用 Ark)
+ // 请将 "ARK_API_KEY", "ARK_REGION", "ARK_MODEL" 替换为实际配置
+ emb, _ := ark.NewEmbedder(ctx, &ark.EmbeddingConfig{
+ APIKey: os.Getenv("ARK_API_KEY"),
+ Region: os.Getenv("ARK_REGION"),
+ Model: os.Getenv("ARK_MODEL"),
+ })
+
+ // 3. 准备文档
+ // 文档通常包含 ID 和 Content
+ // 也可以包含额外的 Metadata 用于过滤或其他用途
+ docs := []*schema.Document{
+ {
+ ID: "1",
+ Content: "Eiffel Tower: Located in Paris, France.",
+ MetaData: map[string]any{
+ docExtraLocation: "France",
+ },
+ },
+ {
+ ID: "2",
+ Content: "The Great Wall: Located in China.",
+ MetaData: map[string]any{
+ docExtraLocation: "China",
+ },
+ },
+ }
+
+ // 4. 创建 ES 索引器组件
+ indexer, _ := es9.NewIndexer(ctx, &es9.IndexerConfig{
+ Client: client,
+ Index: indexName,
+ BatchSize: 10,
+ // DocumentToFields 指定如何将文档字段映射到 ES 字段
+ DocumentToFields: func(ctx context.Context, doc *schema.Document) (field2Value map[string]es9.FieldValue, err error) {
+ return map[string]es9.FieldValue{
+ fieldContent: {
+ Value: doc.Content,
+ EmbedKey: fieldContentVector, // 对文档内容进行向量化并保存到 "content_vector" 字段
+ },
+ fieldExtraLocation: {
+ // 额外的 metadata 字段
+ Value: doc.MetaData[docExtraLocation],
+ },
+ }, nil
+ },
+ // 提供 embedding 组件用于向量化
+ Embedding: emb,
+ })
+
+ // 5. 索引文档
+ ids, err := indexer.Store(ctx, docs)
+ if err != nil {
+ fmt.Printf("index error: %v\n", err)
+ return
+ }
+ fmt.Println("indexed ids:", ids)
+}
+```
+
+## 配置
+
+可以使用 `IndexerConfig` 结构体配置索引器：
+
+```go
+type IndexerConfig struct {
+ Client *elasticsearch.Client // 必填: Elasticsearch 客户端实例
+ Index string // 必填: 存储文档的索引名称
+ BatchSize int // 选填: 用于 embedding 的最大文本数量 (默认: 5)
+
+ // 必填: 将 Document 字段映射到 Elasticsearch 字段的函数
+ DocumentToFields func(ctx context.Context, doc *schema.Document) (map[string]FieldValue, error)
+
+ // 选填: 仅在需要向量化时必填
+ Embedding embedding.Embedder
+}
+
+// FieldValue 定义了字段应如何存储和向量化
+type FieldValue struct {
+ Value any // 要存储的原始值
+ EmbedKey string // 如果设置，Value 将被向量化并保存
+ Stringify func(val any) (string, error) // 选填: 自定义字符串转换
+}
+```
+
+## 获取帮助
+
+如果有任何问题 或者任何功能建议，欢迎进这个群 oncall。
+
+- [Eino 文档](https://www.cloudwego.io/zh/docs/eino/)
+- [Elasticsearch Go Client 文档](https://github.com/elastic/go-elasticsearch)
diff --git a/content/zh/docs/eino/ecosystem_integration/indexer/indexer_es8.md b/content/zh/docs/eino/ecosystem_integration/indexer/indexer_es8.md
index 1f13a3fce0e..414160c14ba 100644
--- a/content/zh/docs/eino/ecosystem_integration/indexer/indexer_es8.md
+++ b/content/zh/docs/eino/ecosystem_integration/indexer/indexer_es8.md
@@ -1,42 +1,49 @@
---
Description: ""
-date: "2025-12-11"
+date: "2026-01-20"
lastmod: ""
tags: []
-title: Indexer - ES8
+title: Indexer - ElasticSearch 8
weight: 0
---
-## **ES8 索引器**
+> **云搜索服务介绍**
+>
+> 云搜索服务（Cloud Search）是一个全托管、一站式信息检索与分析平台，提供了 ElasticSearch 和 OpenSearch 引擎，支持全文检索、向量检索、混合搜索及时空检索等多种核心能力。
-这是一个 [Eino](https://github.com/cloudwego/eino) 的 Elasticsearch 8.x 索引器实现，它实现了 `Indexer` 接口。这使得与 Eino 的向量存储和检索系统无缝集成，从而增强了语义搜索能力。
+为 [Eino](https://github.com/cloudwego/eino) 实现的 Elasticsearch 8.x 索引器，实现了 `Indexer` 接口。这使得可以与 Eino 的向量存储和检索系统无缝集成，从而增强语义搜索能力。
-## **特性**
+## 功能特性
-- 实现了 `github.com/cloudwego/eino/components/indexer.Indexer`
-- 易于与 Eino 的索引系统集成
-- 可配置的 Elasticsearch 参数
+- 实现 `github.com/cloudwego/eino/components/indexer.Indexer`
+- 易于集成 Eino 的索引系统
+- 可配置 Elasticsearch 参数
- 支持向量相似度搜索
- 批量索引操作
-- 支持自定义字段映射
+- 自定义字段映射支持
- 灵活的文档向量化
-## **安装**
+## 安装
```bash
go get github.com/cloudwego/eino-ext/components/indexer/es8@latest
```
-## **快速开始**
+## 快速开始
-这是一个如何使用索引器的快速示例，你可以阅读 `components/indexer/es8/examples/indexer/add_documents.go` 获取更多细节：
+这里是使用索引器的快速示例，更多细节请阅读 components/indexer/es8/examples/indexer/add_documents.go：
```go
import (
+ "context"
+ "os"
+
"github.com/cloudwego/eino/components/embedding"
"github.com/cloudwego/eino/schema"
- "github.com/elastic/go-elasticsearch/v8"
- "github.com/cloudwego/eino-ext/components/indexer/es8" // 导入 es8 索引器
+ elasticsearch "github.com/elastic/go-elasticsearch/v8"
+
+ "github.com/cloudwego/eino-ext/components/embedding/ark"
+ "github.com/cloudwego/eino-ext/components/indexer/es8"
)
const (
@@ -49,92 +56,113 @@ const (
func main() {
ctx := context.Background()
-
- // es 支持多种连接方式
+ // es supports multiple ways to connect
username := os.Getenv("ES_USERNAME")
password := os.Getenv("ES_PASSWORD")
- httpCACertPath := os.Getenv("ES_HTTP_CA_CERT_PATH")
- cert, err := os.ReadFile(httpCACertPath)
- if err != nil {
- log.Fatalf("read file failed, err=%v", err)
+ // 1. 创建 ES 客户端
+ httpCACertPath := os.Getenv("ES_HTTP_CA_CERT_PATH")
+ if httpCACertPath != "" {
+ cert, err := os.ReadFile(httpCACertPath)
+ if err != nil {
+ log.Fatalf("read file failed, err=%v", err)
+ }
}
- client, err := elasticsearch.NewClient(elasticsearch.Config{
+ client, _ := elasticsearch.NewClient(elasticsearch.Config{
Addresses: []string{"https://localhost:9200"},
Username: username,
Password: password,
CACert: cert,
})
- if err != nil {
- log.Panicf("connect es8 failed, err=%v", err)
- }
- // 创建 embedding 组件
- emb := createYourEmbedding()
+ // 2. 创建 embedding 组件
+ // 使用火山引擎 Ark，替换环境变量为真实配置
+ emb, _ := ark.NewEmbedder(ctx, &ark.EmbeddingConfig{
+ APIKey: os.Getenv("ARK_API_KEY"),
+ Region: os.Getenv("ARK_REGION"),
+ Model: os.Getenv("ARK_MODEL"),
+ })
- // 加载文档
- docs := loadYourDocs()
+ // 3. 准备文档
+ // 文档通常包含 ID 和 Content。也可以添加额外的元数据用于过滤等用途。
+ docs := []*schema.Document{
+ {
+ ID: "1",
+ Content: "Eiffel Tower: Located in Paris, France.",
+ MetaData: map[string]any{
+ docExtraLocation: "France",
+ },
+ },
+ {
+ ID: "2",
+ Content: "The Great Wall: Located in China.",
+ MetaData: map[string]any{
+ docExtraLocation: "China",
+ },
+ },
+ }
- // 创建 es 索引器组件
- indexer, err := es8.NewIndexer(ctx, &es8.IndexerConfig{
+ // 4. 创建 ES 索引器组件
+ indexer, _ := es8.NewIndexer(ctx, &es8.IndexerConfig{
Client: client,
Index: indexName,
BatchSize: 10,
+ // DocumentToFields 指定如何将文档字段映射到 ES 字段
DocumentToFields: func(ctx context.Context, doc *schema.Document) (field2Value map[string]es8.FieldValue, err error) {
return map[string]es8.FieldValue{
fieldContent: {
Value: doc.Content,
- EmbedKey: fieldContentVector, // 对文档内容进行向量化并保存向量到 "content_vector" 字段
+ EmbedKey: fieldContentVector, // 向量化文档内容并保存到 "content_vector" 字段
},
fieldExtraLocation: {
+ // 额外的元数据字段
Value: doc.MetaData[docExtraLocation],
},
}, nil
},
- Embedding: emb, // 替换为真实的 embedding 组件
+ // 提供 embedding 组件用于向量化
+ Embedding: emb,
})
- if err != nil {
- log.Panicf("create indexer failed, err=%v", err)
- }
+ // 5. 索引文档
ids, err := indexer.Store(ctx, docs)
if err != nil {
- log.Panicf("create docs failed, err=%v", err)
+ fmt.Printf("index error: %v\n", err)
+ return
}
-
- fmt.Println(ids)
- // 与 Eino 系统一起使用
- // ... 配置并与 Eino 一起使用
+ fmt.Println("indexed ids:", ids)
}
```
-## **配置**
+## 配置
-索引器可以通过 `IndexerConfig` 结构体进行配置：
+可以使用 `IndexerConfig` 结构体配置索引器：
```go
type IndexerConfig struct {
- Client *elasticsearch.Client // 必填：Elasticsearch 客户端实例
- Index string // 必填：存储文档的索引名称
- BatchSize int // 可选：embedding 的最大文本大小（默认：5）
-
- // 必填：将文档字段映射到 Elasticsearch 字段的函数
- DocumentToFields func(ctx context.Context, doc *schema.Document) (map[string]FieldValue, error)
-
- // 可选：仅当需要向量化时才需要
- Embedding embedding.Embedder
+ Client *elasticsearch.Client // 必填: Elasticsearch 客户端实例
+ Index string // 必填: 存储文档的索引名称
+ BatchSize int // 选填: 用于 embedding 的最大文本数量 (默认: 5)
+
+ // 必填: 将 Document 字段映射到 Elasticsearch 字段的函数
+ DocumentToFields func(ctx context.Context, doc *schema.Document) (map[string]FieldValue, error)
+
+ // 选填: 仅在需要向量化时必填
+ Embedding embedding.Embedder
}
// FieldValue 定义了字段应如何存储和向量化
type FieldValue struct {
- Value any // 要存储的原始值
- EmbedKey string // 如果设置，Value 将被向量化并保存
- Stringify func(val any) (string, error) // 可选：自定义字符串转换
+ Value any // 要存储的原始值
+ EmbedKey string // 如果设置，Value 将被向量化并保存
+ Stringify func(val any) (string, error) // 选填: 自定义字符串转换
}
```
-## **更多详情**
+## 获取帮助
+
+如果有任何问题 或者任何功能建议，欢迎进这个群 oncall。
-- [Eino 文档](https://github.com/cloudwego/eino)
-- [Elasticsearch Go 客户端文档](https://github.com/elastic/go-elasticsearch)
+- [Eino 文档](https://www.cloudwego.io/zh/docs/eino/)
+- [Elasticsearch Go Client 文档](https://github.com/elastic/go-elasticsearch)
diff --git a/content/zh/docs/eino/ecosystem_integration/indexer/indexer_milvus.md b/content/zh/docs/eino/ecosystem_integration/indexer/indexer_milvus.md
index e3f036a3d5a..499f9f1c1fd 100644
--- a/content/zh/docs/eino/ecosystem_integration/indexer/indexer_milvus.md
+++ b/content/zh/docs/eino/ecosystem_integration/indexer/indexer_milvus.md
@@ -1,12 +1,16 @@
---
Description: ""
-date: "2025-12-11"
+date: "2026-01-20"
lastmod: ""
tags: []
-title: Indexer - Milvus
+title: Indexer - Milvus v1 (旧版)
weight: 0
---
+> **模块说明：** 本模块 (`EINO-ext/milvus`) 基于 `milvus-sdk-go` 实现。鉴于底层 SDK 已停止维护，且最高仅适配至 Milvus 2.4 版本，本模块仅作为向后兼容组件保留。
+>
+> **建议：** 新接入的用户请直接使用 [Indexer - Milvus 2 (v2.5+)](/zh/docs/eino/ecosystem_integration/indexer/indexer_milvusv2) 模块以获得持续支持。
+
## **Milvus 存储**
基于 Milvus 2.x 的向量存储实现，为 [Eino](https://github.com/cloudwego/eino) 提供了符合 `Indexer` 接口的存储方案。该组件可无缝集成 Eino 的向量存储和检索系统，增强语义搜索能力。
@@ -153,3 +157,21 @@ type IndexerConfig struct {
```
## **默认数据模型**
+
+## 获取帮助
+
+- [[集团内部版] Milvus 快速入门](https://bytedance.larkoffice.com/wiki/P3JBw4PtKiLGPhkUCQZcXbHFnkf)
+
+如果有任何问题 或者任何功能建议，欢迎进这个群 oncall。
+
+### 外部参考
+
+- [Milvus 文档](https://milvus.io/docs)
+- [Milvus 索引类型](https://milvus.io/docs/index.md)
+- [Milvus 度量类型](https://milvus.io/docs/metric.md)
+- [milvus-sdk-go 参考](https://milvus.io/api-reference/go/v2.4.x/About.md)
+
+### 相关文档
+
+- [Eino: Indexer 使用说明](/zh/docs/eino/core_modules/components/indexer_guide)
+- [Eino: Retriever 使用说明](/zh/docs/eino/core_modules/components/retriever_guide)
diff --git a/content/zh/docs/eino/ecosystem_integration/indexer/indexer_milvusv2.md b/content/zh/docs/eino/ecosystem_integration/indexer/indexer_milvusv2.md
new file mode 100644
index 00000000000..b6c27f178d3
--- /dev/null
+++ b/content/zh/docs/eino/ecosystem_integration/indexer/indexer_milvusv2.md
@@ -0,0 +1,395 @@
+---
+Description: ""
+date: "2026-01-20"
+lastmod: ""
+tags: []
+title: Indexer - Milvus v2 (推荐)
+weight: 0
+---
+
+> **向量数据库 Milvus 版介绍**
+>
+> 向量检索服务 Milvus 版为基于开源 Milvus 构建的全托管数据库服务，提供高效的非结构化数据检索能力，适用于多样化 AI 场景，客户无需再关心底层硬件资源，降低使用成本，提高整体效率。
+>
+> 鉴于公司**内场**的 Milvus 服务采用标准 SDK，因此适用 **EINO-ext 社区版本**。
+
+本包为 EINO 框架提供 Milvus 2.x (V2 SDK) 索引器实现，支持文档存储和向量索引。
+
+> **注意**: 本包需要 **Milvus 2.5+** 以支持服务器端函数（如 BM25），基础功能兼容低版本。
+
+## 功能特性
+
+- **Milvus V2 SDK**: 使用最新的 `milvus-io/milvus/client/v2` SDK
+- **灵活的索引类型**: 支持多种索引构建器，包括 Auto, HNSW, IVF 系列, SCANN, DiskANN, GPU 索引以及 RaBitQ (Milvus 2.6+)
+- **混合搜索就绪**: 原生支持稀疏向量 (BM25/SPLADE) 与稠密向量的混合存储
+- **服务端向量生成**: 使用 Milvus Functions (BM25) 自动生成稀疏向量
+- **自动化管理**: 自动处理集合 Schema 创建、索引构建和加载
+- **字段分析**: 可配置的文本分析器（支持中文 Jieba、英文、Standard 等）
+- **自定义文档转换**: Eino 文档到 Milvus 列的灵活映射
+
+## 安装
+
+```bash
+go get github.com/cloudwego/eino-ext/components/indexer/milvus2
+```
+
+## 快速开始
+
+```go
+package main
+
+import (
+ "context"
+ "log"
+ "os"
+
+ "github.com/cloudwego/eino-ext/components/embedding/ark"
+ "github.com/cloudwego/eino/schema"
+ "github.com/milvus-io/milvus/client/v2/milvusclient"
+
+ milvus2 "github.com/cloudwego/eino-ext/components/indexer/milvus2"
+)
+
+func main() {
+ // 获取环境变量
+ addr := os.Getenv("MILVUS_ADDR")
+ username := os.Getenv("MILVUS_USERNAME")
+ password := os.Getenv("MILVUS_PASSWORD")
+ arkApiKey := os.Getenv("ARK_API_KEY")
+ arkModel := os.Getenv("ARK_MODEL")
+
+ ctx := context.Background()
+
+ // 创建 embedding 模型
+ emb, err := ark.NewEmbedder(ctx, &ark.EmbeddingConfig{
+ APIKey: arkApiKey,
+ Model: arkModel,
+ })
+ if err != nil {
+ log.Fatalf("Failed to create embedding: %v", err)
+ return
+ }
+
+ // 创建索引器
+ indexer, err := milvus2.NewIndexer(ctx, &milvus2.IndexerConfig{
+ ClientConfig: &milvusclient.ClientConfig{
+ Address: addr,
+ Username: username,
+ Password: password,
+ },
+ Collection: "my_collection",
+
+ Vector: &milvus2.VectorConfig{
+ Dimension: 1024, // 与 embedding 模型维度匹配
+ MetricType: milvus2.COSINE,
+ IndexBuilder: milvus2.NewHNSWIndexBuilder().WithM(16).WithEfConstruction(200),
+ },
+ Embedding: emb,
+ })
+ if err != nil {
+ log.Fatalf("Failed to create indexer: %v", err)
+ return
+ }
+ log.Printf("Indexer created successfully")
+
+ // 存储文档
+ docs := []*schema.Document{
+ {
+ ID: "doc1",
+ Content: "Milvus is an open-source vector database",
+ MetaData: map[string]any{
+ "category": "database",
+ "year": 2021,
+ },
+ },
+ {
+ ID: "doc2",
+ Content: "EINO is a framework for building AI applications",
+ },
+ }
+ ids, err := indexer.Store(ctx, docs)
+ if err != nil {
+ log.Fatalf("Failed to store: %v", err)
+ return
+ }
+ log.Printf("Store success, ids: %v", ids)
+}
+```
+
+## 配置选项
+
+| 字段 | 类型 | 默认值 | 描述 |
Client | *milvusclient.Client | - | 预配置的 Milvus 客户端（可选） |
ClientConfig | *milvusclient.ClientConfig | - | 客户端配置（Client 为空时必需） |
Collection | string | "eino_collection" | 集合名称 |
Vector | *VectorConfig | - | 稠密向量配置 (维度, MetricType, 字段名) |
Sparse | *SparseVectorConfig | - | 稀疏向量配置 (MetricType, 字段名) |
IndexBuilder | IndexBuilder | AutoIndexBuilder | 索引类型构建器 |
Embedding | embedding.Embedder | - | 用于向量化的 Embedder（可选）。如果为空，文档必须包含向量 (BYOV)。 |
ConsistencyLevel | ConsistencyLevel | ConsistencyLevelDefault | 一致性级别 (ConsistencyLevelDefault使用 Milvus 默认: Bounded; 如果未显式设置，则保持集合级别设置) |
PartitionName | string | - | 插入数据的默认分区 |
EnableDynamicSchema | bool | false | 启用动态字段支持 |
Functions | []*entity.Function | - | Schema 函数定义（如 BM25），用于服务器端处理 |
FieldParams | map[string]map[string]string | - | 字段参数配置（如 enable_analyzer） |
| 字段 | 类型 | 默认值 | 描述 |
Dimension | int64 | - | 向量维度 (必需) |
MetricType | MetricType | L2 | 相似度度量类型 (L2, IP, COSINE 等) |
VectorField | string | "vector" | 稠密向量字段名 |
| 字段 | 类型 | 默认值 | 描述 |
VectorField | string | "sparse_vector" | 稀疏向量字段名 |
MetricType | MetricType | BM25 | 相似度度量类型 |
Method | SparseMethod | SparseMethodAuto | 生成方法 (SparseMethodAuto或 SparseMethodPrecomputed) |
| 构建器 | 描述 | 关键参数 |
NewAutoIndexBuilder() | Milvus 自动选择最优索引 | - |
NewHNSWIndexBuilder() | 基于图的高性能索引 | M, EfConstruction |
NewIVFFlatIndexBuilder() | 基于聚类的搜索 | NList |
NewIVFPQIndexBuilder() | 乘积量化，内存高效 | NList, M, NBits |
NewIVFSQ8IndexBuilder() | 标量量化 | NList |
NewIVFRabitQIndexBuilder() | IVF + RaBitQ 二进制量化 (Milvus 2.6+) | NList |
NewFlatIndexBuilder() | 暴力精确搜索 | - |
NewDiskANNIndexBuilder() | 面向大数据集的磁盘索引 | - |
NewSCANNIndexBuilder() | 高召回率的快速搜索 | NList, WithRawDataEnabled |
NewBinFlatIndexBuilder() | 二进制向量的暴力搜索 | - |
NewBinIVFFlatIndexBuilder() | 二进制向量的聚类搜索 | NList |
NewGPUBruteForceIndexBuilder() | GPU 加速暴力搜索 | - |
NewGPUIVFFlatIndexBuilder() | GPU 加速 IVF_FLAT | - |
NewGPUIVFPQIndexBuilder() | GPU 加速 IVF_PQ | - |
NewGPUCagraIndexBuilder() | GPU 加速图索引 (CAGRA) | IntermediateGraphDegree, GraphDegree |
| 构建器 | 描述 | 关键参数 |
NewSparseInvertedIndexBuilder() | 稀疏向量倒排索引 | DropRatioBuild |
NewSparseWANDIndexBuilder() | 稀疏向量 WAND 算法 | DropRatioBuild |
| 度量类型 | 描述 |
L2 | 欧几里得距离 |
IP | 内积 |
COSINE | 余弦相似度 |
| 度量类型 | 描述 |
BM25 | Okapi BM25 (SparseMethodAuto必需) |
IP | 内积 (适用于预计算的稀疏向量) |
| 度量类型 | 描述 |
HAMMING | 汉明距离 |
JACCARD | 杰卡德距离 |
TANIMOTO | Tanimoto 距离 |
SUBSTRUCTURE | 子结构搜索 |
SUPERSTRUCTURE | 超结构搜索 |
| 字段 | 类型 | 默认值 | 描述 |
Client | *milvusclient.Client | - | 预配置的 Milvus 客户端（可选） |
ClientConfig | *milvusclient.ClientConfig | - | 客户端配置（Client 为空时必需） |
Collection | string | "eino_collection" | 集合名称 |
TopK | int | 5 | 返回结果数量 |
VectorField | string | "vector" | 稠密向量字段名 |
SparseVectorField | string | "sparse_vector" | 稀疏向量字段名 |
OutputFields | []string | 所有字段 | 结果中返回的字段 |
SearchMode | SearchMode | - | 搜索策略（必需） |
Embedding | embedding.Embedder | - | 用于查询向量化的 Embedder（必需） |
DocumentConverter | func | 默认转换器 | 自定义结果到文档转换 |
ConsistencyLevel | ConsistencyLevel | ConsistencyLevelDefault | 一致性级别 (ConsistencyLevelDefault使用 collection 的级别；不应用按请求覆盖) |
Partitions | []string | - | 要搜索的分区 |
| 度量类型 | 描述 |
L2 | 欧几里得距离 |
IP | 内积 |
COSINE | 余弦相似度 |
| 度量类型 | 描述 |
BM25 | Okapi BM25 (BM25 搜索必需) |
IP | 内积 (适用于预计算的稀疏向量) |
| 度量类型 | 描述 |
HAMMING | 汉明距离 |
JACCARD | 杰卡德距离 |
TANIMOTO | Tanimoto 距离 |
SUBSTRUCTURE | 子结构搜索 |
SUPERSTRUCTURE | 超结构搜索 |
| 搜索模式 | 最低 OpenSearch 版本 | 说明 |
ExactMatch | 1.0+ | 标准查询 DSL |
RawString | 1.0+ | 标准查询 DSL |
DenseVectorSimilarity | 1.0+ | 使用 script_score和 painless 向量函数 |
Approximate(KNN) | 1.0+ | 自 1.0 起支持基础 KNN。高效过滤 (Post-filtering) 需要 2.4+ (Lucene HNSW) 或 2.9+ (Faiss)。 |
Approximate(Hybrid) | 2.10+ | 生成 bool查询。需要 2.10+ normalization-processor支持高级分数归一化 (Convex Combination)。基础 bool查询在早期版本 (1.0+) 也可工作。 |
Approximate(RRF) | 2.19+ | 需要 score-ranker-processor(2.19+) 和 neural-search插件。 |
NeuralSparse(Query Text) | 2.11+ | 需要 neural-search插件和已部署的模型。 |
NeuralSparse(TokenWeights) | 2.11+ | 需要 neural-search插件。 |
| 搜索模式 | 最低 OpenSearch 版本 | 说明 |
ExactMatch | 1.0+ | 标准查询 DSL |
RawString | 1.0+ | 标准查询 DSL |
DenseVectorSimilarity | 1.0+ | 使用 script_score和 painless 向量函数 |
Approximate(KNN) | 1.0+ | 自 1.0 起支持基础 KNN。高效过滤 (Post-filtering) 需要 2.4+ (Lucene HNSW) 或 2.9+ (Faiss)。 |
Approximate(Hybrid) | 2.10+ | 生成 bool查询。需要 2.10+ normalization-processor支持高级分数归一化 (Convex Combination)。基础 bool查询在早期版本 (1.0+) 也可工作。 |
Approximate(RRF) | 2.19+ | 需要 score-ranker-processor(2.19+) 和 neural-search插件。 |
NeuralSparse(Query Text) | 2.11+ | 需要 neural-search插件和已部署的模型。 |
NeuralSparse(TokenWeights) | 2.11+ | 需要 neural-search插件。 |
+
我们在 Eino 的 `flow` 包中提供了开箱即用的 ReAct 智能体的完整实现。代码参见： [flow/agent/react](https://github.com/cloudwego/eino/blob/main/flow/agent/react/react.go)
Eino 会在上述代码背后自动完成一些重要工作：
@@ -229,7 +231,7 @@ Eino 框架由几个部分组成：
## 依赖说明
- Go 1.18 及以上版本
-- Eino 依赖了 [kin-openapi](https://github.com/getkin/kin-openapi) 的 OpenAPI JSONSchema 实现。为了能够兼容 Go 1.18 版本，我们将 kin-openapi 的版本固定在了 v0.118.0。
+- Eino 依赖了 [kin-openapi](https://github.com/getkin/kin-openapi) 的 OpenAPI JSONSchema 实现。为了能够兼容 Go 1.18 版本，我们将 kin-openapi 的版本固定在了 v0.118.0。V0.6.0 之后已去除此依赖。
## 安全
diff --git a/content/zh/docs/eino/overview/eino_adk0_1.md b/content/zh/docs/eino/overview/eino_adk0_1.md
index 76cd15f7676..b9b31f98c13 100644
--- a/content/zh/docs/eino/overview/eino_adk0_1.md
+++ b/content/zh/docs/eino/overview/eino_adk0_1.md
@@ -1,6 +1,6 @@
---
Description: ""
-date: "2025-12-09"
+date: "2026-01-20"
lastmod: ""
tags: []
title: Eino ADK：一文搞定 AI Agent 核心设计模式，从 0 到 1 搭建智能体系统
diff --git a/content/zh/docs/eino/overview/graph_or_agent.md b/content/zh/docs/eino/overview/graph_or_agent.md
index f6b7e0ccdca..96912d6f8bc 100644
--- a/content/zh/docs/eino/overview/graph_or_agent.md
+++ b/content/zh/docs/eino/overview/graph_or_agent.md
@@ -1,6 +1,6 @@
---
Description: ""
-date: "2025-12-09"
+date: "2026-01-20"
lastmod: ""
tags: []
title: Agent 还是 Graph？AI 应用路线辨析
@@ -16,7 +16,9 @@ weight: 5
这张看似简单的截图，代表了“AI 应用”的两种形态：
- 以“聊天框”为代表性标志的“Agent（智能体）”。**Agent 以 LLM（大语言模型）为决策中心，自主规划并能进行多轮交互**，天然适合处理开放式、持续性的任务，表现为一种“对话”形态。
-- 以“按钮”或者“API”为代表性标志的“Graph（流程图）”。比如上面的“录音纪要”这个“按钮”，其背后的 Graph 大概是“录音”-》“LLM 理解并总结” -》“保存录音”这种固定流程。**Graph 的核心在于其流程的确定性与任务的封闭性**，通过预定义的节点和边来完成特定目标，表现为一种“功能”形态。
+- 以“按钮”或者“API”为代表性标志的“Graph（流程图）”。比如上面的“录音纪要”这个“按钮”，其背后的 Graph 大概是“录音”-》“LLM 理解并总结” -》“保存录音”这种固定流程。**Graph 的核心在于其流程的确定性与任务的封闭性**，通过预定义的节点和边来完成特定目标，表现为一种“功能”形态。举个例子，视频生成是 “API”形态 AI 应用：
+
+
```mermaid
flowchart TD
@@ -342,3 +344,5 @@ func (g *InvokableGraphTool[I, O]) Info(_ context.Context) (*schema.ToolInfo, er
return g.tInfo, nil
}
```
+
+[eino-example 项目链接](https://github.com/cloudwego/eino-examples/tree/main/adk/common/tool/graphtool)
diff --git a/content/zh/docs/eino/quick_start/_index.md b/content/zh/docs/eino/quick_start/_index.md
index 0094c9c11e8..617e9dc72c8 100644
--- a/content/zh/docs/eino/quick_start/_index.md
+++ b/content/zh/docs/eino/quick_start/_index.md
@@ -1,6 +1,6 @@
---
Description: ""
-date: "2025-11-20"
+date: "2026-01-20"
lastmod: ""
tags: []
title: 'Eino: 快速开始'
diff --git a/content/zh/docs/eino/quick_start/agent_llm_with_tools.md b/content/zh/docs/eino/quick_start/agent_llm_with_tools.md
index 18047b6e2d7..acf81e09b89 100644
--- a/content/zh/docs/eino/quick_start/agent_llm_with_tools.md
+++ b/content/zh/docs/eino/quick_start/agent_llm_with_tools.md
@@ -1,6 +1,6 @@
---
Description: ""
-date: "2025-12-09"
+date: "2026-01-20"
lastmod: ""
tags: []
title: Agent-让大模型拥有双手
diff --git a/content/zh/docs/eino/quick_start/simple_llm_application.md b/content/zh/docs/eino/quick_start/simple_llm_application.md
index a14be0795be..64da61872df 100644
--- a/content/zh/docs/eino/quick_start/simple_llm_application.md
+++ b/content/zh/docs/eino/quick_start/simple_llm_application.md
@@ -1,6 +1,6 @@
---
Description: ""
-date: "2025-12-09"
+date: "2026-01-20"
lastmod: ""
tags: []
title: 实现一个最简 LLM 应用
diff --git a/static/img/eino/eino_adk_react_illustration.png b/static/img/eino/eino_adk_react_illustration.png
new file mode 100644
index 00000000000..a92ada1efb4
Binary files /dev/null and b/static/img/eino/eino_adk_react_illustration.png differ
diff --git a/static/img/eino/eino_architecture_overview.png b/static/img/eino/eino_architecture_overview.png
index bd619e7db1b..f2667c92322 100644
Binary files a/static/img/eino/eino_architecture_overview.png and b/static/img/eino/eino_architecture_overview.png differ
diff --git a/static/img/eino/eino_complex_workflow_as_api.png b/static/img/eino/eino_complex_workflow_as_api.png
new file mode 100644
index 00000000000..f48dc4232c8
Binary files /dev/null and b/static/img/eino/eino_complex_workflow_as_api.png differ
diff --git a/static/img/eino/eino_practice_agent_graph.png b/static/img/eino/eino_practice_agent_graph.png
index 2a8cd827d55..8f267952488 100644
Binary files a/static/img/eino/eino_practice_agent_graph.png and b/static/img/eino/eino_practice_agent_graph.png differ
diff --git a/static/img/eino/eino_practice_cognition_loop.png b/static/img/eino/eino_practice_cognition_loop.png
index a9ac09854cd..47a8f58e78b 100644
Binary files a/static/img/eino/eino_practice_cognition_loop.png and b/static/img/eino/eino_practice_cognition_loop.png differ
diff --git a/static/img/eino/eino_practice_index_flow.png b/static/img/eino/eino_practice_index_flow.png
index 6e475e23aff..38428eac93d 100644
Binary files a/static/img/eino/eino_practice_index_flow.png and b/static/img/eino/eino_practice_index_flow.png differ
diff --git a/static/img/eino/eino_project_structure_and_modules.png b/static/img/eino/eino_project_structure_and_modules.png
index d5c4d05124f..78a94cd8523 100644
Binary files a/static/img/eino/eino_project_structure_and_modules.png and b/static/img/eino/eino_project_structure_and_modules.png differ