Building an LLM Workflow

This guide walks through building an end-to-end agentic workflow that uses an LLM to make a decision over your microservices. It picks up where Building Agentic Workflows leaves off and adds the LLM layer.

By the end you will have:

  • The LLM machinery added to your app — llm.core, a real provider, your API key.
  • A microservice handler making a synchronous Chat call.
  • A functional endpoint authored as an LLM tool, exposed automatically through its OpenAPI document.
  • The same flow promoted to a durable ChatLoop step inside a parent workflow that branches on the LLM’s verdict.

The Worked Example

You’re building expense report triage. A user submits an expense; the system needs to decide one of three outcomes:

  • Auto-approve if the request is small, in-policy, and the requester has budget.
  • Route to manager if it’s borderline — over a soft threshold, or unusual for the category.
  • Reject if it clearly violates policy.

The decision is delegated to an LLM that has access to three tools:

ToolPurpose
policy.example/checkChecks whether the expense conforms to the corporate policy.
budget.example/remainingReturns the requester’s remaining budget for the relevant category.
expenses.example/similarReturns recent comparable expenses for context.

The LLM is instructed to end with a structured JSON verdict; a downstream task parses it and the workflow branches accordingly.

The shape, drawn the same way as creditflow:

graph LR
    classDef task fill:#32a7c1,color:#f4f2ef,stroke:#434343
    classDef sub fill:#ed2e92,color:#f4f2ef,stroke:#434343
    classDef term fill:#e5f4f3,color:#434343,stroke:#434343

    _start(( )):::term --> s1
    s1["SubmitExpense"]:::task --> s2
    s2["TriageWithLLM"]:::task --> s3
    s3["Decide"]:::task -->|"approve"| s4
    s3 -->|"manager"| s5
    s3 -->|"reject"| s6
    s4["AutoApprove"]:::task --> _end(( )):::term
    s5["RouteToManager"]:::task --> _end
    s6["RejectWithReason"]:::task --> _end

You’ll build it in five steps.

Step 1: Add the LLM Services to Your App

The LLM machinery is three pieces: the orchestrator llm.core (which runs the tool-calling loop), one or more providers (claudellm, chatgptllm, geminillm, litellm), and the Foreman (because ChatLoop is a workflow). Add them all to your app in main/main.go:

import (
    "github.com/microbus-io/fabric/coreservices/foreman"
    "github.com/microbus-io/fabric/coreservices/llm"
    "github.com/microbus-io/fabric/coreservices/claudellm"
    // "github.com/microbus-io/fabric/coreservices/chatgptllm"
    // "github.com/microbus-io/fabric/coreservices/geminillm"

    "github.com/example/myapp/expense"
    "github.com/example/myapp/policy"
    "github.com/example/myapp/budget"
    "github.com/example/myapp/expenses"
)

app.Add(
    foreman.NewService(),
    llm.NewService(),
    claudellm.NewService(),

    // Your services.
    expense.NewService(),
    policy.NewService(),
    budget.NewService(),
    expenses.NewService(),
)

Add additional providers only if you want to swap models at call time — provider is chosen per Chat call, not globally.

Configure the provider’s API key in config.local.yaml (git-ignored). The key is keyed by the provider’s hostname:

# config.local.yaml
claude.llm.core:
  APIKey: sk-ant-your-key-here
# chatgpt.llm.core:
#   APIKey: sk-your-openai-key
# gemini.llm.core:
#   APIKey: your-gemini-key

If you’re using ChatLoop, the Foreman also needs a SQL data source for its durable state. In production set it explicitly:

# config.yaml
foreman.core:
  SQLDataSourceName: root:root@tcp(127.0.0.1:3306)/mydatabase

In tests the Foreman defaults to in-memory SQLite — no config needed.

Step 2: Make Your First Chat Call

Before bringing in workflows or tools, prove the wiring works with a one-line synchronous Chat:

import (
    "github.com/microbus-io/fabric/coreservices/llm/llmapi"
)

func (svc *Service) AskLLM(ctx context.Context, prompt string) (answer string, err error) {
    items := []llmapi.Item{
        llmapi.NewMessage("user", prompt).AsItem(),
    }
    itemsOut, _, _, err := llmapi.NewClient(svc).Chat(
        ctx,
        llmapi.ProviderAny,    // let llm.core resolve a configured provider
        llmapi.ModelDefault,   // a capability-tier alias
        items,
        nil, // no tools yet
        nil, // default options
    )
    if err != nil {
        return "", errors.Trace(err)
    }
    // The assistant's reply is the last item in itemsOut.
    last := itemsOut[len(itemsOut)-1]
    if last.Message != nil {
        return last.Message.Content, nil
    }
    return "", nil
}

Three things are happening here:

  • llmapi.NewClient(svc).Chat(...) is a normal Microbus client call — same actor propagation, same tracing, same time budget as any other service-to-service call.
  • Passing llmapi.ProviderAny (or empty) lets llm.core resolve a configured provider at runtime, and llmapi.ModelDefault is a capability-tier alias. A model can be named three ways: a tier alias (ModelFast / ModelDefault / ModelSmart), a provider-family alias (e.g. opus), or a concrete vendor-prefixed model string. A single API key of any one brand is enough to run this.
  • The conversation is an ordered []llmapi.Item log. Chat returns the full conversation, including the assistant’s reply appended to your input items, so read the reply through the last item’s Message variant. To continue, append a new user item to itemsOut and call Chat again.

If this works end-to-end (response text appears, no errors), the LLM layer is correctly wired. On to giving the LLM tools.

Step 3: Author an Endpoint as an LLM Tool

Tools in Microbus are just endpoints. Any functional, web, or workflow endpoint on the bus is tool-eligible — llm.core discovers them by fetching each tool host’s OpenAPI document and reflecting the matching operation’s schema. There is no separate tool registration step.

What makes an endpoint a good LLM tool is the same set of conventions that make it pleasant for another Go microservice to call. Add the policy.example service for this guide:

package policy

import (
    "context"

    "github.com/microbus-io/errors"
    "net/http"
)

type Service struct {
    *Intermediate
}

// Check verifies that an expense conforms to the corporate policy.
func (svc *Service) Check(ctx context.Context, category string, amountUSD float64, vendor string) (withinPolicy bool, reason string, err error) {
    if amountUSD <= 0 {
        return false, "amount must be positive", errors.New("invalid amount", http.StatusBadRequest)
    }
    switch category {
    case "travel":
        if amountUSD > 5000 {
            return false, "travel expenses over $5000 require pre-approval", nil
        }
    case "meals":
        if amountUSD > 150 {
            return false, "meal expenses over $150 require receipts and approval", nil
        }
    case "supplies", "software":
        // No category-specific ceiling.
    default:
        return false, "unknown category: " + category, nil
    }
    return true, "within policy", nil
}

The endpoint is declared in policyapi/definition.go, where the per-argument descriptions live as jsonschema_description tags on the input and output struct fields:

// Check verifies that an expense conforms to the corporate policy.
var Check = define.Function{ // MARKER: Check
    Host: Hostname, Method: "POST", Route: ":443/check",
    In: CheckIn{}, Out: CheckOut{},
}

// CheckIn are the input arguments of Check.
type CheckIn struct { // MARKER: Check
    Category  string  `json:"category,omitzero" jsonschema_description:"The expense category. One of: travel, meals, supplies, software"`
    AmountUSD float64 `json:"amountUSD,omitzero" jsonschema_description:"The expense amount in US dollars. Must be positive"`
    Vendor    string  `json:"vendor,omitzero" jsonschema_description:"The vendor name. Pass empty string if unknown"`
}

// CheckOut are the output arguments of Check.
type CheckOut struct { // MARKER: Check
    WithinPolicy bool   `json:"withinPolicy,omitzero" jsonschema_description:"True if the expense complies with all applicable policy rules"`
    Reason       string `json:"reason,omitzero" jsonschema_description:"Short human-readable explanation of the verdict"`
}

Use the dedicated jsonschema_description tag rather than the jsonschema:"description=..." subtag: it is read whole, so a description may contain commas (the jsonschema tag is comma-split into directives and would truncate at the first comma). Four conventions matter for tool quality:

  1. jsonschema_description tags on the input and output struct fields. The OpenAPI generator reflects each CheckIn / CheckOut field’s tag into the LLM’s per-parameter schema documentation. Without them the LLM sees amountUSD: number and has to guess what’s in scope; with them it sees the description and constraints.

  2. Specific argument types and enums. Use a category string whose description tag enumerates the valid values rather than an open-ended string. The LLM constructs valid calls much more reliably from a small enumerated set than from “any string.”

  3. requiredClaims on the subscription. The OpenAPI document llm.core fetches is actor-aware — operations whose requiredClaims the caller cannot satisfy are omitted from the document and therefore never appear in the LLM’s tool catalog. Scope tools narrowly:

    svc.Subscribe(
    "Check", svc.Check,
    sub.At(policyapi.Check.Method, policyapi.Check.Route),
    sub.Description(`Check verifies that an expense conforms to the corporate policy.`),
    sub.Function(policyapi.CheckIn{}, policyapi.CheckOut{}),
    sub.RequiredClaims(`roles.employee`),
)

  4. Return structured errors, not free text. Validation failures return errors.New(..., http.StatusBadRequest) so the LLM (and any caller) sees a 4xx that they can act on rather than a 500 that looks like a system failure.

Repeat the same pattern for budget.example/remaining and expenses.example/similar. The exact bodies don’t matter — they’re stand-ins for whatever data sources you’d plug into.

Step 4: Call Chat With Your Tool

Now wire the tools into the LLM call. The change to step 2 is two lines:

import (
    "github.com/microbus-io/fabric/coreservices/llm/llmapi"

    "github.com/example/myapp/budget/budgetapi"
    "github.com/example/myapp/expenses/expensesapi"
    "github.com/example/myapp/policy/policyapi"
)

func (svc *Service) TriageExpense(ctx context.Context, expense Expense) (verdict string, err error) {
    items := []llmapi.Item{
        llmapi.NewMessage("system", triageSystemPrompt).AsItem(),
        llmapi.NewMessage("user", expense.Describe()).AsItem(),
    }
    toolURLs := []string{
        policyapi.Check.URL(),
        budgetapi.Remaining.URL(),
        expensesapi.Similar.URL(),
    }
    itemsOut, _, _, err := llmapi.NewClient(svc).Chat(
        ctx,
        llmapi.ProviderAny,
        llmapi.ModelSmart,    // a stronger model for decisions
        items,
        toolURLs,
        nil,
    )
    if err != nil {
        return "", errors.Trace(err)
    }
    last := itemsOut[len(itemsOut)-1]
    if last.Message != nil {
        return last.Message.Content, nil
    }
    return "", nil
}

llm.core fetches each tool host’s :888/openapi.json in parallel (carrying the calling actor’s JWT so the documents are actor-filtered), reflects each matching operation’s request-body schema into the tool format the provider expects, and exposes them to the LLM. When the model emits a tool call, llm.core dispatches it as a normal Microbus request — the calculator, policy, or budget service handles it like any other call, the result comes back, and the loop continues until the LLM produces a final text response.

The system prompt is where you constrain the LLM’s output shape — for example:

const triageSystemPrompt = `You are an expense triage assistant. Use the provided tools to gather information about the expense. After gathering enough information, end your final message with a JSON object on its own line, in this exact shape:

{"decision": "approve" | "manager" | "reject", "reason": "..."}

Do not include any other JSON in your response.`

This is what the downstream Decide task will parse.

Step 5: Promote to a Durable ChatLoop Workflow

The synchronous Chat in step 4 works, but it has two problems for production use:

  • No durability. If the request times out mid-loop (e.g., the LLM is slow on a tool round-trip), the whole conversation is lost — even the rounds that already completed.
  • No composition. The triage call is a leaf. There’s no place to attach a downstream Decide task or branch on the LLM’s verdict from inside a workflow.

ChatLoop is the durable workflow form of Chat. It performs the same logic — fetch tools, run the model, dispatch tool calls — but as a series of Foreman steps, each persisted before the next runs. See the ChatLoop graph in LLM integration.

The Parent Workflow Graph

Define a parent workflow that invokes ChatLoop as a subgraph from a task body — ChatLoop is a child workflow launched via flow.Subgraph, not a node in the graph:

import (
    "github.com/microbus-io/fabric/coreservices/llm/llmapi"
    "github.com/microbus-io/dwarf/workflow"
)

func (svc *Service) ExpenseTriage(ctx context.Context) (graph *workflow.Graph, err error) {
    graph = workflow.NewGraph("ExpenseTriage")

    graph.SetEndpoint("SubmitExpense", expenseapi.SubmitExpense.URL())
    graph.SetEndpoint("TriageWithLLM", expenseapi.TriageWithLLM.URL()) // calls flow.Subgraph(llmapi.ChatLoop, ...)
    graph.SetEndpoint("Decide", expenseapi.Decide.URL())
    graph.SetEndpoint("AutoApprove", expenseapi.AutoApprove.URL())
    graph.SetEndpoint("RouteToManager", expenseapi.RouteToManager.URL())
    graph.SetEndpoint("RejectWithReason", expenseapi.RejectWithReason.URL())

    graph.AddTransition("SubmitExpense", "TriageWithLLM")
    graph.AddTransition("TriageWithLLM", "Decide")

    // Conditional fan-out on the LLM's verdict.
    graph.AddTransitionWhen("Decide", "AutoApprove",      `decision == "approve"`)
    graph.AddTransitionWhen("Decide", "RouteToManager",   `decision == "manager"`)
    graph.AddTransitionWhen("Decide", "RejectWithReason", `decision == "reject"`)

    graph.AddTransition("AutoApprove",      workflow.END)
    graph.AddTransition("RouteToManager",   workflow.END)
    graph.AddTransition("RejectWithReason", workflow.END)
    return graph, nil
}

The SubmitExpense Task

SubmitExpense is the entry task. Its job is to translate the typed expense input into the four fields the ChatLoop subgraph expects (provider, model, items, toolURLs):

func (svc *Service) SubmitExpense(ctx context.Context, flow *workflow.Flow, expense Expense) (
    provider string,
    model string,
    items []llmapi.Item,
    toolURLs []string,
    err error,
) {
    return llmapi.ProviderAny,
        llmapi.ModelSmart,
        []llmapi.Item{
            llmapi.NewMessage("system", triageSystemPrompt).AsItem(),
            llmapi.NewMessage("user", expense.Describe()).AsItem(),
        },
        []string{
            policyapi.Check.URL(),
            budgetapi.Remaining.URL(),
            expensesapi.Similar.URL(),
        },
        nil
}

The TriageWithLLM Task

ChatLoop is invoked as a subgraph from a task body — there is no static subgraph node. TriageWithLLM reads the four fields SubmitExpense wrote, calls flow.Subgraph(llmapi.ChatLoop.URL(), in, &out) with them as a typed llmapi.ChatLoopIn, and adopts the conversation from out on re-entry. Only the fields in in cross into the child, and only the child’s final state (items, usage) crosses back:

func (svc *Service) TriageWithLLM(ctx context.Context, flow *workflow.Flow,
    provider string, model string, items []llmapi.Item, toolURLs []string,
) (itemsOut []llmapi.Item, usage llmapi.Usage, err error) {
    var out llmapi.ChatLoopOut
    yield, err := flow.Subgraph(llmapi.ChatLoop.URL(), llmapi.ChatLoopIn{
        Provider: provider,
        Model:    model,
        Items:    items,
        ToolURLs: toolURLs,
    }, &out)
    if yield {
        return nil, llmapi.Usage{}, nil // parked, ChatLoop running
    }
    if err != nil {
        return nil, llmapi.Usage{}, errors.Trace(err)
    }
    // The out pointer is unmarshaled straight into ChatLoop's declared outputs.
    return out.ItemsOut, out.Usage, nil
}

The output itemsOut writes the full conversation back to state under items, which Decide reads next. ChatLoop runs the model and dispatches tool calls across its per-tool-call fan-in, then carries the whole conversation forward in the items output; the parent receives one merged value.

The Decide Task

Decide reads the final assistant message out of items and writes a decision field plus a reason. The conditional transitions in the graph then route the flow:

func (svc *Service) Decide(ctx context.Context, flow *workflow.Flow, items []llmapi.Item) (decision string, reason string, err error) {
    // Find the last assistant message in the conversation.
    var last string
    for i := len(items) - 1; i >= 0; i-- {
        if items[i].Type() == llmapi.ItemMessage && items[i].Message.Role == "assistant" {
            last = items[i].Message.Content
            break
        }
    }
    if last == "" {
        return "reject", "no LLM response", nil
    }

    // The system prompt told the model to end with a JSON object on its own line.
    // Find the last line that parses as the expected verdict shape.
    var verdict struct {
        Decision string `json:"decision"`
        Reason   string `json:"reason"`
    }
    for i := len(last) - 1; i >= 0; i-- {
        if last[i] != '\n' {
            continue
        }
        if json.Unmarshal([]byte(strings.TrimSpace(last[i+1:])), &verdict) == nil && verdict.Decision != "" {
            break
        }
    }
    // Fall back to rejecting on parse failure rather than auto-approving.
    if verdict.Decision != "approve" && verdict.Decision != "manager" && verdict.Decision != "reject" {
        return "reject", "could not parse LLM verdict", nil
    }
    return verdict.Decision, verdict.Reason, nil
}

Defaulting the parse-failure case to reject rather than approve is a deliberate choice — when in doubt about an automated approval, the safer default is to deny rather than to spend money. The same instinct applies to most LLM-driven gates: design the parse failure to fail closed.

The Three Outcome Tasks

The three terminal tasks consume the decision and reason and act on them — write to a database, post to Slack, send an email, etc. They look the same as any ordinary functional task; the LLM is gone by the time control reaches them:

func (svc *Service) AutoApprove(ctx context.Context, flow *workflow.Flow, expense Expense, reason string) (err error) {
    // Mark approved, notify requester, write audit log.
    return nil
}

func (svc *Service) RouteToManager(ctx context.Context, flow *workflow.Flow, expense Expense, reason string) (err error) {
    // Post to the manager's Slack channel; the manager's approval comes back
    // through a separate flow that Continues this one.
    return nil
}

func (svc *Service) RejectWithReason(ctx context.Context, flow *workflow.Flow, expense Expense, reason string) (err error) {
    // Email the requester with the reason.
    return nil
}

Running the Workflow

From any microservice on the bus:

client := foremanapi.NewClient(svc)
outcome, err := client.Run(ctx, expenseapi.ExpenseTriage.URL(), map[string]any{
    "expense": expense,
}, nil)

outcome.Status is the terminal flow status (completed, failed, interrupted, …). outcome.State["decision"] is the LLM’s verdict. outcome.State["items"] is the full conversation, available for audit. A tool-calling loop can outlast a single request’s time budget, so rather than blocking on Run, bridge it to a bounded request by launching with Create and long-polling with Poll.

Because every step of ChatLoop is durable, you get three things for free:

  • Crash recovery. A restart mid-loop resumes from the last completed step.
  • Auditability. foremanapi.NewClient(svc).History(ctx, flowKey) returns the full step-by-step trace — every tool call, every LLM response, every input/output state diff.
  • Human-in-the-loop continuation. If RouteToManager posts to Slack and the manager responds tomorrow, a follow-up call to Continue resumes the conversation with the manager’s input appended to items — the LLM picks up where it left off, with full context.

Where to Next

  • LLM integration — the full API surface for Chat, ChatLoop, ChatOptions, provider switching, and mocking.
  • LLM tooling — the conceptual model: OpenAPI as the tool description, internal vs. external delivery paths (llm.core vs MCP portal), end-to-end authorization.
  • Chatbox example — a worked example of a custom provider implementing the Turn contract.
  • Credit Flow example — the same workflow patterns (fan-out, fan-in, subgraphs, goto loops) without the LLM, for when you need workflow machinery but not a model.
Backward Building Agentic Workflows Building a Microservice Forward