Building an LLM Workflow

This guide walks through building an end-to-end agentic workflow that uses an LLM to make a decision over your microservices. It picks up where Building Agentic Workflows leaves off and adds the LLM layer.

By the end you will have:

  • The LLM machinery added to your app — llm.core, a real provider, your API key.
  • A microservice handler making a synchronous Chat call.
  • A functional endpoint authored as an LLM tool, exposed automatically through its OpenAPI document.
  • The same flow promoted to a durable ChatLoop step inside a parent workflow that branches on the LLM’s verdict.

The Worked Example

You’re building expense report triage. A user submits an expense; the system needs to decide one of three outcomes:

  • Auto-approve if the request is small, in-policy, and the requester has budget.
  • Route to manager if it’s borderline — over a soft threshold, or unusual for the category.
  • Reject if it clearly violates policy.

The decision is delegated to an LLM that has access to three tools:

ToolPurpose
policy.example/checkChecks whether the expense conforms to the corporate policy.
budget.example/remainingReturns the requester’s remaining budget for the relevant category.
expenses.example/similarReturns recent comparable expenses for context.

The LLM is instructed to end with a structured JSON verdict; a downstream task parses it and the workflow branches accordingly.

The shape, drawn the same way as creditflow:

graph LR
    classDef task fill:#32a7c1,color:#f4f2ef,stroke:#434343
    classDef sub fill:#ed2e92,color:#f4f2ef,stroke:#434343
    classDef term fill:#e5f4f3,color:#434343,stroke:#434343

    _start(( )):::term --> s1
    s1["SubmitExpense"]:::task --> s2
    s2["TriageWithLLM"]:::task --> s3
    s3["Decide"]:::task -->|"approve"| s4
    s3 -->|"manager"| s5
    s3 -->|"reject"| s6
    s4["AutoApprove"]:::task --> _end(( )):::term
    s5["RouteToManager"]:::task --> _end
    s6["RejectWithReason"]:::task --> _end

You’ll build it in five steps.

Step 1: Add the LLM Services to Your App

The LLM machinery is three pieces: the orchestrator llm.core (which runs the tool-calling loop), one or more providers (claudellm, chatgptllm, geminillm, litellm), and the Foreman (because ChatLoop is a workflow). Add them all to your app in main/main.go:

import (
    "github.com/microbus-io/fabric/coreservices/foreman"
    "github.com/microbus-io/fabric/coreservices/llm"
    "github.com/microbus-io/fabric/coreservices/claudellm"
    // "github.com/microbus-io/fabric/coreservices/chatgptllm"
    // "github.com/microbus-io/fabric/coreservices/geminillm"

    "github.com/example/myapp/expense"
    "github.com/example/myapp/policy"
    "github.com/example/myapp/budget"
    "github.com/example/myapp/expenses"
)

app.Add(
    foreman.NewService(),
    llm.NewService(),
    claudellm.NewService(),

    // Your services.
    expense.NewService(),
    policy.NewService(),
    budget.NewService(),
    expenses.NewService(),
)

Add additional providers only if you want to swap models at call time — provider is chosen per Chat call, not globally.

Configure the provider’s API key in config.local.yaml (git-ignored). The key is keyed by the provider’s hostname:

# config.local.yaml
claude.llm.core:
  APIKey: sk-ant-your-key-here
# chatgpt.llm.core:
#   APIKey: sk-your-openai-key
# gemini.llm.core:
#   APIKey: your-gemini-key

If you’re using ChatLoop, the Foreman also needs a database for its durable state. In production declare one shard per database:

# config.yaml
foreman.core:
  Shards: '[{"index":1,"dsn":"root:root@tcp(127.0.0.1:3306)/mydatabase"}]'

Locally and in tests you can skip this entirely — a LOCAL deployment falls back to a single SQLite file shard, and tests get isolated in-memory databases.

Step 2: Make Your First Chat Call

Before bringing in workflows or tools, prove the wiring works with a one-line synchronous Chat:

import (
    "github.com/microbus-io/fabric/coreservices/llm/llmapi"
)

func (svc *Service) AskLLM(ctx context.Context, prompt string) (answer string, err error) {
    items := []llmapi.Item{
        llmapi.NewMessage("user", prompt).AsItem(),
    }
    itemsOut, _, _, err := llmapi.NewClient(svc).Chat(
        ctx,
        llmapi.ProviderAny,    // let llm.core resolve a configured provider
        llmapi.ModelDefault,   // a capability-tier alias
        items,
        nil, // no tools yet
        nil, // default options
    )
    if err != nil {
        return "", errors.Trace(err)
    }
    // The assistant's reply is the last item in itemsOut.
    last := itemsOut[len(itemsOut)-1]
    if last.Message != nil {
        return last.Message.Content, nil
    }
    return "", nil
}

Three things are happening here:

  • llmapi.NewClient(svc).Chat(...) is a normal Microbus client call — same actor propagation, same tracing, same time budget as any other service-to-service call.
  • Passing llmapi.ProviderAny (or empty) lets llm.core resolve a configured provider at runtime, and llmapi.ModelDefault is a capability-tier alias. A model can be named three ways: a tier alias (ModelFast / ModelDefault / ModelSmart), a provider-family alias (e.g. opus), or a concrete vendor-prefixed model string. A single API key of any one brand is enough to run this.
  • The conversation is an ordered []llmapi.Item log. Chat returns the full conversation, including the assistant’s reply appended to your input items, so read the reply through the last item’s Message variant. To continue, append a new user item to itemsOut and call Chat again.

If this works end-to-end (response text appears, no errors), the LLM layer is correctly wired. On to giving the LLM tools.

Step 3: Author an Endpoint as an LLM Tool

Tools in Microbus are just endpoints. Any functional, web, or workflow endpoint on the bus is tool-eligible — llm.core discovers them by fetching each tool host’s OpenAPI document and reflecting the matching operation’s schema. There is no separate tool registration step.

What makes an endpoint a good LLM tool is the same set of conventions that make it pleasant for another Go microservice to call. Add the policy.example service for this guide:

package policy

import (
    "context"

    "github.com/microbus-io/errors"
    "net/http"
)

type Service struct {
    *Intermediate
}

// Check verifies that an expense conforms to the corporate policy.
func (svc *Service) Check(ctx context.Context, category string, amountUSD float64, vendor string) (withinPolicy bool, reason string, err error) {
    if amountUSD <= 0 {
        return false, "amount must be positive", errors.New("invalid amount", http.StatusBadRequest)
    }
    switch category {
    case "travel":
        if amountUSD > 5000 {
            return false, "travel expenses over $5000 require pre-approval", nil
        }
    case "meals":
        if amountUSD > 150 {
            return false, "meal expenses over $150 require receipts and approval", nil
        }
    case "supplies", "software":
        // No category-specific ceiling.
    default:
        return false, "unknown category: " + category, nil
    }
    return true, "within policy", nil
}

The endpoint is declared in policyapi/definition.go, where the per-argument descriptions live as jsonschema_description tags on the input and output struct fields:

// Check verifies that an expense conforms to the corporate policy.
var Check = define.Function{ // MARKER: Check
    Host: Hostname, Method: "POST", Route: ":443/check",
    In: CheckIn{}, Out: CheckOut{},
}

// CheckIn are the input arguments of Check.
type CheckIn struct { // MARKER: Check
    Category  string  `json:"category,omitzero" jsonschema_description:"The expense category. One of: travel, meals, supplies, software"`
    AmountUSD float64 `json:"amountUSD,omitzero" jsonschema_description:"The expense amount in US dollars. Must be positive"`
    Vendor    string  `json:"vendor,omitzero" jsonschema_description:"The vendor name. Pass empty string if unknown"`
}

// CheckOut are the output arguments of Check.
type CheckOut struct { // MARKER: Check
    WithinPolicy bool   `json:"withinPolicy,omitzero" jsonschema_description:"True if the expense complies with all applicable policy rules"`
    Reason       string `json:"reason,omitzero" jsonschema_description:"Short human-readable explanation of the verdict"`
}

Use the dedicated jsonschema_description tag rather than the jsonschema:"description=..." subtag: it is read whole, so a description may contain commas (the jsonschema tag is comma-split into directives and would truncate at the first comma). Four conventions matter for tool quality:

  1. jsonschema_description tags on the input and output struct fields. The OpenAPI generator reflects each CheckIn / CheckOut field’s tag into the LLM’s per-parameter schema documentation. Without them the LLM sees amountUSD: number and has to guess what’s in scope; with them it sees the description and constraints.

  2. Specific argument types and enums. Use a category string whose description tag enumerates the valid values rather than an open-ended string. The LLM constructs valid calls much more reliably from a small enumerated set than from “any string.”

  3. requiredClaims on the subscription. The OpenAPI document llm.core fetches is actor-aware — operations whose requiredClaims the caller cannot satisfy are omitted from the document and therefore never appear in the LLM’s tool catalog. Scope tools narrowly:

    svc.Subscribe(
    "Check", svc.Check,
    sub.At(policyapi.Check.Method, policyapi.Check.Route),
    sub.Description(`Check verifies that an expense conforms to the corporate policy.`),
    sub.Function(policyapi.CheckIn{}, policyapi.CheckOut{}),
    sub.RequiredClaims(`roles.employee`),
)

  4. Return structured errors, not free text. Validation failures return errors.New(..., http.StatusBadRequest) so the LLM (and any caller) sees a 4xx that they can act on rather than a 500 that looks like a system failure.

Repeat the same pattern for budget.example/remaining and expenses.example/similar. The exact bodies don’t matter — they’re stand-ins for whatever data sources you’d plug into.

Step 4: Call Chat with Your Tool

Now wire the tools into the LLM call. The change to step 2 is two lines:

import (
    "github.com/microbus-io/fabric/coreservices/llm/llmapi"

    "github.com/example/myapp/budget/budgetapi"
    "github.com/example/myapp/expenses/expensesapi"
    "github.com/example/myapp/policy/policyapi"
)

func (svc *Service) TriageExpense(ctx context.Context, expense Expense) (verdict string, err error) {
    items := []llmapi.Item{
        llmapi.NewMessage("system", triageSystemPrompt).AsItem(),
        llmapi.NewMessage("user", expense.Describe()).AsItem(),
    }
    toolURLs := []string{
        policyapi.Check.URL(),
        budgetapi.Remaining.URL(),
        expensesapi.Similar.URL(),
    }
    itemsOut, _, _, err := llmapi.NewClient(svc).Chat(
        ctx,
        llmapi.ProviderAny,
        llmapi.ModelSmart,    // a stronger model for decisions
        items,
        toolURLs,
        nil,
    )
    if err != nil {
        return "", errors.Trace(err)
    }
    last := itemsOut[len(itemsOut)-1]
    if last.Message != nil {
        return last.Message.Content, nil
    }
    return "", nil
}

llm.core fetches each tool host’s :888/openapi.json in parallel (carrying the calling actor’s JWT so the documents are actor-filtered), reflects each matching operation’s request-body schema into the tool format the provider expects, and exposes them to the LLM. When the model emits a tool call, llm.core dispatches it as a normal Microbus request — the calculator, policy, or budget service handles it like any other call, the result comes back, and the loop continues until the LLM produces a final text response.

The system prompt is where you constrain the LLM’s output shape — for example:

const triageSystemPrompt = `You are an expense triage assistant. Use the provided tools to gather information about the expense. After gathering enough information, end your final message with a JSON object on its own line, in this exact shape:

{"decision": "approve" | "manager" | "reject", "reason": "..."}

Do not include any other JSON in your response.`

This is what the downstream Decide task will parse.

Step 5: Promote to a Durable ChatLoop Workflow

The synchronous Chat in step 4 works, but it has two problems for production use:

  • No durability. If the request times out mid-loop (e.g., the LLM is slow on a tool round-trip), the whole conversation is lost — even the rounds that already completed.
  • No composition. The triage call is a leaf. There’s no place to attach a downstream Decide task or branch on the LLM’s verdict from inside a workflow.

ChatLoop is the durable workflow form of Chat. It performs the same logic — fetch tools, run the model, dispatch tool calls — but as a series of Foreman steps, each persisted before the next runs. See the ChatLoop graph in LLM integration.

The Parent Workflow Graph

Define a parent workflow that invokes ChatLoop as a subgraph from a task body — ChatLoop is a child workflow launched from TriageWithLLM, not a node in the graph:

import (
    "github.com/microbus-io/fabric/coreservices/llm/llmapi"
    "github.com/microbus-io/dwarf/workflow"
)

func (svc *Service) ExpenseTriage(ctx context.Context) (graph *workflow.Graph, err error) {
    graph = workflow.NewGraph("ExpenseTriage")

    graph.SetURL("SubmitExpense", expenseapi.SubmitExpense.URL())
    graph.SetURL("TriageWithLLM", expenseapi.TriageWithLLM.URL()) // runs llmapi.ChatLoop as a subgraph
    graph.SetURL("Decide", expenseapi.Decide.URL())
    graph.SetURL("AutoApprove", expenseapi.AutoApprove.URL())
    graph.SetURL("RouteToManager", expenseapi.RouteToManager.URL())
    graph.SetURL("RejectWithReason", expenseapi.RejectWithReason.URL())

    graph.AddTransition("SubmitExpense", "TriageWithLLM")
    graph.AddTransition("TriageWithLLM", "Decide")

    // First-match routing on the LLM's verdict: exactly one branch runs.
    graph.AddTransitionSwitch("Decide", "AutoApprove",    `decision == "approve"`)
    graph.AddTransitionSwitch("Decide", "RouteToManager", `decision == "manager"`)
    graph.AddTransitionSwitch("Decide", "RejectWithReason", "true")

    graph.AddTransition("AutoApprove",      workflow.End)
    graph.AddTransition("RouteToManager",   workflow.End)
    graph.AddTransition("RejectWithReason", workflow.End)
    return graph, nil
}

The SubmitExpense Task

SubmitExpense is the entry task. Its job is to translate the typed expense input into the four fields the ChatLoop subgraph expects (provider, model, items, toolURLs):

func (svc *Service) SubmitExpense(ctx context.Context, flow *workflow.Flow, expense Expense) (
    provider string,
    model string,
    items []llmapi.Item,
    toolURLs []string,
    err error,
) {
    return llmapi.ProviderAny,
        llmapi.ModelSmart,
        []llmapi.Item{
            llmapi.NewMessage("system", triageSystemPrompt).AsItem(),
            llmapi.NewMessage("user", expense.Describe()).AsItem(),
        },
        []string{
            policyapi.Check.URL(),
            budgetapi.Remaining.URL(),
            expensesapi.Similar.URL(),
        },
        nil
}

The TriageWithLLM Task

ChatLoop is invoked as a subgraph from a task body — there is no static subgraph node. TriageWithLLM reads the four fields SubmitExpense wrote, launches ChatLoop with them through the typed llmapi.NewSubgraph(flow) client, and adopts the conversation on re-entry. Only the arguments cross into the child, and only the child’s final state (items, usage) crosses back:

func (svc *Service) TriageWithLLM(ctx context.Context, flow *workflow.Flow,
    provider string, model string, items []llmapi.Item, toolURLs []string,
) (itemsOut []llmapi.Item, usage llmapi.Usage, err error) {
    conversation, usage, yield, err := llmapi.NewSubgraph(flow).ChatLoop(ctx, provider, model, items, toolURLs, nil)
    if err != nil {
        return nil, llmapi.Usage{}, errors.Trace(err)
    }
    if yield {
        return items, llmapi.Usage{}, nil // parked, ChatLoop running; leave items as they are
    }
    return conversation, usage, nil
}

ChatLoop returns the whole conversation, the prompt included, and the output itemsOut writes it to state under items, which Decide reads next. The ExpenseTriage graph gives items no reducer, so the returned conversation replaces the prompt SubmitExpense wrote. A parent that wires items to workflow.ReducerAppend would instead have to return only the items past the prompt, or the prompt would be added twice.

The yield path returns items unchanged. Every value a task returns without an error is applied to the state, on the yield path too, so returning nil there would delete the prompt while ChatLoop runs. An error return applies nothing, so the error path can return nil.

The Decide Task

Decide reads the final assistant message out of items and writes a decision field plus a reason. The switch transitions in the graph then route the flow:

func (svc *Service) Decide(ctx context.Context, flow *workflow.Flow, items []llmapi.Item) (decision string, reason string, err error) {
    // Find the last assistant message in the conversation.
    var last string
    for i := len(items) - 1; i >= 0; i-- {
        if items[i].Type() == llmapi.ItemMessage && items[i].Message.Role == "assistant" {
            last = items[i].Message.Content
            break
        }
    }
    if last == "" {
        return "reject", "no LLM response", nil
    }

    // The system prompt told the model to end with a JSON object on its own line.
    // Find the last line that parses as the expected verdict shape.
    var verdict struct {
        Decision string `json:"decision"`
        Reason   string `json:"reason"`
    }
    for i := len(last) - 1; i >= 0; i-- {
        if last[i] != '\n' {
            continue
        }
        if json.Unmarshal([]byte(strings.TrimSpace(last[i+1:])), &verdict) == nil && verdict.Decision != "" {
            break
        }
    }
    // Fall back to rejecting on parse failure rather than auto-approving.
    if verdict.Decision != "approve" && verdict.Decision != "manager" && verdict.Decision != "reject" {
        return "reject", "could not parse LLM verdict", nil
    }
    return verdict.Decision, verdict.Reason, nil
}

Defaulting the parse-failure case to reject rather than approve is a deliberate choice — when in doubt about an automated approval, the safer default is to deny rather than to spend money. The same instinct applies to most LLM-driven gates: design the parse failure to fail closed.

The Three Outcome Tasks

The three terminal tasks consume the decision and reason and act on them — write to a database, post to Slack, send an email, etc. They look the same as any ordinary functional task; the LLM is gone by the time control reaches them:

func (svc *Service) AutoApprove(ctx context.Context, flow *workflow.Flow, expense Expense, reason string) (err error) {
    // Mark approved, notify requester, write audit log.
    return nil
}

func (svc *Service) RouteToManager(ctx context.Context, flow *workflow.Flow, expense Expense, reason string) (err error) {
    // Post to the manager's Slack channel; the manager's approval comes back
    // through a separate flow that Continues this one.
    return nil
}

func (svc *Service) RejectWithReason(ctx context.Context, flow *workflow.Flow, expense Expense, reason string) (err error) {
    // Email the requester with the reason.
    return nil
}

Running the Workflow

From any microservice on the bus:

client := foremanapi.NewClient(svc)
outcome, err := client.Run(ctx, expenseapi.ExpenseTriage.URL(), map[string]any{
    "expense": expense,
}, nil)

outcome.Status is the terminal flow status (completed, failed, interrupted, …). outcome.State.GetString("decision") is the LLM’s verdict, and reading items off the same state with outcome.State.Get("items", &items) gives the full conversation for audit. A tool-calling loop can outlast a single request’s time budget, so rather than blocking on Run, bridge it to a bounded request by launching with Create and long-polling with Poll.

Because every step of ChatLoop is durable, you get three things for free:

  • Crash recovery. A restart mid-loop resumes from the last completed step.
  • Auditability. foremanapi.NewClient(svc).History(ctx, flowKey) returns the full step-by-step trace — every tool call, every LLM response, every input/output state diff.
  • Human-in-the-loop continuation. If RouteToManager posts to Slack and the manager responds tomorrow, a follow-up call to Continue starts a new flow in the same thread from this flow’s final state, with the manager’s input added to it, so the follow-up has the expense, the verdict and the conversation at hand.

Where to Next

  • LLM integration — the full API surface for Chat, ChatLoop, ChatOptions, provider switching, and mocking.
  • LLM tooling — the conceptual model: OpenAPI as the tool description, internal vs. external delivery paths (llm.core vs MCP portal), end-to-end authorization.
  • Chatbox example — a worked example of a custom provider implementing the Turn contract.
  • Credit Flow example — the same workflow patterns (fan-out, fan-in, subgraphs, goto loops) without the LLM, for when you need workflow machinery but not a model.
Backward Building Agentic Workflows Building a Microservice Forward