Python Integration

Some workloads have to run in Python - typically machine-learning inference, model training, scientific computing, or wrapping a library that has no usable Go equivalent. A Python-backed microservice is similar to any other microservice in that it exposes typed Go endpoints to upstream microservices over the bus, participates in actor propagation, tracing, ACLs, and durability like any other Microbus microservice, etc. It just happens to delegate implementation to a Python function via the pyvenv library.

This page covers the integration patterns: how to wrap Python in a typed Go microservice, the lifecycle that brings Python-backed endpoints onto the bus only after the venv is ready, and how to drive long-running Python work from a durable workflow.

Calling Python from Microbus

The Go-Python Boundary

Go owns every framework seam: logging, distributed tracing, configuration, downstream microservice calls, cache, request lifecycle, actor identity. Python is treated as a pure compute kernel - it sees typed JSON in, returns typed JSON out, and has no awareness of ctx, spans, client stubs, configs, or the bus at all.

Upstream microservices only ever talk to the Go handler; Python is an implementation detail behind a typed endpoint.

Architecture

A Python-backed microservice looks like a normal Microbus microservice to upstream microservices on the bus. It owns its hostname, declares typed endpoints, generates a client, and gets its interservice ACL grant derived from its source code - the same shape as any other domain microservice. An upstream microservice sends a request to the typed endpoint, the Go handler runs, and a typed response comes back. The upstream does not know - and should not need to know - that the handler delegates its compute to a Python subprocess. The Go-Python hop is implementation detail of the wrapper microservice.

Behind the typed endpoint, the microservice uses the pyvenv library to spawn and own a long-lived Python subprocess in-process. The library handles the venv directory, pip install, worker process, and JSON bridge; the microservice gets a typed Go API for calling into Python and reading results back. Nothing in the venv touches NATS, the bus, or the framework.

The relationship is 1:1: one microservice replica owns one venv subprocess. If multiple microservices need Python, each owns its own.

Deployment Pattern

The recommended deployment is one venv per upstream replica:

When a replica starts up, it constructs its own venv and starts it. Each replica routes its own request to its own subprocess. There is no stickiness between an inbound request and a specific replica.
Requests load-balance freely across upstream replicas. Identical seeding - same requirements.txt, same .py files - means any replica can serve any request.
When a replica shuts down cleanly, it releases the subprocess and reclaims its on-disk venv immediately. A crash takes the subprocess with it because the kernel cleans up the child process when the parent dies - there is no external state to reclaim.

This shape keeps the deployment story simple: scale upstream replicas to scale Python concurrency. There is no cross-replica venv sharing, no warm pool to manage, no orchestration of which call goes to which Python process. The operator’s only knob is replica count.

GPU Coordination

GPU pinning is operator territory, not framework territory. If a Python-backed microservice needs GPU exclusivity, run one replica per GPU and pin via CUDA_VISIBLE_DEVICES=N in the orchestrator’s env block (Kubernetes pod spec, docker-compose service env, systemd unit). The Python subprocess inherits the parent’s environment, so the worker sees only the assigned GPU. Set MaxWorkers to 1 for full-VRAM models; raise it only when the model is small enough to multiplex.

When Not to Use This

pyvenv is the right tool for ML inference, training, data-frame transforms, scientific computing, and any I/O-bound or GIL-releasing workload where Python has the library that Go does not. It is not the right tool for:

Microsecond-latency hot paths. Local pipe plus a JSON round-trip adds hundreds of microseconds, sometimes milliseconds. If the call has to land in microseconds, write it in Go.
Untrusted code. pyvenv is designed for first-party, version-pinned Python. LLM-generated code, user-supplied scripts, or anything the operator cannot audit belongs in a sandboxed execution service with container or Firecracker isolation, not a long-lived in-process subprocess.
Pure-Python CPU-bound work that needs process-level parallelism. The worker is one Python process. Concurrent Calls run on threads in a ThreadPoolExecutor, which is fine for GIL-releasing native work (NumPy, PyTorch, pandas, I/O) but does not help GIL-bound Python. For that case, run multiple replicas of your microservice rather than multiple workers in one venv.

Provisioning Takes Time

A fresh venv is not instant. python3 -m venv runs in milliseconds, but a real Python-backed microservice typically needs more:

pip install for a non-trivial set of packages runs from 30 seconds (a few small libraries) to several minutes (torch with CUDA, tensorflow, the full sentence-transformers dependency tree).
Model loading on first Define can dwarf the pip install when the model has to be downloaded from a hub before being held in memory. Multi-gigabyte downloads are routine for foundation models.
First-call warmup inside Python - CUDA context initialization, JIT compilation, vectorizer cache population - can add another few seconds on top.

Running any of this on the inbound request path is unacceptable; running it synchronously during startup would render the microservice unreachable to upstream microservices during the entire window. The framework’s answer is deferred subscriptions for Python-backed endpoints combined with background provisioning: the microservice goes live on the bus immediately for control-plane traffic, provisions its venv off the critical path, and brings its Python-backed endpoints onto the bus only when the venv is ready.

Because traffic for Python-backed endpoints only reaches replicas whose venv is ready, the operational recipe is explicit: always run at least one warm replica of any Python-backed microservice. A rolling restart that takes one replica cold for several minutes is harmless as long as a sibling is still warm; requests load-balance to the warm replica and the cluster behaves like a normal Microbus microservice during the window, just with one fewer replica’s worth of capacity. The 404 ack-timeout that upstream microservices would otherwise see when no replica is warm - the framework’s “no responder answered” signal, never a slow hang or a confusing 503 - is the fallback for the degenerate case where every replica is cold at once, not the normal path.

The canonical lifecycle is:

OnStartup constructs the *Venv with embedded sources, requirements, MaxWorkers, the connector as Logger, and a LivenessCallback. No subprocess is spawned yet.
OnStartup launches venv.Start(ctx) in a background goroutine via svc.Go and returns immediately so the microservice is responsive on the bus during the provisioning window.
On success, the library fires LivenessCallback(StateReady, nil) from its own goroutine. The microservice’s handler activates every "python"-tagged subscription on the connector, bringing the Python-backed endpoints onto the bus.
If the subprocess later dies unexpectedly (segfault, OOM-kill, etc.), the library fires LivenessCallback(StateDied, err). The handler deactivates the Python-tagged subs and schedules venv.Start(ctx) again in the background. Recovery is fast: the on-disk venv is reused and pip install is skipped, so only the worker spawn and define re-run.
OnShutdown calls venv.Close(ctx) to kill the subprocess and remove the on-disk venv.

Provisioning can instead be triggered synchronously in OnStartup (eager, but startup blocks for the full pip install) or behind a control endpoint (manual, useful when the venv shouldn’t come up until a human triggers it).

Function Endpoint Example

A function-style handler that delegates a single sub-second-to-multi-minute call to Python:

import (
    "github.com/microbus-io/pyvenv"
)

func (svc *Service) Embed(ctx context.Context, text string) (vector []float64, err error) {
    if svc.venv == nil || !svc.venv.Ready() {
        return nil, errors.New("venv not ready", http.StatusServiceUnavailable)
    }
    in := embedderapi.EmbedIn{Text: text}
    var out embedderapi.EmbedOut
    err := svc.venv.Call(ctx, "embed", in, &out)
    if err != nil {
        return nil, errors.Trace(err)
    }
    return out.Vector, nil
}

# service.py - lives next to service.go, embedded via //go:embed *.py.
from sentence_transformers import SentenceTransformer

model = SentenceTransformer("all-MiniLM-L6-v2")

def embed(args):
    """Embed returns the sentence-embedding vector for the input text."""
    vector = model.encode(args["text"]).tolist()
    return {"vector": vector}

The Ready guard is defense-in-depth. In normal operation the manual subscription is off the bus until the venv reaches Ready, so callers see a clean 404 ack-timeout rather than a 503. The 503 guard only fires if a caller somehow reaches the handler while the venv is starting or recovering.

Call blocks the calling goroutine until the Python function returns or the context expires. Concurrent calls dispatch to the worker’s ThreadPoolExecutor up to MaxWorkers in flight. For GIL-releasing work (NumPy, PyTorch, pandas, I/O) those threads run truly in parallel.

Workflow Task Example

When the Python call may exceed the framework’s 15-minute ctx ceiling - LLM inference, training, batch processing - drive it from a workflow task. The Foreman retries the task on its own schedule, so the task can survive across hops that exceed any single RPC budget.

func (svc *Service) Train(ctx context.Context, flow *workflow.Flow, datasetURI string, epochs int) (modelPath string, err error) {
    if svc.venv == nil || !svc.venv.Ready() {
        return "", errors.New("venv not ready", http.StatusServiceUnavailable)
    }
    in := myserviceapi.TrainIn{DatasetURI: datasetURI, Epochs: epochs}
    var out myserviceapi.TrainOut
    err := svc.venv.Call(ctx, "train", in, &out)
    if err != nil {
        // 408 = step time-budget expired; retry the task immediately, with no give-up horizon.
        if errors.StatusCode(err) == http.StatusRequestTimeout && flow.Retry(0, 0, 0, 0) {
            return "", nil
        }
        return "", errors.Trace(err)
    }
    return out.ModelPath, nil
}

flow.Retry is the heart of the long-running pattern. Gate it on HTTP status 408 (the step’s time budget expired) so only a timeout schedules another attempt; non-timeout errors fall through and surface to the workflow. Zero delays with giveUpAfter <= 0 gives an immediate retry with no give-up horizon — the task re-runs until it finally completes. (flow.Retry is the single retry primitive; the retryable condition is written explicitly in the surrounding if, and the bound is wall-clock giveUpAfter, not a count, as the workflow package reference sets out.)

Unlike a bus-mediated venv service, the in-process model has no cross-replica result handoff: if the replica that issued the call dies mid-call, the call dies with it. The workflow task retries from scratch on the next attempt. Combined with AddTransitionOnError for catastrophic failures, the workflow keeps a clear separation between “wait longer” and “give up”.

When a bounded retry horizon is appropriate - e.g. exponential backoff between 2 s and 1 min, giving up an hour after the step first ran - pass an explicit giveUpAfter:

if errors.StatusCode(err) == http.StatusRequestTimeout && flow.Retry(2*time.Second, 2.0, time.Minute, time.Hour) {
    return "", nil
}

Choosing Sync vs. Workflow

Expose Python through a function endpoint when:

The Python call reliably completes within a single RPC budget (sub-second to multi-minute).
The caller already has a synchronous shape and there is no benefit to durable persistence.

Expose Python through a workflow task when:

The Python call may exceed the framework’s 15-minute ctx ceiling (LLM, training, large batch processing).
The call needs workflow-level retry semantics (flow.Retry gated on a timeout, AddTransitionOnError) or composes into a larger graph (fan-out, subgraphs, error fallbacks).

Errors

pip install failures are usually transient (network, mirror flakes, lock contention). The microservice can call Start again after the underlying issue is resolved; the on-disk venv is reused and only pip is re-run.
define failures often leave the Python namespace partially mutated because exec aborts mid-script. The library kills the worker on a define failure, so a subsequent Start spawns a fresh process with a clean namespace.
Python exceptions propagate as Go errors with the Python exception type and message embedded in the error string (e.g. python call embed failed: ValueError: expected positive integer).
Subprocess death (pyvenv.ErrDied) indicates the worker exited unexpectedly between calls. LivenessCallback fires StateDied, the microservice deactivates Python subs, and recovery via venv.Start(ctx) is scheduled in the background. Calls in flight at the moment of death return ErrDied; the workflow task or caller is expected to retry.

Testing Without Real Python

The venv does not auto-start in the TESTING deployment. The microservice’s startup wiring gates the background Start on the deployment mode, parallel to how tickers don’t fire under tests, so a test that instantiates the real microservice gets it live on the bus with its Python-backed endpoints off and no subprocess spawned. Tests that want to exercise the real Python path opt in with one line: svc.StartPyVenv(ctx), which synchronously runs the venv lifecycle and returns when the worker is ready.

Most tests don’t even want the opt-in. They mock at the microservice’s interface layer instead: the generated Mock type shadows each Python-backed handler with a plain Go function, so upstream microservices and integration tests treat the Python-backed microservice like any other Microbus microservice without knowledge of pyvenv.

End-to-end coverage of the Python path (subprocess lifecycle, pip install, define, call, recovery) lives in pyvenv’s own integration tests; microservice tests don’t need to spawn a venv or mock one.