Python-Backed Microservices

A Microbus microservice can delegate its compute to Python while keeping every framework concern - typed endpoints, tracing, ACLs, configuration, durability - on the Go side. The pyvenv library spawns a long-lived Python subprocess in-process, the Go handler calls into it over local pipes, and upstream microservices never see anything but a normal Microbus endpoint. Dedicated coding agent skills scaffold the pattern from a few prompts.

Scaffold a New Python-backed Microservice

Instruct your coding agent to:

Create a new Python-backed microservice “embedder”

The agent lays down the embed directives for the *.py sources and requirements.txt, the OnStartup / OnShutdown venv lifecycle, the liveness callback that activates manual subscriptions once the venv is ready, the recovery wiring on subprocess death, and the empty service.py and requirements.txt files ready for content.

embedder/
├── embedderapi/
│   ├── definition.go     # Typed endpoint definitions (the API's source of truth)
│   └── client.go         # Generated client stubs for upstream callers
├── intermediate.go       # Generated framework plumbing
├── manifest.yaml         # Endpoint, config, and metric declarations
├── mock.go               # Generated mock (skips the venv)
├── mock_test.go          # Test that the generated mock honors every endpoint
├── python.go             # Embeds the Python sources and wires the venv lifecycle
├── requirements.txt      # Python dependencies, embedded into the binary
├── service.go            # Go handlers that delegate to Python
├── service.py            # Python implementations
└── service_test.go       # Tests

Add a Function Endpoint Backed by Python

Add a function Embed to embedder that takes a text string and returns a vector of floats. Implement it in Python.

The agent generates a typed Go endpoint that delegates to a Python function, plus a Python stub ready for you to fill in. Use this shape when the call reliably completes within a single RPC budget.

Add a Workflow Task Backed by Python

Add a workflow task TrainModel to embedder that takes a dataset URI and number of epochs and returns the trained model path. Implement it in Python.

The agent generates a workflow task that delegates to a Python function. Use this shape when the call has to compose into a workflow graph - with retries, fan-out, or error fallbacks - rather than fit inside a single RPC.

Deferred Subscriptions

The Python venv takes time to come online - pip install plus model loading can run for minutes on a cold start. The Python-backed endpoints are tied to the venv’s lifecycle: the microservice goes live on the bus immediately for control-plane traffic, but the Python-backed endpoints stay off until the venv reports ready. Requests load-balance to warm peer replicas in the meantime, so upstream microservices keep being served during the window.

Editor Tooling

Python source lives in .py files at the microservice’s root, embedded via //go:embed *.py. Authoring Python in real files keeps editor tooling - syntax highlighting, formatting, linting, test runners - working naturally. The embed makes the source part of the binary at build time.

In-Memory Testing

The venv does not auto-start in the TESTING deployment - the scaffolded OnStartup gates the background Start on the deployment mode, parallel to how tickers don’t fire under tests. So a test that instantiates the real Python-backed microservice (rather than its Mock) gets the microservice live on the bus with its Python-backed endpoints off, no subprocess spawned, no pip install running. Tests that want to exercise the real Python path opt in with one line: svc.StartPyVenv(ctx), which synchronously runs the lifecycle and returns when the worker is ready.

Most tests don’t want to pay even the opt-in cost - the microservice’s generated Mock shadows each handler with a plain Go function and skips the venv entirely. Upstream microservices that want integration tests alongside a Python-backed dependency use the same Mock. End-to-end coverage of the subprocess lifecycle, pip install, define, call, and recovery lives in pyvenv’s own integration tests.