Six architectures that should not be confused
Compare topology, model capacity and failure behavior.
LOCAL INFERENCE / FIELD GUIDE
More machines. A deliberate architecture.
Plan local AI around the work you need to run. Compare request routing, distributed execution and the responsibilities that come with operating multiple nodes.
Open the plannerArchitecture 01 / Request routing
One endpoint.
Separate inference nodes.
APPLICATIONLocal inference request
ROUTERChoose an eligible node
NODE A / NODE B / NODE CEach runs its own model
Routing can share work. It does not automatically combine memory into a larger virtual GPU.
Plan your nodesA useful starting point
Choose the topology
for the workload.
Independent requests, a model too large for one device, and a service that must survive a node failure are different engineering problems. Start by naming yours.
Read the field guideInside the guide
Compatibility before capacity
A checklist for operating systems, backends, models and networks.
Privacy power and daily operation
An operating plan for locally controlled AI inference.