Local inference can keep a model’s computation on equipment you control, but privacy depends on the whole application. A local model can still receive content from cloud services or trigger an agent tool that sends information elsewhere. Check model downloads, telemetry, plugins, remote fallback, logs and backups separately.
The local network is a boundary to manage
Ollama’s documented default bind address is loopback. Exposing an endpoint to other devices changes who may reach it. Use the authentication and network controls appropriate to the backend, and restrict access to intended clients. A service reachable from the home network should not become publicly reachable by accident. [Source 3]
Power and heat
Measure whole-system input power under representative workloads if you need an operating estimate. A GPU’s rated power does not include the entire computer, and measured peaks do not describe average use. Cooling, standby consumption and network equipment may contribute additional demand. Follow equipment ratings and suitable electrical and thermal conditions.
The node planner uses a transparent scenario: sum of the entered average node watts × hours per day × days ÷ 1,000 = kilowatt-hours. Optional energy cost is kilowatt-hours × your entered price per kilowatt-hour. Inputs are user estimates, not measured benchmarks or current electricity prices. The calculation excludes capital cost, maintenance, cooling and standby outside the chosen hours.
Household reliability
Set expectations for sleeping laptops, updates, changing Wi-Fi quality and other users of the same machines. A node that is present only occasionally may suit batch work better than an always-available service. Define how to stop jobs, recover after a restart and retire an old model or credential.
Observability
Track request volume, queue length, failures, response times and which backend handled the request. Redact prompts or responses when their storage is unnecessary. Keep enough detail to compare topologies without turning logs into a second store of confidential data. Give the operator a clear way to disable a backend and inspect fallback behavior.
A decision before expansion
Compare the cluster with an adequately configured single machine and the cloud option relevant to your workload. Include time spent maintaining it, task quality and privacy requirements alongside energy. No universal cost saving or performance gain follows from adding machines. Expand only when measured results or a concrete operational requirement justify it.