Frequently Asked Question

When should I use a local model instead of a hosted one?

Four situations make the local option clearly correct.

Confidential data. Text sent to a hosted model leaves your machine. For patient records, legal documents, unpublished research, or a client's proprietary source code, that transfer may be prohibited outright, and running locally removes the question.

Offline operation. A model on local disk works with no network at all, which matters in the field, at sea, or in a hospital basement.

High-volume repetitive work. Hosted inference is priced per token; local inference costs electricity. At volume the arithmetic flips.

Reproducibility. A hosted endpoint may be updated or retired underneath you. A file on your disk is the same file next year, which matters for published research.

Against that, a model you can run on a laptop is meaningfully less capable than the largest hosted ones. The most sensible arrangement is usually both: a local model for bulk and sensitive work, and a hosted one for the problems that genuinely need the extra capability.