On-Device vs Cloud Models
Rephlo can run AI models on your own computer or send your request to a cloud model. Both appear in the same model picker and answer in the same chat window, which makes it easy to assume they're interchangeable.
They aren't. This page explains the difference in plain terms, so you can pick the right one — and shows you how to get most of the privacy of local processing and the capability of a large cloud model at the same time.
The short answer
An on-device model is a much smaller AI than a cloud model. It runs on your laptop, so it has to fit in your laptop's memory. A cloud model runs in a data center on hardware that costs more than a car.
Expect an on-device model to be genuinely useful for everyday text work, and expect it to struggle where a cloud model wouldn't. It is not a smaller-but-equal version of the same thing — it is a different tier of capability.
Why the gap is so large
AI models are measured in parameters — very roughly, the number of internal adjustable values the model learned during training. More parameters generally means more capability, and it also means more memory.
Rephlo's on-device catalog ranges from about 2 billion to 14 billion parameters, because that's what fits comfortably on a personal computer. Large cloud models are tens to hundreds of times larger than that.
That gap shows up as a difference in judgment: following a long, layered instruction; holding a complicated document in mind while answering about it; noticing that your question contains a false assumption. Smaller models do the obvious thing well and the subtle thing unreliably.
Side by side
| On-device model | Cloud model | |
|---|---|---|
| Where your text goes | Nowhere — it stays on your computer | To the provider serving the model |
| Works offline | Yes | No |
| Cost per message | None | Credits, or your own provider bill with BYOK |
| Speed | Depends on your hardware | Consistent, generally faster |
| Reasoning on hard problems | Limited | Strong |
| Long documents | Limited | Strong |
| Web search | No | Available with providers that support it |
| Reading images | No | Available with vision-capable models |
| Reading attached documents | Yes — PDFs, text, and Office files | Yes |
| Setup | One-time download of several GB | None |
| Availability | Pro, Enterprise, or a perpetual license | All plans |
What on-device models are good at
These are the jobs where the gap barely shows:
- Rewriting and rephrasing — tone changes, shortening, making something more formal
- Fixing grammar and spelling
- Summarizing short to medium text
- Simple extraction — pulling names, dates, or list items out of a paragraph
- Quick translation of everyday text
- Working offline — on a plane, in a secure facility, or with no connection
- Anything confidential enough that you'd rather it never left the machine
Where they struggle
Be prepared for weaker results here:
- Complex reasoning — multi-step logic, math, careful analysis
- Long documents — a small model's working memory fills up quickly
- Code generation beyond short snippets
- Long, layered instructions — a small model tends to follow the first few and drift from the rest
- Nuance and factual precision — smaller models are more prone to stating something confidently and wrongly
Two hard limits worth knowing up front
No web search. Because on-device models run entirely offline, they can't look anything up. Their knowledge stops at their training data. A web-search connector is in development; for live information today, use a cloud provider that supports it.
No vision. On-device models can't look at images. If you attach one, Rephlo reads any text out of it on your device and passes that along instead — useful, but not the same as a model actually seeing your chart. See How Rephlo Reads Your Files.
The best of both: redact locally, then use a cloud model
This is the setup we'd recommend for most people handling sensitive work, and it's the one most users don't realize exists.
You don't have to choose between "powerful but my data leaves" and "private but limited." Rephlo can strip personal information out of your request on your own computer, and only then send it to a large cloud model. The cloud model does the hard thinking; the personal details never make the trip.
To set it up: Settings ▸ Privacy ▸ Remove personal info before cloud requests, then choose which categories to redact.
Two layers do the detecting:
- Pattern matching, built in and always available, catches things with a predictable shape — email addresses, phone numbers, SSNs, and full card details.
- An on-device AI model, downloaded on demand (about 1.16 GB), catches things that have no fixed shape — names, dates of birth, addresses, and ID numbers like driver's licenses and passports.
Until you download that second model, the categories that depend on it are shown as unavailable, so you're never misled about what's actually being removed.
Be realistic about what this gives you. Redaction meaningfully reduces accidental exposure, but it is detection, not a guarantee — it cannot catch every case. If your content is sensitive enough that a miss would be unacceptable, use one of the stronger options:
- Turn on Block send if redaction is uncertain, and Rephlo holds the request back rather than risk sending something it isn't sure about.
- Turn on Force Local for Sensitive, and requests marked sensitive are handled by an on-device model, so they never reach a cloud provider at all.
See Privacy & Data for every setting in detail.
So which should I use?
| Your situation | Use |
|---|---|
| Everyday rewriting, tone, grammar | On-device is plenty |
| No internet connection | On-device |
| Content that must never leave the machine | On-device, or Force Local for Sensitive |
| Sensitive content, but you need real capability | Redact locally, then a cloud model |
| Hard reasoning, code, long documents | Cloud |
| You need current information from the web | Cloud, with a provider that supports search |
| You need the model to look at an image | Cloud, with a vision-capable model |
| Watching your credit spend | On-device costs no credits |
You don't have to commit. Switching models is a dropdown, so a reasonable habit is to draft with an on-device model and escalate to a cloud model when an answer isn't good enough.
Common questions
Is an on-device model the same one Rephlo uses in the cloud? No. They're different models entirely, from different families and at very different sizes.
Why is my on-device model slow? Speed depends on your hardware — memory and processor. Use Check my machine on the On-Device Models screen to see what your computer can comfortably run, and prefer a smaller model if a larger one is sluggish.
Do on-device models cost credits? No. Once downloaded, they run on your hardware at no per-message cost. The feature requires a Pro, Enterprise, or perpetual plan.
Does an on-device model make my data private automatically? Your content stays on your machine in local mode. That's a property of processing locally, not a compliance certification — if your work is regulated, contact us at [email protected].
Can I use OCR with an on-device model? Yes. Reading scanned documents runs locally regardless of which model you pick, so a scanned PDF works with an on-device model just as it does with a cloud one.
Related pages
- On-Device Models — the catalog, downloads, and memory settings
- Models & Inference — how Rephlo chooses between cloud and on-device
- Privacy & Data — redaction, routing, and local processing
- How Rephlo Reads Your Files — attachments, Spaces, scanned documents, and images