AI on the phone instead of the cloud: when it makes sense

What “AI on the device” actually means

An AI model is a file of learned parameters. In the cloud approach the app sends data to a server, the model computes the result there and sends the answer back. In the on-device approach the model sits on the phone (in the app or in the operating system) and computes the result on the processor, the graphics chip or a dedicated AI chip, which Apple calls the Neural Engine. Apple states the benefit plainly: a model that runs strictly on the device needs no network connection, which helps keep user data private and the app responsive.

In practice there are two routes. The first is your own specialised model, for example for image recognition, shipped inside the app. The second is a language model built into the phone’s operating system, which the app uses through a ready-made API. The main tools in October 2026:

When the model on the phone wins

• Privacy: a photo, a document or a recording that never leaves the phone never reaches the company’s server. It does not have to be secured, stored or deleted there, and it cannot leak from there. The GDPR requires processing only the data needed for the purpose (art. 5(1)(c)) and building data protection in from the design stage (art. 25). Analysis on the phone builds data minimisation into the app’s architecture itself: the company never receives data it does not need.
• Offline and no network delay: the answer does not wait for a server, so the app works in a basement, on a plane and with a weak signal. Google notes fairly that the speed of such a model depends on the phone’s hardware.
• No fee per use: cloud models are billed by tokens, pieces of text and images. For example, Gemini 3.5 Flash-Lite on the paid plan costs $0.30 per million input tokens and $2.50 per million output tokens (Google’s price list of 9 October 2026). For Gemini Nano on the phone, Google says outright that there are no extra server costs per call.
• The data does not feed someone else’s models: on the free Gemini API plan, Google notes in its price list that the data is used to improve its products (on the paid plan it is not). A model on the phone sidesteps the question, because the data is not sent anywhere.

When the cloud is better

• What the model can do: a model on the phone is smaller than a cloud one. Google lists the limits of Gemini Nano: token limits, the device’s hardware and the bounds of the model itself. Apple writes that when you need more reasoning and a longer context, you have to reach for a model on a server.
• Device coverage: built-in language models work only on newer phones. Apple Intelligence needs at least an iPhone 15 Pro, and Gemini Nano is available on selected Android models. An app for all your customers needs a plan B, for example a version without the feature or a request to the cloud.
• Storage on the phone: Apple says Apple Intelligence needs up to 8 GB of free space (up to 14 GB on iPhone 17 Pro, 17 Pro Max and iPhone Air). Your own model can be shrunk by quantisation: storing the weights in 8 bits instead of 32 makes the model about four times smaller. Store limits are rarely the problem (4 GB uncompressed in the App Store, a 4 GB compressed download in Google Play), but every megabyte is a longer download.
• Battery and heat: heavy computation warms the phone up. Apple recommends that apps cut their resource use when the thermal state is high, and the AICore service on Android has per-app limits: a number of requests and a long-term (e.g. daily) battery budget. Gemini Nano through ML Kit also runs only while the app is in the foreground.
• Model updates: a model shipped in the app changes with an app update, after store review. A model in the cloud can be swapped at once, for everyone.

The hybrid approach: phone first, cloud when needed

Both platforms describe combining models as normal practice. Google calls it hybrid inference: a request goes to the model on the phone, and to the cloud when the device does not support it or the task is too much for it. In Foundation Models, Apple lets you use its Private Cloud Compute servers or any model provider in that situation. A well-designed app processes sensitive data locally, sends to the cloud only what is necessary, and tells the user so.

Example: SomaScope

SomaScope is my app for analysing physique proportions and logging training, available in the App Store and on Google Play. The principle I built it on is written down in its privacy policy: “your body data belongs to you”. In practice:

• The photos are analysed by an AI model running on the phone. The photo files are not sent to a server for the analysis.
• The result is numbers, such as estimated proportion scores. Only those, together with the training log, can go to the cloud, and only when the optional sync in the PRO version is switched on.
• An honest limitation: the terms speak of on-device processing “where supported by the app version and the user’s device”. That is the reality of AI on phones: you have to plan up front what happens on weaker hardware.

Publishing SomaScope took me through the whole process in both stores, including the descriptions Apple and Google require of what data an app collects. More about the app itself is on the SomaScope page.

What an app with on-device AI costs

A model on the phone raises the cost of building: it has to be prepared and shrunk for phones, then tested on weaker and stronger devices. In return, the server cost per use disappears, which pays back quickly with many users. A cloud model is cheaper to start with, but you pay for every call for the whole life of the app.

The starting price covers the smallest sensible scope. I give the exact, fixed price in writing within 48 hours of a free call. Details are on the mobile apps page, and I describe the general costs of an app in the guide how much a mobile app costs (in Polish).

Phone or cloud: five questions to start with

1. Is the data sensitive (photos of people, documents, health, finance)? If so, start with a model on the phone.
2. Does the app have to work without internet? Then a cloud model cannot be the only solution.
3. How broad is the knowledge the task needs? Recognising an object in a photo fits on a phone; a conversation about everything probably does not.
4. Which phones do your customers have? If older ones too, plan a version without AI or a fallback request to the cloud.
5. How many calls a day do you expect? At thousands a day, cloud token fees become a fixed cost worth working out in advance.

Prepared with the help of AI tools, checked and completed by the author. Numbers and regulations come with sources.