An On-Device Assistant That Never Makes Up a Number

What was at stake

In a money app, one invented amount breaks trust in all the others.

Finely has an assistant. You type a question about your money, such as “How much did I spend at Carrefour last month?”, and it answers with a sentence and a small card of figures.

A language model is good at understanding the many ways people ask. It can also produce a figure that looks right and is not. In a finance app, that mistake must never happen. And not every iPhone can run Apple’s model.

My responsibility — the whole assistant. Finely is my own app. I designed and built the assistant: what the model is asked, the rules that complete and check each question, and the calculations behind every answer.

The model reads, Finely counts

The work is split in two. Apple’s on-device model (Foundation Models, the language model behind Apple Intelligence, which runs on the iPhone itself) only works out what is being asked. Finely’s own code does everything else: it reads the period, checks that the question is about money, and calculates every amount from the transactions stored on the phone. The assistant sends nothing off the iPhone.

1. Choose who reads the question

When Finely sets up the chat, it asks the system whether the on-device model is available. It is not on an iPhone that cannot run Apple Intelligence, when Apple Intelligence is turned off, or while the model is still getting ready. In that case, plain rules read each question instead: known phrases in English and French, the shop names in your own transactions, and a list of well-known shops built into the app.

Both readers produce the same thing and hand it to the same next steps, so the figures in an answer never depend on which one read the question. The rules simply understand fewer ways of asking.

The chat when it opens, with suggested questions. Tapping a suggestion skips the reading step: it is already a question Finely understands.

2. Ask the model for two things only

Finely asks the model which of Finely’s kinds of question the message is, and which shop it names, if any.

The reply is not free text. Apple’s framework makes the model fill in a fixed form with two fields: a choice from a list of twenty kinds of question, and a shop name. Eighteen of those kinds are questions Finely can answer, such as spending at a shop, what is left this month, or upcoming charges. The last two are “not about money” and “unclear”. The form has no field for a period or an amount.

The model sees the new message, plus a short note of the last three questions and how each was read. That helps a follow-up like “And last year?” stay about Carrefour. It never sees a transaction, a balance, or an earlier answer. Each question starts a new session, so the model remembers nothing on its own. Finely also asks for its single most likely reading, rather than a varied one, so the same sentence gets the same reading.

The reading is cleaned up before anything relies on it:

if the message is longer than 2,000 characters:
  let the rules read it
otherwise:
  ask the model what kind of question it is, and which shop it names
  if the model fails for any reason:
    let the rules read it instead
  if the model answers "unclear":
    let the rules have a second look
  if the shop it names is "none", or shorter than three letters:
    drop it

No real money question runs to 2,000 characters, and a longer message could overflow what the model can take in at once.

3. Complete the question and check it

The period comes from the words, read by Finely. The first version asked the model for the period too, from a short list. Testing on an iPhone showed it guessing, and answers came back for the wrong period. Finely now reads the period itself, with rules that combine a number, a unit, and words like “last” or “this”, in English and French: “last month”, “the last 8 months”, “in March”, “2024”. The same question now gets the same period, with or without Apple Intelligence.

Follow-ups keep their subject. When a message names a period but nothing Finely can place, it keeps the subject and the shop of the question before. A question about spending at a shop that names no shop takes the last one.

Finely decides what is on topic. The model does not always stay on topic, so Finely checks for itself:

if the message asks what to invest in, or what to buy or sell:
  refuse
otherwise, if it has no money word, currency sign, category, or known shop,
  and it is not a follow-up that only changes the period of a money question:
  refuse

4. Calculate from the transactions

Only this last step touches money. It reads the same list of past and upcoming money that every Finely screen reads (see The Ledger Behind Finely), and adds it up with exact decimal numbers, so no amount drifts by a cent. For the Carrefour question:

take the money already spent in the period, leaving out upcoming charges
keep the payments whose label names the shop
if none are left:
  use the whole history instead, and say so
total = the sum of those payments
answer with the total, the number of payments, and the average

A question about Carrefour also counts payments labelled “Carrefour Market”: Finely reads the shop’s name in each label the same way it reads it in a question.

The sentence around the figures is Finely’s too. It is written in advance, in English and French, with blanks for the figures: “You’ve spent 340,00 € at Carrefour last month.” Apart from the shop’s name, no text from the model reaches the screen. With Hide amounts turned on, the answers hide their figures too.

The answer, from a screen recording of Finely made with sample data. Watch the 14-second recording of the question being typed and answered. The screen does not show who read the question. Finely’s rules read this exact sentence as spending at Carrefour, last month, and a test checks it.

When a question does not fit

Testing what decides the answer

The tests cover everything that decides an answer: the rules, the period reading, the topic check, follow-ups, and the calculations. They use Swift Testing, made-up transactions, and a fixed “today”, so the result never depends on the day they run.

record four Carrefour payments:
  40 and 35 last month, 500 this month, 1,000 three years ago
ask how much was spent at Carrefour last month
check that the answer says 75, and not 1,575
check that it says "last month"
check that it counts 2 payments, not 4

The tests never call the model, so how well it reads questions is not covered by automated tests.

Trade-offs

• A fixed list of questions. The assistant answers eighteen kinds of question, not anything at all. Outside them, it says it cannot match the question and offers suggestions, instead of guessing.
• The model can still misread a question. The figures would then be right, but for the wrong question. Answers say in words what they calculated, so a misreading shows.
• Rules need upkeep. Every new way of asking, in either language, can mean a new rule.