A user left a review on the App Store. The Polish translation had errors, and in a few places the app showed English text where Polish should have been.
That is how I found out. Not from a test. Not from CI. From a review, after the build shipped.
Everything else in the release is automated
Version bumps, lint, tests, checks. All of it runs without me. GitHub Actions watches for a tagged release commit and kicks off the builds. When they finish, the artifacts go to Cloudflare, and the landing page links to them directly.
The only manual step left is release notes, and even those are half automated. They get generated in the app repo and I paste them into the landing repo by hand.
So the pipeline is in decent shape. Localization is the hole in it.
Adding one string means opening five files
Here is the whole problem, and it is not clever.
I add a key to the English ARB file:
{
"checkoutButtonLabel": "Continue to checkout"
}Then I open app_pl.arb and add it. Then app_de.arb. Then app_uk.arb. Then app_es.arb. One string, five files, and no step in that process checks whether I got all five.
That's the actual pain. Not an edge case, not a rare failure. Just typing the same key into the same shape of file five times and hoping I did not skip one.
Then the translations. I paste the new strings into an AI, get them back, and paste each one into the right file. Add three keys and you do that three times over. And then you have output from a model in five languages, at least two of which you cannot read, and nothing has checked it.
The build will not save you here
You would expect a missing translation to be loud. It is not.
Run flutter gen-l10n with a locale that is short a few keys and you get this:
"pl": 12 untranslated message(s).
Then it exits successfully and the build continues.
It is worth being precise about this, because the usual version of this complaint is wrong. Flutter does report the gap. It prints a count, and if you set untranslated-messages-file in l10n.yaml it will write out a JSON report listing every missing key per locale. The information exists.
What it does not do is stop. Look at what the generator does when a message is missing for a locale, in packages/flutter_tools/lib/src/localizations/gen_l10n.dart:
localeWithFallback = _templateArbLocale;The missing Polish string falls back to the template locale. It renders in English. The app compiles, runs, and looks fine to me, because I read English and I wrote the template.
That is the review I got.
A count is not a check
Say you do read the count, and you fix all twelve. You still know nothing about:
- A string that exists and is wrong. Present in the file, so it is not untranslated.
- A broken placeholder.
{count}came back as{Anzahl}because the model translated the variable name. The count does not look at placeholders. - Missing plural forms. Ukrainian needs
one,few,many,other. English needs two. If the model gave you two forms for Ukrainian, the key is present and the count is happy. - The strings that are not in ARB. Your app's display name and every iOS permission prompt live in
ios/Runner/<lang>.lproj/InfoPlist.strings, not in your ARB files.gen_l10nnever opens them. So you can have a perfect ARB report and still ship a camera permission dialog in English.
So the count answers "did someone type something here." It does not answer "is this correct."
Three things you can do today without any new tool
These all work. I used the first two for a while.
1. Turn on the report. In l10n.yaml:
untranslated-messages-file: l10n_missing.jsonYou now get a file listing every missing key per locale instead of just a number. Catches absent keys. Catches nothing else.
2. Diff the keys yourself. ARB is JSON, so this is short:
import json, pathlib
l10n = pathlib.Path("lib/l10n")
template = json.loads((l10n / "app_en.arb").read_text())
keys = {k for k in template if not k.startswith("@")}
for f in sorted(l10n.glob("app_*.arb")):
other = json.loads(f.read_text())
missing = keys - {k for k in other if not k.startswith("@")}
if missing:
print(f.name, sorted(missing))Run it before you tag. It is twenty lines and it will catch the case that got me.
3. Fail the build in CI. Wire the report from step 1 into a check that exits non-zero when the file is not empty. This is the strongest of the three, and the one I would set up first if you only do one.
The limits are the same for all three. They compare key sets. They do not look inside the values, they do not understand ICU, and they only know about ARB.
If you ship ARB only and you read every language in your app, that is genuinely enough. Set up the CI check and stop reading here. It stops being enough at the point where the values themselves need checking, or where strings start living outside ARB: the InfoPlist.strings above, a marketing site, a web dashboard sharing the same copy.
What I ended up building
I wrote StringLane because I wanted the un-automated part of my release to stop being un-automated.
It opens a project folder and shows every locale side by side. Adding a key adds it across all of them at once, which removes the five-files problem outright. Missing keys, broken placeholders and ICU errors are flagged as you type. It reads your ARB files and the .strings your permission prompts live in, plus .xcstrings, Android XML and i18next JSON when your product reaches that far, all in one view. Edits write straight back to your source files in their original format, so the git diff shows only the lines you changed.
Translation is bring your own key, or a local model with no key at all. There is no server in the middle. The point is not that an AI fills the gaps, because that part is easy now. The point is that you can see what it produced before it ships.
It is a one-time license, and the trial is the full app for 14 days with no signup. Download it here if you recognised any of this. The five files, or the part where a model hands you back a language you cannot read.
If you would rather write the CI check, write the CI check. It is a good use of an afternoon, and it would have caught my Polish review.