Until very recently, on device translation was not marketed as AI.

Yet translation was the main application for applied language machine learning.

When would've thought that to solve natural language translation, one would first need to solve... natural language.

All those arguments about agents and hallucinations kind of distracted people from noticing we've accodentally built a universal translator.

It's been at least 10 years that google translate had hallucinations. Some translation simply change depending of a ponctuation mark. But peoples complain only now that they heard about AI.

Of course it's not perfect, but I agree that we didn't had a machine translation as good before.

As someone both exposed to this new wave of LLM style translation in various media, and someone who has background in translation, no we didn't.

Could you please explain briefly then why my statement is wrong? What are the fundamental challenges not addressed by LLMs today? Do you think the whole approach has insurmountable roadblocks ahead, or is it more of a matter of refinement?

Context dependant phrases, from simple pronouns to whole domain specific terms, are still randomly wrong, sometimes appallingly so. Hallucinations still happen. Auto-AI translation youtube uses is, bluntly, horrid. Any jokes, even obvious ones, are still fumbled frequently.

LLM based translation looks more convincing but requires the same level of scrutiny that previous tools did. From a workflow POV they only added higher compute costs for very questionable gains.

> Auto-AI translation youtube uses is, bluntly, horrid. Any jokes, even obvious ones, are still fumbled frequently.

Youtube auto-translations are horrible indeed, and I say that as someone that has to live with the fact that Youtube decides to badly translate titles from a language I understad to Spanish because bilingual people don't exist I suppose. But that is because they use some dumb cheap model to make the translations; probably not even a Gemini-based model.

> Hallucinations still happen. Auto-AI translation youtube uses is, bluntly, horrid. Any jokes, even obvious ones, are still fumbled frequently.

I've seen that too, but these were all dedicated translation tools and auto-translate functionality.

My benchmark is against SOTA LLMs used directly. I.e. I copy the text (or media) in question, paste directly to ChatGPT or Gemini (using the best model on basic paid tier), and ask for translation. Not always perfect, but nearly so - and they naturally ingest additional context if available - such as the surrounding text, or title/ID/URI of the document/website you're looking at, or additional explanations in the prompt - and make very good use of it. This has always been missing in dedicated tools, historically built around the mistaken assumption that translation is merely a function of input text and pair of language designators (from, to). The shorter the input, the more apparent it becomes how much context matters.

RE YouTube and such - or, like any auto-transcription in video calls I've seen - I can't explain that by anything other than service providers cheapening out on this.

> From a workflow POV they only added higher compute costs for very questionable gains.

Regarding the costs - I imagine they may be an issue at scale, but for regular use (on-demand translation of individual passages, documents, recordings), it feels like it shouldn't be that noticeable anymore. You don't need to run GPT-5 for everything, some models you can run client-side already seem decent enough, and they keep improving.

> LLM based translation looks more convincing but requires the same level of scrutiny that previous tools did.

That's fair. Ultimately, if you don't know both languages, you can only trust the translation as much as you trust the translator (human or otherwise). We'll have to get a feel for this as much as we did with Google Translate, et al. In my experience, whenever I can verify them, results from LLMs are already vastly superior to prior art.

--

Tangent, and why I started considering LLMs as solving universal translation in the first place: 6 months ago, when I needed to talk with someone with whom I had zero language overlap, I tried several well-known translation apps (notably Google and Samsung), and none could manage - but then, on a whim, I just asked ChatGPT (in "advanced voice" mode) to "play a game" where it listens in and repeats whatever was just said in language A, but translated to language B, and vice versa -- and it worked flawlessly on first try.

On device or not, it's a transformer model, so some view it as tainted.

I don’t want nor need on device translation enabled by default. I’ve gone without it for the three decades in which browsers have been around. I’m sure it’s brilliantly useful for some people. A one time ‘would you like to enable AI on startup’ for at least years with user profiles that are significantly old would at least be a show of good faith.

It literally breaks YouTube