Why Gemini Translates Into Russian Better Than Claude and GPT

Russian is my native language. For months I have sent every Russian translation through the gemini CLI, never through Claude. Claude’s Russian is correct, but it reads like American English with Russian words. Gemini’s reads like a person wrote it.

I had a theory for why. Google has crawled the Russian web for twenty years, and owns Books and YouTube. Or maybe Google has a team that works on Russian. I wanted evidence either way, so I went looking.

Gemini is ahead, and the best test agrees

The strongest public test of translation is WMT, the yearly machine translation competition. Professional translators score each system’s output without knowing which system wrote it. In WMT25, English to Russian came out like this:

Gemini was first on its own, with a clear gap. The human reference translation landed behind most of the top machines. The same paper names Gemini 2.5 Pro the best system overall, winning 14 language pairs.

The Russian LLM Arena points the same way. Russian speakers vote blind between two answers. When I pulled its data on 31 October 2026, Google models held the top four scores out of 112 models. The margins are small, though. Nine models share first place within the error bars, including GPT-5.4 and Claude Opus 4.7.

There is one exception worth knowing. On the arena’s creative writing topic, GPT-5.4 leads Gemini 3 Pro, 1219 to 1181. So for translation, Gemini is clearly ahead. For writing original Russian text, it is less clear.

But a year ago, Claude was ahead

This is where my theory broke. In WMT24, the year before, English to Russian looked different:

Gemini 1.5 Pro was behind Claude. Google already had its search index, Books and YouTube in 2024. So those alone cannot be the reason Gemini pulled ahead in 2025.

What Google says it changed

Google published a language mix only once, in the PaLM 2 report from 2023. Russian was 8.73% of the non-English web data, third after Spanish and Chinese. The report also says PaLM 2 trained on parallel text, the same content in English and another language, and that this “ingrains an inherent ability to translate”. Google has not published a mix for any Gemini model.

The Gemini 2.5 report explains the jump directly:

This transformation stems from a holistic strategy, meticulously refining pre- and post-training data quality, advancing tokenization techniques, innovating core modeling, and executing targeted capability hillclimbing.

So the gain was deliberate. But the report names Indian, Chinese, Japanese and Korean as the biggest winners. It never mentions Russian.

Translation is also a product goal for Google. Google Translate started using Gemini for text translation in December 2025, and Google cited its WMT25 result when it did.

I found no sign of a team working on Russian specifically. Bard added the Russian language in 2023, while Russia itself stayed off its list of countries. Russian rater jobs exist at the usual vendors, but none of the postings I found names Google or Gemini as the client.

The web index is not unique to Google

The web corpus does not explain much either. Russian is the second largest language in Common Crawl, the free web archive that many labs train on. It is 6.3% of pages, behind only English. Every lab can get Russian web text.

Books, YouTube transcripts and Google Translate’s store of English-Russian pairs are real advantages that only Google has. I could not find numbers for any of them. They are plausible, but unmeasured.

The tokenizer is not the reason

A tokenizer splits text into tokens, the pieces a model reads. A tokenizer built mostly for English cuts Russian into many small pieces, and that can hurt quality. I measured this on 2,002 sentence pairs from the WMT20 English-Russian test set. It is the same news content in both languages.

Gemini and current GPT use almost exactly the same number of Russian tokens. Gemini’s own count came from its free countTokens API, and it matched Google’s open Gemma 3 tokenizer to the token. Against Qwen and DeepSeek, Gemini uses 13-21% fewer tokens for Russian, so the tokenizer may help there. Against GPT it does not. Claude’s tokenizer is not public, so I left it out.

There is one more clue. Gemma 3 27B uses the same tokenizer and was trained by the same company, yet it ranked 15th to 16th in WMT25, with a score of 61.7. A good tokenizer and Google’s data do not produce good Russian on their own. Model scale and training after the base model do the rest.

What I believe now

My impression was right. My explanation was wrong. Gemini is ahead on Russian translation, and the best human evaluation available confirms it. The cause looks like Google making translation and multilingual quality an explicit goal for Gemini 2.5. Russian benefits as one of the biggest languages in that effort. It does not look like a Russia-specific project, or Google’s search index, or the tokenizer.

I’ll keep sending Russian through Gemini. The ranking moved a lot in one year, so I’ll check again when WMT26 is published.

To reproduce the token counts:

uv run --with sacrebleu sacrebleu -t wmt20 -l en-ru --echo src > en.txt
uv run --with sacrebleu sacrebleu -t wmt20 -l en-ru --echo ref > ru.txt
# Count each file with tiktoken (o200k_base, cl100k_base) and with Hugging Face
# tokenizers (unsloth/gemma-3-27b-it, Qwen/Qwen3-8B, deepseek-ai/DeepSeek-V3,
# unsloth/Llama-3.1-8B-Instruct). For Gemini itself, POST the text to the
# models/<model>:countTokens endpoint.