Each section below summarizes one paper's reported method, experiments, and limits from its full review. The studies use different tasks and metrics; their results are not directly comparable.
arXiv:2610.38788
Gold vs.~Translation: Are We Measuring Language, Knowledge, or Locality in Translated Benchmarks?
This paper evaluates whether machine-translated MMLU benchmarks are valid proxies for native-language benchmarks. Across 29 instruction-tuned LLMs, seven native-versus-translated pairings, and five Nordic and Baltic languages, it finds that translated benchmarks are most informative when their source content overlaps with the native target, especially for curriculum-based exams. Proxy validity is weaker for locally anchored knowledge: SwedishFacts shows low ranking agreement and no significant incremental predictive value in the main specification. The study therefore supports using translated benchmarks selectively, with native evaluations remaining important for local factual knowledge and other targets not represented in the source benchmark.
How it works
The comparison covers Estonian, Swedish, Danish, Norwegian Bokmål, and Lithuanian. Translated proxies come from EU20-MMLU for Estonian, Lithuanian, Swedish, and Danish, and from m_mmlu for Norwegian Bokmål. The native sources are Exam-ET, INCLUDE-ET, Skolprov, SwedishFacts, DanishCitizenTest, NRK-Quiz, and INCLUDE-LT. The evaluation uses 29 instruction-tuned LLMs across four size categories and runs through the LM Evaluation Harness with custom native-benchmark tasks.
What the paper evaluates
The main comparison includes Exam-ET, INCLUDE-ET, Skolprov, SwedishFacts, DanishCitizenTest, NRK-Quiz, and INCLUDE-LT, each paired with a same-language DeepL-translated MMLU proxy. The broader evaluation also uses G-MMLUX, English G-MMLUX for sensitivity control, Belebele for language proficiency, GlobalMMLU-Lite for annotation calibration, and Native-500 for native-language annotation validation. Ranking agreement is strongest for Skolprov (τ_b=.89, top-5 overlap 5/5), Exam-ET (.88, 5/5), INCLUDE-LT (.87, 5/5), and NRK-Quiz (.85, 4/5). DanishCitizenTest reaches .82 with 3/5 top-5 overlap, INCLUDE-ET reaches .72, and SwedishFacts reaches only .51 with 3/5 top-5 overlap.
Limit to keep in view
The regression analyses use only 29 models, so effect sizes and confidence intervals describe uncertainty over the evaluated model sample rather than all possible LLMs.
arXiv:2611.04303
FlowTranslator: Training-Free In-Image Machine Translation with Flow-Based Editing
FlowTranslator is a training-free framework for in-image machine translation that combines OCR, frozen LLM translation, and region-masked flow editing from a frozen text-to-image model. Its key idea is to solve the small-text-region ignition failure of ordinary flow editing with Region Coupling Release (RCR), which partially decorrelates target-branch noise, and Coupling Expansion (CE), which amplifies the emerging edit displacement. On PRIM, it reports average BLEU 20.6 and COMET 57.2, exceeding UniTranslator's 12.8 and 50.7; on whole-image ScenePair English-to-Chinese translation, it reaches 40.08% exact-match accuracy.
How it works
The pipeline first extracts text boxes and strings with OCR, then asks a frozen LLM to produce matched source and target scene descriptions. A frozen VAE encodes the source image, and the OCR boxes are dilated and rasterized into the latent mask defined by equation 1. The same frozen DiT evaluates source- and target-conditioned velocities, while equation 2 updates only masked latent positions.
What the paper evaluates
The evaluation uses TTV, PRIM, and ScenePair. TTV contains 5,122 synthetic text images built from IWSLT14/17 and evaluates De→En, En→De, Fr→En, and Ro→En. PRIM contains photographed real-world 512 × 46 text strips and evaluates En→De, En→Fr, En→Cs, En→Ru, and En→Ro with 340 images per direction. ScenePair contains 1,280 real scene photographs and evaluates whole-image En→Zh translation with fixed text boxes and target translations. For TTV and PRIM, edited text is read back with EasyOCR v1.7, scored with lowercased sacreBLEU and COMET, and compared with source-image SSIM where reported. ScenePair uses PP-OCRv6 exact-match accuracy and SSIM.
Limit to keep in view
The method inherits the multilingual rendering capabilities of the frozen Z-Image-Turbo backbone; the paper reports that Cyrillic rendering fails on PRIM En→Ru, producing near-Latin substitutions and only 4.6 BLEU.
arXiv:2610.41232
TRIDENT and SHAKTI: A Framework and Benchmark for Hindi AI-Text Detection
TRIDENT detects AI-generated Hindi text by translating a passage into another language and back, then measuring how much the reconstructed text differs from the input. Its premise is that generated text tends to remain more consistent through this transformation than human writing. The paper introduces three variants: TRIDENT-S uses a fixed similarity threshold, TRIDENT-D estimates a threshold with a two-component Gaussian mixture, and TRIDENT-C uses logistic regression over multiple RTT-derived features. The authors also present SHAKTI, a Hindi benchmark spanning four genres, multiple LLM families, and five adversarial transformations.
How it works
TRIDENT first translates an input from source language L1 into intermediate language L2, then translates the result back into L1. A sentence encoder represents the original and reconstructed text, allowing comparison of their consistency and drift. The paper motivates this operation as a controlled perturbation: it aims to retain meaning while changing lexical, syntactic, or stylistic form. The authors use several RTT-derived signals rather than relying only on a single surface feature.
What the paper evaluates
The paper reports experiments on SHAKTI and publicly available datasets, with evaluations described across Hindi and seven additional languages: Urdu, Arabic, Russian, Chinese, English, and Korean. The stated comparisons include conventional statistical, rewriting-based, and supervised detection methods. The paper also describes cross-domain and cross-generator evaluation, adversarial perturbation tests, unseen-LLM generalization, and an analysis of translation-model choice.
Limit to keep in view
Detection performance depends on the translation model; the paper itself reports sensitivity to this choice, and the supplied evidence does not give numeric comparisons for the alternatives.
arXiv:2610.40516
How I Aligned Your Model: Scaling Up Multilingual Safety Training with Templates
The paper introduces MSAFETEMP, a template-based pipeline for scaling multilingual safety training and evaluation across low-resource languages and culturally specific harms. It extracts harm-bearing phrases into typed slots, translates and paraphrases the surrounding template, fills slots with harmful or benign entities, and adds culturally localized variants. The resulting corpus covers 44 training languages and 60 evaluation languages. Experiments show that English-only safety evaluation can substantially underestimate multilingual attack success, while training with MSAFETEMP reduces attack success across several open models, although supervised fine-tuning can increase over-refusal and reduce general capability.
How it works
The pipeline begins with 200 standard HarmBench behaviors reserved for evaluation and 2,500 harmful WildGuardMix prompts for training. The authors remove contextual and copyright categories from HarmBench because they are not defined solely by the prompt. For each seed, a generator extracts single words or short phrases that make the prompt harmful and replaces them with typed slots. A first judge checks whether the template remains valid and self-contained.
What the paper evaluates
The evaluation uses HarmBench and WildGuardMix with GPT-OSS-Safeguard-120B judging greedy target-model responses. Attack success rate (ASR) is the share of harmful prompts receiving unsafe compliance, and over-refusal (OR) is the share of DIRECT benign prompts that are refused; lower is better for both. The main comparison uses original English seeds, translated TR variants, culturally localized PH variants, and benign OR controls. The paper also evaluates jailbreak-wrapped HarmBench prompts with 300 prompts per language.
Limit to keep in view
The dataset and headline evaluations rely heavily on automated LLM generation and judging; the paper reports a 5,000-response judge-model check but does not provide a full judge-model ablation.
Read the full reviews
- Gold vs.~Translation: Are We Measuring Language, Knowledge, or Locality in Translated Benchmarks? · arXiv:2610.38788
- FlowTranslator: Training-Free In-Image Machine Translation with Flow-Based Editing · arXiv:2611.04303
- TRIDENT and SHAKTI: A Framework and Benchmark for Hindi AI-Text Detection · arXiv:2610.41232
- How I Aligned Your Model: Scaling Up Multilingual Safety Training with Templates · arXiv:2610.40516
Source note: findings and limitations above come from the linked full paper reviews. Follow each original arXiv paper for the authors' complete methods and results.