MobTranslate Translation Systems Research, Benchmark, and Operator Handoff

Date: 2026-07-13 UTC
Languages in scope: Kuku Yalanji (gvn) and Mi'gmaq (mic)
Primary experiment: kuku-elder43-hybrid-postedit-v2
Status: completed research benchmark; guarded deployment only
Promotion authority: Kuku Yalanji and Mi'gmaq speakers, not an automatic metric or model vendor

1. Executive ruling

There are two different questions, and the evidence gives two different answers.

1. Which custom-trained Kuku Yalanji model should be retained as the current research candidate?
   Use v21.2, the balanced-replay continuation trained in the Claude lane. It retains the synthetic treatment's performance while recovering large Bible and dictionary-usage losses seen in v21.1. This is a training-recipe result, not proof that Claude as a model vendor is intrinsically better.
2. Can an LLM improve v21.2 at inference time?
   On the 43-row, reference-blind elder-shared benchmark, both tested post-editors improved their point estimates. Codex gpt-5.4-mini was highest at 30.96 corpus chrF++, Claude Opus scored 30.13, and unedited v21.2 scored 29.36. Codex's paired gain over v21.2 was +1.52 mean sentence chrF++, 95% bootstrap CI [+0.34, +2.86]. Claude's gain was +0.77, CI [-0.18, +1.91]. Claude minus Codex was -0.75, CI [-1.57, +0.05]. Therefore the observed ordering is Codex > Claude > v21.2, but this experiment does not establish a reliable Codex-versus-Claude winner.

The defensible product architecture is therefore retrieval-first and explicitly guarded:

1. Return an exact, rights-cleared sentence/resource match where one exists.
2. Return a unique exact dictionary match for a single lexical query where its sense and direction are clear.
3. For novel text, expose v21.2 only as a labelled research draft with degeneration checks and provenance.
4. Keep LLM post-editing dark until the benchmark is larger and speakers have blind-rated the outputs.
5. Preserve alternatives and corrections; never silently turn a model output into linguistic authority.

No system evaluated here is speaker-certified. All 20,047 authored corpus pairs remain synthetic research material pending speaker review. The elder-shared set is a valuable natural-register control, but 43 rows are not a general translation benchmark.

2. What changed in the product

The guarded Kuku Yalanji research translator now designates v21.2 as its candidate of record. v21.1 is preserved as the synthetic-only ablation. The existing unrestricted GPT-backed translator was not silently replaced, and no LLM post-editor was added to the public request path.

The distinction matters:

An aggregate audit of successful historical GPT translation requests shows how much exact dictionary routing can realistically cover without inspecting or exporting user text:

These are upper-bound eligibility counts, not correctness claims. A unique string match can still have the wrong sense, inflection, dialect, or discourse role. The audit exports aggregate counts only; it contains no request or user content. No historical request exactly matched a stored usage-example sentence under the same normalization.

3. Systems under test

3.1 Custom model A: v21.1

v21.1-codex-synthetic-direct-gvn-3epoch-lr2e-5 is a LoRA continuation of the frozen v12 base derived from facebook/nllb-200-distilled-1.3B. It trained for three epochs on 16,642 leakage-audited synthetic rows and no Bible, dictionary-usage, or elder replay. Its purpose is a clean synthetic-only treatment, not a claim that Codex itself trained the neural weights.

3.2 Custom model B: v21.2

v21.2-claude-balanced-replay-gvn-3epoch-lr2e-5 uses the same frozen base and hyperparameter family. Each epoch contains 16,642 synthetic rows, 2,047 direct-Bible rows, 2,047 reference-conditioned Bible rows, and 1,428 dictionary-usage rows. No elder rows are included. Ten overlapping replay task rows, corresponding to 34 expanded training rows, were quarantined before training.

v21.2 trained for 4,155 optimizer steps versus 3,120 for v21.1. The comparison establishes the better practical recipe, but replay composition is confounded with a 33% larger update budget. A step-matched, multi-seed ablation is still required for a narrow causal claim.

3.3 Codex post-editor

The Codex lane used gpt-5.4-mini through the Codex CLI and the existing ChatGPT subscription. It received the English source, two anonymized custom-model drafts, retrieved training examples, dictionary entries, and grammar sections. It did not receive the held-out reference.

3.4 Claude post-editor

The Claude lane used Claude Code Opus at high effort through the existing Claude Max subscription. It received the same blind batch payload and task contract. Its provider-specific schema differed only because Claude rejected the draft-2020 $schema URI; the semantic constraints and exact output IDs were unchanged.

3.5 Reference-leaking oracle

For diagnosis only, a two-candidate oracle selects whichever custom draft has greater reference overlap on each row. It scores 30.10 corpus chrF++. This is an unattainable, reference-leaking ceiling and is never a deployable system. Its value is diagnostic: a post-editor can sometimes generate useful novel text rather than merely select between the two custom drafts.

4. Blind benchmark design

The frozen test is the 43-row elder-shared English-to-Kuku-Yalanji set. It contains 42 unique English surfaces because one surface is duplicated. Neither v21.1 nor v21.2 trained on these rows.

The preparation program writes two physically separate trees:

• blind/: prompt, model-visible evidence, anonymized candidates, and exact-ID JSON schemas;
• private/: references and the randomized candidate-to-system map used only during scoring.

The two candidates are randomized independently per row. The providers see only draft_1 and draft_2, not the training-lane labels. The output schema is an object keyed by the exact expected row IDs. Scoring refuses missing, duplicate, or unexpected IDs, preventing a partially completed batch from being treated as a result.

The sealed prompt hash is:

11d23261445745194efff65526bcb2bd779c5bba71fe11ea469102eaac940487

4.1 Retrieval evidence

Each row receives deterministic evidence selected by BM25 with a character-trigram tie-break:

The preparation stage may inspect private references only to enforce exclusion. It removes training examples whose normalized source or target exactly equals any held-out source or reference. There were 42 blocked source surfaces, 42 blocked target surfaces, and zero complete held-out reference surfaces in the final retrieval payloads.

This controls direct surface leakage from the project collections. It does not prove that a provider never saw the story during pretraining, and it cannot remove all partial lexical overlap. Provider pretraining exposure is an explicit residual limitation.

4.2 Provider isolation

The accepted runs used 11 batches of four rows, with three rows in the final batch. They ran from a newly created, empty temporary directory. The prompt prohibited tools, web access, and filesystem search. Claude had an empty tool set. Codex used a read-only sandbox and ignored repository instructions.

Three prohibited Codex shell commands nevertheless ran in two accepted batches. All three were confined to the empty isolated directory and returned no benchmark evidence. This is recorded as a protocol deviation. Contract sensitivity analysis shows that enforcing the declared draft decision instead of the emitted modified text raises, rather than lowers, Codex's corpus chrF++ from 30.9603 to 30.9696. The deviation did not create the measured gain.

4.3 Metrics and uncertainty

• corpus BLEU and chrF++ are computed with SacreBLEU; chrF++ uses word order 2;
• the primary paired estimand is difference in mean sentence chrF++;
• 95% intervals use 50,000 paired nonparametric bootstrap resamples with seed 42;
• degeneration gates count empty output, source copy, repeated word trigrams, and output/reference length ratio;
• exact match collapses whitespace but otherwise preserves Unicode and punctuation.

chrF++ is useful for morphologically rich, orthographically variable low-resource settings, but it is not a measure of semantic adequacy, morphosyntactic well-formedness, dialect fit, or community acceptance. BLEU is reported only as a secondary overlap metric. No automatic metric is a substitute for blind speaker review.

5. Main results

All systems continue to under-translate relative to the references. The post-editors reduce the average character length deficit slightly, but neither reaches reference length or exact match. Longer output is not automatically better; the point is that the known omission signal remains unresolved.

5.1 Paired comparisons

The interval interpretation is descriptive, not a multiple-comparison-adjusted confirmatory trial. On this set, Codex has a clear paired overlap gain over v21.2 and both post-editors beat v21.1. The Codex-versus-Claude interval nearly touches but still includes zero. A statement that Claude did better is not supported by this test; a statement that Codex is definitively the better linguistic system is also too strong.

5.2 Post-editor behavior

The confidence numbers are uncalibrated self-reports and must not be exposed as probabilities of correctness. Claude is more conservative and more contract-disciplined; Codex changes more rows and achieves the higher overlap point estimate. That is a behavioral trade-off, not a certification result.

6. v21.1 versus v21.2 beyond the 43-row set

The independent sealed-archive comparison remains the governing custom-model decision:

v21.2 is the candidate of record because it is a practical Pareto improvement across the domains that were replayed, not because its elder score is adequate. It also creates more long synthetic segment loops: 37 rows versus 25 on the tagged synthetic test. Every future model must therefore pass both domain retention and hard degeneration gates.

7. What previous experiments already ruled in or out

The local program has already tested more than a single NLLB continuation. The useful result is the accumulated failure map.

The core lesson is that routing and evidence use are more promising than repeatedly merging every corpus into one checkpoint. Retrieval works reliably as an exact-answer layer and appears more useful as evidence for a capable post-editor than as unstructured text prepended to NLLB.

8. Recommended MobTranslate architecture

8.1 Route 1: exact translation memory

Normalize only for lookup, preserve the original form for display, and require an exact rights-cleared resource identity. Return the stored translation, source, dialect/register metadata, and recording or dictionary links. If multiple records conflict, show the alternatives rather than selecting one by a hardcoded priority list.

8.2 Route 2: lexical dictionary answer

Use this only for short lexical queries with exactly one distinct matching headword after normalization. Display all senses and provenance. Ambiguous matches remain a user-visible choice. Do not insert a dictionary lemma into a sentence and call the result a translation; inflection, valency, case, agreement, discourse, and idiomatic choice still require analysis.

8.3 Route 3: custom-model research draft

For a novel English sentence within the supported length, generate with v21.2's sealed greedy configuration. Before returning it, apply language-neutral structural gates:

• empty or whitespace-only output;
• source copy or implausibly high source overlap;
• repeated word or orthographic-segment loops;
• decoder length-limit termination;
• severe output-length tails calibrated on speaker-reviewed data;
• unavailable or hash-mismatched model artifact.

A failed gate should produce an honest abstention or side-by-side evidence view, never a silent fallback presented as equivalent quality.

8.4 Route 4: optional evidence-grounded post-edit

This remains a dark experiment. When eventually enabled, the post-editor should receive:

• the source sentence;
• at least two independently useful custom drafts;
• retrieved, rights-cleared parallel examples with provenance;
• relevant dictionary senses, not just headwords;
• relevant grammar passages;
• a structured output schema with exact IDs, decision, translation, uncertainty reason, and cited evidence IDs.

It should not receive private held-out references. Provider calls must use subscription lanes under the standing budget policy or a separately approved serving arrangement. The system should store the model/version, prompt hash, evidence IDs, draft hashes, output, and user/speaker corrections for every generation.

8.5 Route 5: correction capture

Corrections are the highest-value future data only after provenance and authority are explicit. Store the original draft, correction, reviewer identity or pseudonymous role, dialect, date, task context, rights grant, confidence, adjudication state, and supersession chain. Never overwrite an earlier form. Only speaker-approved/adjudicated rows may enter the high-trust training tier.

9. Linguistically rigorous evaluation program

The next decisive experiment is not another automatic score. It is a blind, dialect-aware speaker evaluation.

9.1 Freeze before collecting more training data

Before any new elder-shared or natural narrative material enters training, freeze a new held-out set with checksums. Stratify it by source, speaker, genre, sentence length, lexical novelty, clause complexity, and discourse dependency. Group related sentences and all variants from one source entirely within one split. Surface-only random splitting is not sufficient.

9.2 Human rating instrument

Randomize and anonymize system outputs. At minimum, speakers should mark:

Use an ordinal severity scale with explicit examples and allow valid alternative, cannot judge, and free-form correction. Collect at least two independent ratings where feasible, report agreement, and preserve disagreement. Adjudication should not erase the original judgments.

9.3 Evaluation outputs

Report system-level and slice-level:

• exact match only where one canonical form is genuinely expected;
• chrF++, BLEU, and at least one independently validated semantic metric as secondary diagnostics;
• speaker adequacy and acceptability distributions with intervals;
• critical-error and omission rates;
• case/TAM/pronominal/derivational error rates;
• dialect and register mismatch rates;
• abstention coverage and selective risk;
• repetition, length-limit, empty, and source-copy rates;
• performance by source, genre, sentence length, lexical novelty, and construction;
• inter-rater agreement and adjudication rate.

Do not select a system with one metric and then use only that metric to claim success. Metric-based minimum Bayes risk or quality estimation can be explored only after the metric is calibrated against this speaker-reviewed set, and the selection metric must be separated from the final evaluation evidence.

10. Model-improvement experiment queue

Experiment A: isolate the v21 replay effect

Run v21.1 and v21.2 as a step-matched factorial design:

• synthetic-only at 3,120 and 4,155 optimizer steps;
• balanced replay at 3,120 and 4,155 steps;
• at least three seeds per cell if compute permits;
• identical hardware, software, decoder, and frozen evaluations.

This separates corpus composition from extra optimization and estimates run-to-run variance.

Experiment B: improve data diversity, not synthetic count

The 20,047 pairs are valuable for controlled coverage, but more template permutations are not the current bottleneck. Prioritize speaker-approved natural material across conversation, instruction, narrative, questions, commands, negation, repair, kinship, place, motion, reported speech, and multi-clause discourse. Track lexical types, morphological feature bundles, construction coverage, source diversity, and near-duplicate clusters before training.

Experiment C: speaker-approved post-edit distillation

Use the custom model plus retrieval to generate candidates. Let one or more LLMs propose analyses and revisions, but train only on corrections accepted or authored by speakers. Retain rejected suggestions as negative evidence. This turns the LLM into an annotation accelerator rather than an unverified teacher.

Experiment D: contrastive grammar and loanword controls

Create speaker-reviewed minimal contrasts for case, pronouns, tense/aspect/mood, valency, derivation, negation, question formation, constituent order, and loanword adaptation. Train or rerank against grammatical and minimally corrupted alternatives. Avoid hand-maintained lists as a general classifier; the contrasts belong in a versioned, evidence-bearing dataset.

Experiment E: stronger multilingual checkpoints

Benchmark, do not assume, larger or translation-specialized multilingual bases such as MADLAD-400 and Tower-family checkpoints, alongside NLLB variants. Verify license, tokenizer behavior, language-tag strategy, memory requirement, and contamination risk before RunPod use. Every candidate must pass a tiny memorization gate, the frozen held-outs, and degeneration tests before full training. The prior ByT5-small failure is evidence against scaling that exact configuration, not against every character/byte-level model.

Experiment F: learned quality estimation and MBR

Once speaker ratings exist, train or calibrate quality estimation on project errors and investigate multi-candidate selection or minimum Bayes risk with metric ensembles. Include the raw custom draft as a candidate so the reranker can decline an LLM edit. Hold out a final speaker-rated set that neither trains the quality estimator nor chooses its thresholds.

Experiment G: reverse direction

Kuku Yalanji-to-English is a separate task. Build and evaluate it separately rather than assuming bidirectional symmetry. English adequacy may be easier to judge automatically, but source-language analysis, ambiguity, and cultural authority remain speaker-dependent.

11. RunPod execution plan

RunPod should be treated as disposable compute; the mounted drive remains the canonical archive.

Stage 0: local freeze

1. Build immutable train/validation/test manifests and source-group leakage audits.
2. Hash every input, base checkpoint, tokenizer, script, lockfile, and configuration.
3. Write the promotion gates before observing the new model's test output.
4. Confirm no test or speaker-review row occurs in training by ID, source group, normalized pair, or near-duplicate cluster.

Stage 1: pod preflight

1. Pin the image, CUDA/PyTorch/Transformers/PEFT versions, GPU type, and seed.
2. Verify model and tokenizer hashes after upload.
3. Run one training batch, one validation pass, one save/load cycle, and one inference batch.
4. Monitor actual step/output progress, GPU memory, host memory, disk, and logs; a started process is not proof of a working run.

Stage 2: memorization and pipeline gate

Use a small, checksum-frozen sample to prove that the model can overfit the task and that the target language token, decoder, labels, masking, save/merge, and evaluation path are correct. A failed gate stops the run. It must not be reinterpreted as a linguistic result.

Stage 3: controlled training

Run one planned cell at a time, with periodic validation and resumable checkpoints. Never tune on the frozen final test or speaker-review set. Record optimizer steps rather than only epochs because corpus mixtures have different sizes.

Stage 4: evaluation

Merge or load adapters reproducibly and decode every system with the same frozen settings. Apply automatic gates, paired intervals, linguistic slices, and blind speaker review. Keep raw predictions, including failures.

Stage 5: archival closure

1. Download every checkpoint, adapter, merged model, log, prediction, environment record, and report.
2. Create and verify SHA-256 manifests locally against remote artifacts.
3. Perform a local load and smoke inference from the archived copy.
4. Delete the pod only after verification.
5. Query the account and record that no active pods remain.

No model is promoted merely because a RunPod job completed or loss decreased.

12. Mi'gmaq program: separate evidence, separate gates

The Mi'gmaq v1 NLLB-LoRA candidate passed machinery and deployment checks but failed linguistic promotion on its frozen 742-row test: corpus chrF++ 21.4298, BLEU 1.7795, exact 0/742, with material lexical/content errors. It remains a guarded noncommercial research preview.

Do not transfer a Kuku Yalanji threshold, grammar prompt, or synthetic recipe into Mi'gmaq unchanged. The next Mi'gmaq cycle should:

1. inventory rights-cleared parallel material by source, dialect, orthography, genre, speaker, and translation direction;
2. keep dictionary definitions, usage examples, recordings, Bible material, and natural discourse as separately tagged evidence families;
3. freeze source-group-disjoint validation and test sets before adding more data;
4. link recordings to canonical dictionary entries with provenance, but do not treat audio existence as proof of a sentence translation;
5. establish Mi'gmaq-speaker review and orthography/dialect policies;
6. compare exact retrieval, custom NLLB, stronger multilingual bases, and evidence-grounded post-editing under the same blind protocol;
7. distill only adjudicated corrections.

The shared software can implement manifests, leakage checks, schemas, metrics, deployment guards, and archives. The linguistic evidence and promotion decision remain language-specific.

13. Failure and deviation ledger

The benchmark was rebuilt rather than concealing failed attempts:

1. The first v1 Codex batch duplicated one output ID and omitted another. That run was rejected.
2. The first strict array schema used prefixItems; Codex rejected it. The contract was changed to an exact object keyed by required IDs, which is both provider-compatible and stricter about identity.
3. A Codex attempt searched sibling blind-batch files. The entire run was discarded, and all accepted runs moved to a fresh empty working directory.
4. The prompt and runner then explicitly prohibited tools and filesystem search. Three empty-directory Codex commands still occurred in two accepted batches; they found no benchmark evidence and are recorded above.
5. Claude rejected the draft-2020 $schema URI. Provider schemas were generated by removing only that metadata key; both versions are retained and hashed.
6. Claude used no web search or web fetch. Codex had no useful external or neighboring-file access in accepted runs.
7. The provider CLIs expose different usage accounting. The raw logs and normalized usage record are both retained.

Failed and superseded benchmark directories are preserved rather than overwritten:

kuku-elder-postedit-v1/
kuku-elder-postedit-v2/

14. Provider and spend record

No per-token API key was used.

Claude's CLI reported an API-equivalent estimate of USD 7.44593. This is metadata only; the run authenticated through the subscription and did not incur a per-token API charge. Provider accounting fields are not directly comparable.

15. Reproduction

Repository implementation:

/mnt/donto-data/workspace/mobtranslate.com/training/translation/benchmark_hybrid_postedit.py
/mnt/donto-data/workspace/mobtranslate.com/training/translation/test_benchmark_hybrid_postedit.py
/mnt/donto-data/workspace/mobtranslate.com/training/translation/run_hybrid_postedit_benchmark.sh

Canonical benchmark root:

/mnt/donto-data/donto-resources/research/translation-training/
  translation-systems-benchmark-2026-07-13/kuku-elder-postedit-v2/

Reproduction commands:

cd /mnt/donto-data/workspace/mobtranslate.com

# Rebuild blind/private inputs from the sealed sources.
CREATED_AT=2026-07-13T15:28:48Z \
  training/translation/run_hybrid_postedit_benchmark.sh prepare

# Subscription-backed provider lanes. Existing outputs are skipped unless FORCE=1.
training/translation/run_hybrid_postedit_benchmark.sh codex
training/translation/run_hybrid_postedit_benchmark.sh claude

# Validate exact row IDs and compute the frozen metric battery.
training/translation/run_hybrid_postedit_benchmark.sh score

Unit tests:

cd /mnt/donto-data/workspace/mobtranslate.com
uv run --with 'numpy>=1.26,<3' --with 'sacrebleu>=2.4,<3' \
  python -m unittest training.translation.test_benchmark_hybrid_postedit

The provider logs are under:

/opencode/logs/mobtranslate-translation-benchmark-20260713/

16. Sealed artifact hashes

e2c41cbd7cd665fbe6950662e25a52b96352411cfa0ba65283008b4244970f7d
  kuku-elder-postedit-v2/results/full-comparison.json

b8079511bdbceeacf739023e9077e1790d3db1f7a4bc3ae1fe00ba02f6d4bfa1
  kuku-elder-postedit-v2/results/provider-usage.json

2880493fc7a206b6d0db663459130402877cb854ddce3a82e436d6907409fc1a
  product-coverage.sql

f56625406ad7ff40090ef1ca97a4cdfc113882e40484a6aac15ab6b98a6b7a34
  product-coverage.json

The benchmark manifest contains hashes for every blind batch, both provider schemas, private reference and candidate maps, source prediction archives, training collection, dictionary, grammar book, and each provider output.

17. Research synthesis

The external literature supports a cautious hybrid program rather than blind scaling:

• Retrieval and dictionary examples can help endangered/low-resource MT, but domain mismatch and retrieval method matter: Mambai retrieval-augmented MT and retrieval-method comparison.
• Grammar materials can support LLM post-correction, which directly motivates this benchmark's evidence bundle: grammar-informed post-correction.
• Diverse, language-specific corpora and direction-specific synthetic effects matter more than one undifferentiated data count: Amis corpus and MT study and Indigenous-language preprocessing and synthetic data.
• Loanwords and grammatical negative examples need explicit treatment: loanword constraints and grammar-negative samples.
• Quality estimation and MBR can combine candidates, but metric bias must be measured rather than assumed: QE fusion, COMET MBR, and metric bias in MBR.
• Automatic metrics correlate weakly in some low-resource conditions; Indigenous-language evaluation needs language-aware validation: low-resource metric study and FUSE.
• Benchmark difficulty can be distorted by overlap and pretraining exposure: FRED and multiple references and leakage.
• Candidate multilingual bases worth controlled evaluation include NLLB, MADLAD-400, Tower, and ByT5. Their published breadth does not establish support for either project language; each needs a local tokenizer, memorization, leakage, and speaker-evaluation gate.

These papers motivate experiments; they do not override the local evidence or grant community authority.

18. Limitations

1. The elder-shared set has 43 rows and one duplicated English surface.
2. There is no independent blind Kuku Yalanji speaker rating in this comparison.
3. The two custom models are single-seed systems and are not step-matched.
4. Automatic overlap can reward an invalid form and penalize a valid alternative.
5. The natural-register references represent a narrow story, source, orthographic practice, and discourse domain.
6. LLM providers may have undisclosed pretraining exposure and may vary across reruns.
7. Retrieved evidence excludes exact held-out surfaces but can contain partial lexical or construction overlap.
8. Codex executed three prohibited empty-directory commands; no evidence was exposed, but this remains a deviation.
9. The current GPT product path has not been subjected to this full frozen benchmark; one qualitative probe is not a system comparison.
10. No reverse-direction, speech, calibrated-abstention, latency, throughput, or production-cost benchmark is included here.
11. Mi'gmaq results are not comparable to Kuku Yalanji scores because their corpora, splits, domains, and review states differ.

19. Promotion checklist

A future system may replace the guarded research label only when all applicable items pass:

• immutable artifact and dataset hashes verified;
• source-group and near-duplicate leakage audit passed;
• exact-resource retrieval non-regression passed;
• automatic metric battery and paired intervals reported without cherry-picking;
• zero empty, source-copy, repeated-loop, and length-limit safety failures, or an explicit tested abstention path;
• no material regression on dictionary usage, Bible, synthetic grammar slices, and new natural-register controls;
• blind speaker adequacy, morphosyntax, dialect/register, and acceptability review passed under a predeclared rule;
• disagreements and valid alternatives preserved;
• rights, consent, provenance, and correction authority documented;
• live endpoint, UI labeling, rate limit, length limit, model hash, fallback behavior, and rollback verified;
• local archive restored and loaded successfully;
• RunPod account confirmed to have no unintended active resources.

Until then, the correct label is research draft, not translation authority.

20. Immediate next action

Freeze a new, source-group-disjoint natural-register Kuku Yalanji set before adding any further natural data to training. Blind-review v21.2, Codex post-edit, Claude post-edit, and the unchanged draft with Kuku Yalanji speakers. Use their adjudicated corrections to define the next training corpus and to calibrate an abstention or quality model. Only after that evidence exists should another RunPod training matrix or public LLM-postedit route be promoted.

21. Live deployment and browser verification

The guarded v21.2 custom-model candidate is now the live Kuku Yalanji research translator at https://mobtranslate.com/translate/v2. The public route invokes the local custom model directly; Codex and Claude postediting remain offline research candidates pending speaker evidence.

The final production build completed all 359 static pages. Focused tests passed for the benchmark harness (5 tests) and mixed JSON/HTML model-result loading (2 tests), and the focused web lint check passed. During deployment, four defects were found and corrected:

1. an HTML evaluation report was being parsed as JSON during static generation;
2. the versioned API route did not recognize the inference service's legacy kuku field;
3. mobile grid children could exceed their parent because their implicit minimum width was not reset;
4. the initial build lacked the production environment loaded by the service.

Fresh public probes after the final restart returned HTTP 200 from both local custom systems:

These rows prove execution, not linguistic correctness.

Desktop browser verification used a 1440 by 1000 viewport. Mobile verification used 390 by 844: the form, input, counter, and output stayed within the 312-pixel form; document scroll width equalled client width; no overlay was present; and an Enter-key submission rendered Jalbu-ngku bana nyajin. with the v21.2 model label. Clean initial, desktop-result, and mobile-result viewport captures are retained under browser-verification/.

CDP stack capture attributed the browser CLI's sole uncaught exception to third-party analytics at https://openpanel.dev/op1.js:0:8617; no Mob Translate bundle appeared in the stack. The optional unprovisioned Vercel Analytics loader also logs a non-blocking warning. These telemetry defects should be handled separately from the translation system.

The complete deployment ledger, exact responsive geometry, service state, request payloads/responses, screenshot inventory, build corrections, and remaining gates are preserved at:

/mnt/donto-data/donto-resources/research/translation-training/translation-systems-benchmark-2026-07-13/
  BROWSER-AND-LIVE-DEPLOYMENT-VERIFICATION-2026-07-13.md