Download MobTranslate language models

CC-BY-NC-4.0 upstream NLLB base plus source-specific project data terms. Noncommercial research and explicitly labelled drafts only; not speaker-certified, community-approved, or authoritative.

optimizer steps

3598.0

train loss

0.9482

closed set lexical rows

2724.0

closed set lexical accepted exact

2696.0

closed set lexical accepted exact percent

99.0

closed set lexical wilson 95 low percent

98.5

closed set lexical failures

28.0

historical lexicon rows

297.0

• The headline 98.9721% score is accepted-reference reconstruction on 2,724 governed dictionary records shown 29 times each during training. It is not unseen-word generalisation or sentence accuracy.
• The selected step improved every reported development sentence corpus-chrF endpoint over untouched v21.2, but the run used one seed and development data; elder exact remained 0/43.
• The older 297-prompt benchmark scored 91 exact because only 208 prompts overlap and the two resources have different accepted surfaces; 206/208 overlapping predictions were supported by the governed census.
• All 28 remaining governed-census failures are published with token, part-of-speech, semantic-domain, closest-reference, and checkpoint-trajectory evidence.
• Dynamic CPU loading was verified with both task prefixes. It peaked at 8.00 GiB cgroup memory, settled around 7.0-7.2 GiB RSS, and produced 4.9-23.2 second requests under variable shared-disk pressure.
• The adapter is large because PEFT persisted full embedding and output matrices for the trainable <lexeme> row. Repacking or merging requires full adapter-parity evaluation before release.
• Known dictionary queries should continue to use deterministic lookup. Every generated sentence must be visibly labelled as an unverified research draft.

CC-BY-NC-4.0 base; project-approved synthetic and governed replay data for noncommercial research. The model is not speaker-certified and is not approved for authoritative or unrestricted translation.

weights train loss

1.199

synthetic dev chrf

55.2

synthetic dev exact

394.0

synthetic dev segment loop rows

1.000

synthetic test tagged chrf

55.7

synthetic test tagged exact

426.0

synthetic test tagged segment loop rows

2.000

synthetic test untagged chrf

55.7

• The merged model SHA-256 is exactly the published v21.2 identity; this is a decoding-policy release, not a renamed checkpoint.
• The decoder was selected on the v22 development set and locked before v22 test/control inference and before any v21.2 transfer inference.
• A first transfer attempt reconstructed non-identical float32 bytes and was rejected before evaluation; only the exact published v21.2 merged model was scored.
• Guarded versus greedy tagged synthetic corpus chrF changed by +1.8723 while repeated-segment rows fell from 37 to 2; all seven frozen sets had zero empty outputs.
• Exact match fell on some sets and Bible chrF changed slightly downward; these outcomes are reported in the experiment book rather than hidden.
• A post-training 297-prompt canonical-headword probe scored this release at 48/297 normalized accepted-reference exact matches (16.16%); the result was 46/192 for target forms seen as training tokens and 2/105 for unseen targets.
• Under the same fixed decoder, step 4,155 exceeded step 3,120 by 1.68 exact-match percentage points on the lexicon probe (paired 95% interval +0.34 to +3.37), while the 43-row elder difference remained inconclusive and every checkpoint was 0/43 exact.
• An exact NVIDIA A40 reproduction evaluated all 340 frozen rows for steps 2,770, 3,120, and 4,155. All 1,020 GPU predictions matched the frozen local baselines, and a second A40 run reproduced all 1,020 predictions exactly.
• The portable evaluator fail-closes on input, model-file, runtime, decoder, determinism, GPU-use, empty-output, baseline-parity, and output-checksum violations. Its compressed kit is 100,526 bytes and excludes model weights.
• A first complete A40 attempt was rejected when a post-seal log append invalidated its checksum inventory. The failed attempt and repair evidence remain in the private research archive; only the clean from-scratch rerun is published.
• On 2026-07-14 the Kuku Yalanji and Mi'gmaq CPU inference workers were stopped and disabled, their web-service dependencies were removed, and custom-model endpoints were cleared. This release remains downloadable and reproducible but is not currently loaded for live translation.
• Automatic metrics are regression evidence, not community validation. Lookup-first routing and explicit research-draft labelling remain required.

Noncommercial research result. Source materials retain their own terms. The candidate was not speaker-reviewed, community-approved, published as weights, or authorized for sentence generation.

selected seed

73.0

optimizer steps per seed

882.0

text3 natural test rows

56.0

text3 baseline corpus chrf

31.7

text3 candidate corpus chrf

32.3

text3 corpus chrf delta

0.6414

text3 mean sentence chrf delta

0.4494

text3 paired ci95 low

-0.2448

• The speaker-disjoint held-out natural test was opened only after development selection, but all texts share one published grammar and editorial tradition.
• No fluent speaker reviewed model outputs; automatic character overlap cannot establish semantic, grammatical, dialectal, or cultural validity.
• All three seeds, checkpoints, adapters, merged weights, optimizer states, and trainer caches were deleted after compact evidence and checksums were verified.
• This registry row records a completed negative result. It is not a downloadable model edition and cannot become the latest serving alias.

Noncommercial research artifact only. Rejected by the preregistered promotion gate; not speaker-certified, production-approved, or offered as a model download.

global step

3120.0

additional optimizer steps

350.0

greedy synthetic dev chrf

52.9

greedy synthetic test tagged chrf

53.2

greedy synthetic test tagged exact

417.0

greedy synthetic test tagged segment loop rows

37.0

greedy synthetic test untagged chrf

53.3

greedy synthetic test untagged exact

419.0

• The hypothesis, exact stop, hashes, promotion floors, and analysis rules were frozen before the locked test and control sets were opened.
• The development-only guarded decoder controlled degeneration but could not rescue a checkpoint that failed the primary training-exposure gate.
• The final v21.2 step 4,155 checkpoint remained better on tagged synthetic, untagged synthetic, and dictionary-usage paired comparisons.
• All 182 remote-manifest files were verified after pullback with zero failures; the RunPod pod was then deleted and an empty post-deletion pod list was saved.
• The rejected weights remain in the internal research archive but are intentionally not presented as a public model download.

CC-BY-NC-4.0 base; project-approved synthetic and governed replay data for noncommercial research. The model is not speaker-certified and is not approved for authoritative or unrestricted translation.

train loss

1.199

synthetic dev bleu

26.2

synthetic dev chrf

52.8

synthetic test tagged bleu

26.8

synthetic test tagged chrf

53.8

synthetic test tagged exact

426.0

synthetic test untagged bleu

26.8

synthetic test untagged chrf

53.8

• The independent comparison revalidated ordered IDs, inputs, references, decoder settings, and archive checksums before scoring.
• Compared with v21.1, corpus chrF changed by -0.87 on tagged synthetic test, +12.74 on dictionary usage, +15.23 on Bible direct, and +15.15 on Bible reference.
• The 43-row elder control remains 0 exact with severe under-translation; its paired confidence interval versus v21.1 includes zero.
• Thirty-seven tagged synthetic rows contain a segment repeated at least ten times, so generation guards and a frozen-battery rerun are mandatory before broader routing.
• The original release record preserves its greedy evaluation; current serving uses the separately validated guarded-policy release and remains visibly labelled as a research draft.

Project-approved synthetic research data pending elder verification. The model is not speaker-certified and is not approved for public translation routing.

train loss

1.691

validation bleu

26.2

validation chrf

53.9

synthetic test tagged bleu

26.1

synthetic test tagged chrf

54.7

synthetic test untagged bleu

26.3

synthetic test untagged chrf

54.4

elder shared chrf

28.8

• The mandatory 30-epoch gate failed and correctly blocked full training; its complete model is retained as negative evidence.
• A documented 100-epoch gate amendment passed 128/128 exact before the full treatment unlocked.
• All eight frozen evaluation sets completed at exact row counts with zero empty outputs.
• All 192 model files matched the RunPod SHA-256 inventory; local CPU and remote CUDA smoke outputs were identical; the pod was deleted.
• Compare with Claude's independent v21.2 balanced-replay lane before any release decision.

Rights granted for elder-shared/DB examples; Bible-derived rows used as research diagnostics. Known approved resources remain lookup-first.

train loss

1.485

trainer validation bleu

13.4

trainer validation chrf

39.9

trainer test bleu

13.2

trainer test chrf

40.3

validation bible direct chrf

39.3

validation bible direct exact

52.0

validation bible ref chrf

39.1

• This was the first full-candidate-corpus run after the smaller v18/v19 replay runs.
• The corpus was expanded into tagged direct/reference Bible tasks plus DB usage and elder-shared sentence-pair replay.
• The result shows full-corpus scale alone is not enough; route-specific lookup and mixture control remain necessary before another expensive run.
• Use v20 as a diagnostic artifact, not as the product default for faithful output.

Rights granted for elder-shared/DB examples; Bible-derived rows used as research diagnostics. Known approved resources remain lookup-first.

train loss

0.2129

validation bleu

95.8

validation chrf

97.2

heldout usage chrf

53.0

heldout usage exact

6.000

heldout bible direct chrf

44.2

heldout bible ref chrf

44.3

heldout all chrf

44.5

• Next move after v18: balanced replay continuation from v12, not another single-domain continuation.
• Best broad heldout score after elder sentence-pair ingestion, but not a universal replacement.
• Use for current research state; keep exact Bible/DB/elder sentence-pair resources lookup-first.

Elder-shared sentence pairs and DB examples are approved for training; exact known rows remain lookup-first.

train loss

0.2773

train usage elder sentence pair chrf

100.0

train usage elder sentence pair exact

408.0

elder sentence pair chrf

100.0

elder sentence pair exact

43.0

heldout usage chrf

54.5

heldout usage exact

3.000

heldout bible direct chrf

41.4

• Proved the 16 uploaded source pages could be transcribed into 43 useful supervised sentence-pair rows.
• Route-specific artifact: good proof for elder sentence-pair data ingestion, not a general model winner.

Research diagnostic; not an approved translation source.

heldout bible direct chrf

44.1

heldout bible ref chrf

43.7

• Soft lexical hints were not the missing ingredient; do not scale this branch.

Research diagnostic; not an approved translation source.

heldout bible direct chrf

29.5

heldout bible ref chrf

29.7

• Retrieval remains useful for lookup/evidence display, not as this NLLB prefix format.

Research diagnostic; exact Bible references should be served from approved lookup.

heldout bible direct chrf

44.4

heldout bible ref chrf

44.4

heldout exact

0.0000

• Useful as Bible draft fallback only; not faithful canonical reproduction.

Research diagnostic only.

heldout bible direct chrf

6.090

heldout bible ref chrf

5.710

heldout all chrf

5.900

• Kept as a control result, not a candidate model.

DB usage examples are approved; exact known DB rows remain lookup-first.

heldout usage chrf

56.2

heldout usage exact

3.000

heldout bible direct chrf

43.2

heldout bible ref chrf

43.3

heldout all chrf

43.6

• Keep as usage/general draft fallback until a later run clearly beats it on word-id heldout usage.

Research diagnostic with approved DB usage examples.

heldout usage chrf

39.5

heldout bible ref chrf

44.0

heldout all chrf

43.9

• Important because it proved DB examples can be learned, but not enough to route as usage fallback.

Research diagnostic; exact Bible references remain lookup-first.

heldout bible direct chrf

44.1

heldout bible ref chrf

44.3

heldout exact

0.0000

• Reference-conditioned input helped; exact canonical Bible output still belongs to retrieval/lookup.

Research diagnostic only.

overfit gate

1.000

pipeline alive

1.000

• Not a product model; it is the reason later runs are meaningful.

Kuku Yalanji eBible snapshot rights granted for MobTranslate model training by project owner attestation on 2026-06-30. This is an internal experimental artifact for evaluation, not a community-approved production translator.

validation loss

1.689

validation bleu

8.745

validation chrf

37.5

test loss

1.678

test bleu

8.401

test chrf

37.1

standalone bleu

6.377

standalone chrf

34.5

• Best current experimental English to Kuku Yalanji model and the first run to clearly beat v0.4 across automatic and row-level comparison metrics.
• Served through the existing NLLB tpi_Latn token as a proxy target, which outperformed the failed 1.3B custom-token probe.
• Still not faithful enough for production or community-approved use: long passages compress and some clauses drift or disappear.
• Added non-Bible MobTranslate DB usage-example eval preview so translate/v2 shows everyday dictionary-context failures, not only Bible verse rows.

Kuku Yalanji eBible snapshot rights granted for MobTranslate model training by project owner attestation on 2026-06-30. This is an internal experimental artifact for evaluation, not a community-approved production translator.

train loss

2.561

validation loss

1.966

validation bleu

5.539

validation chrf

33.1

test loss

1.989

test bleu

5.274

test chrf

32.7

standalone bleu

3.645

• Best current experimental English to Kuku Yalanji model and the first full 8-epoch high-confidence run.
• Better than v0.3 on held-out metrics and samples, but still not faithful enough for production or community-approved release.
• Primary remaining failure mode is semantic looseness and dropped content, especially on longer verses.

Kuku Yalanji eBible snapshot rights granted for MobTranslate model training by project owner attestation on 2026-06-30. Proof artifacts are local/internal and are not a public release.

train loss

6.295

validation loss

5.773

validation bleu

0.0313

validation chrf

4.744

test loss

5.817

test bleu

0.0275

test chrf

4.272

standalone bleu

0.0269

• Budget-safe proof run on the high-confidence corpus, not a production translator.
• A40 utilization was healthy: 100% max GPU, 26.5 GiB max VRAM, and 295 W max power draw.
• Use this release to test the translate/v2 model-lab path and inference server wiring.

Kuku Yalanji eBible snapshot rights granted for MobTranslate model training by project owner attestation on 2026-06-30. Smoke artifacts are only a pipeline proof.

train loss

6.537

validation loss

6.013

validation bleu

0.1085

validation chrf

4.913

test loss

6.416

test bleu

0.0540

test chrf

5.485

standalone bleu

0.0552

• First end-to-end RunPod proof: dataset upload, CUDA validation, LoRA training, merge, eval, artifact pullback.
• Not useful as a translator; kept for regression testing the model project.

Kuku Yalanji eBible snapshot rights granted for MobTranslate model training by project owner attestation on 2026-06-30. Base model license and final release terms still apply to trained artifacts.

• Full baseline is intentionally blocked until the model project, download registry, and translate/v2 test bench are working.
• Planned first publishable candidate: high-confidence rows only, English to Kuku Yalanji, 8 epochs, A40/RTX A6000 first.

Noncommercial research only. Mi'gmaq/Mi'kmaq Online Talking Dictionary data and NLLB base are CC BY-NC 4.0. Output is not speaker-reviewed or community-endorsed.

validation bleu

1.387

validation chrf

21.0

validation chrf ci95 low

20.4

validation chrf ci95 high

21.7

test bleu

1.780

test chrf

21.4

test chrf ci95 low

20.8

test chrf ci95 high

22.0

• The first plumbing gate exposed a PEFT/NLLB embedding-alias serialization defect and correctly blocked full training.
• The corrected gate reproduced merged-model predictions and reached 128/128 exact on a training-only memorization control.
• The full trainer received no test file. Decoding was selected on validation before the frozen test was accessed once under an exclusive hash ledger.
• All validation and test examples belong to source-headword groups absent from training.
• The written-data scope is the source collection's contemporary Listuguj spelling; audio was not used for this text model.