Kuku Yalanji v21.2 Model, Training, Data, and Hosting Guide
Published: 2026-07-13 UTC
Updated: 2026-07-14 UTC
Model ID: kuku-yalanji-nllb-lora
Current downloadable release:
v21.2-claude-balanced-replay-guarded-20260714
Immutable weights release:
v21.2-claude-balanced-replay-v2-candidate
Task: English (eng_Latn) -> Kuku
Yalanji (gvn_Latn) translation
Status: noncommercial research only; not
speaker-certified; not approved for authoritative translation
1. Read this first
This is the public, end-to-end handoff for downloading, loading, hosting, auditing, and understanding the current Kuku Yalanji model candidate. The guarded release preserves the exact v21.2 weights and changes only the validated decoding policy. It remains downloadable, but local custom-model CPU inference is stopped and disabled because v21.2 fails the separate 80% dictionary deployment gate. It accompanies the versioned model API and immutable model and dataset bundles.
The release is a sequence-to-sequence translation model, not a chat
LLM. It is a PEFT LoRA continuation of a project-specific NLLB
checkpoint derived from facebook/nllb-200-distilled-1.3B.
The exact frozen base contains the project's gvn_Latn token
at vocabulary ID 256204. The v21.2 adapter is therefore not
compatible with an untouched stock NLLB checkpoint, even if the
upstream model name is the same.
The release decision is deliberately narrow:
| Question | Evidence-based answer |
|---|---|
| Best current Kuku Yalanji custom research candidate | v21.2 balanced replay |
| Current inference policy | Beam 1, no-repeat 4, repetition penalty 1.10, length penalty 1.0 |
| New model weights in the guarded release | No; weights are byte-identical to v21.2 |
| Safe unrestricted translator | No |
| Speaker or elder certified | No |
| Appropriate role | Labelled research draft after exact-resource lookup and generation guards |
| Exact known sentence and dictionary resources | Continue to serve by retrieval, not generation |
| Commercial use | Not granted; the NLLB base is CC-BY-NC-4.0 and project data has additional restrictions |
The model learned the project-authored synthetic treatment and retained Bible and dictionary-usage domains much better than the v21.1 synthetic-only ablation. It did not establish general natural Kuku Yalanji competence. Both v21 models scored 0/43 exact on the frozen elder-shared control. The archived v21.2 greedy decoder exhibited severe under-translation and morpheme-loop failures; the guarded policy reduced tagged-test repeated-segment rows from 37 to 2 without materially degrading the Bible controls. Natural-register transfer remains unresolved. A successful checksum or smoke test establishes artifact compatibility only.
2. Distribution API
The public API is read-only, requires no API key, permits
cross-origin GET requests, and returns absolute URLs. No
server filesystem paths are exposed.
| Purpose | URL |
|---|---|
| List every model and version | https://mobtranslate.com/api/v1/models |
| Kuku Yalanji model record | https://mobtranslate.com/api/v1/models/kuku-yalanji-nllb-lora |
| Current downloadable Kuku Yalanji release | https://mobtranslate.com/api/v1/models/kuku-yalanji-nllb-lora/versions/latest |
| Current guarded downloadable release | https://mobtranslate.com/api/v1/models/kuku-yalanji-nllb-lora/versions/v21.2-claude-balanced-replay-guarded-20260714 |
| Historical weights/greedy release | https://mobtranslate.com/api/v1/models/kuku-yalanji-nllb-lora/versions/v21.2-claude-balanced-replay-v2-candidate |
| Mi'gmaq model record | https://mobtranslate.com/api/v1/models/migmaq-nllb-lora |
| OpenAPI 3.1 document | https://mobtranslate.com/api/v1/models/openapi.json |
| Human model browser | https://mobtranslate.com/models |
Discover model downloads programmatically:
curl -fsS \
https://mobtranslate.com/api/v1/models/kuku-yalanji-nllb-lora/versions/latest \
| jq '.release | {version, rights, downloads: .downloads.models}'Resolve a URL and continue an interrupted large download:
API=https://mobtranslate.com/api/v1/models/kuku-yalanji-nllb-lora/versions/latest
URL="$(curl -fsS "$API" | jq -r '.release.downloads.models[] | select(.kind == "bundle") | .url')"
curl --fail --location --continue-at - --output kuku-yalanji-v21.2-guarded-atlas-lora-bundle.tar.gz "$URL"latest is a convenience alias and may resolve to a
different version after a future release. Hosts must record and pin the
immutable version and SHA-256 value returned by the API. The literal
version endpoint is the reproducible contract.
3. Release downloads
3.1 Models and hosting metadata
| Artifact | Bytes | SHA-256 | Direct URL |
|---|---|---|---|
| Atlas dynamic LoRA bundle: exact base + adapter + guarded policy | 2,530,623,924 | 012335cfb393ca386c7817bd3850dd52f1a751e0e1a58e80e9e5bc9bae9c89d5 |
tar.gz |
| Standalone merged model (weights archive unchanged) | 2,200,740,249 | 920e179d8b4fe45ce75bdd1e9cc2784be51ff8d55d1a190cd45e19cf01a5f490 |
tar.gz |
| Machine hosting manifest | 3,487 | e40fd2586a3d4eabfdf6052bd65e4f3a92532b1141ea446bc81feb03a7a01c95 |
JSON |
| Release metadata | 1,915 | a5ded1fba823e2596bfaa966298961d81c91087c6a424fd1abd8744a371b33df |
JSON |
| Release checksum file | 1,302 | 81e992b0e07ca37d99422ddcf65449fca88bf831a18d4ab1cfec08b1d37c15a1 |
SHA256SUMS |
Use the Atlas bundle when the host supports dynamic
encoder-decoder PEFT adapters. It contains base/,
adapter/, README.md, and
HOSTING-MANIFEST.json. The adapter configuration is
portable and points to ../base.
Use the merged bundle when the serving stack cannot dynamically attach a LoRA adapter. It extracts directly into one Transformers-compatible model directory. Both modes implement the same frozen release and must pass the same smoke tests.
3.2 Training data and process evidence
| Artifact | Bytes | SHA-256 | Direct URL |
|---|---|---|---|
| Exact v21.2 train, validation, test, gate, manifest, and quarantine package | 5,969,287 | 91fc18f539b863f9a0e87ed3c372b5f040f359e9bcd427dc506875e1b4f6a467 |
tar.gz |
| Complete 20,047-sentence synthetic process corpus v2 | 22,069,344 | 5db11f1f10f60b1c6df0ea00f0bd4b2f8a313dfc487ae11145c495c96d880e73 |
zip |
| Complete process corpus, alternate format | 19,438,024 | f3d671921944a065e83c11d028471fd7bf0127e6665cae4ded699cd1b23e56ba |
tar.gz |
| All 20,047 sentences | 17,606,394 | b06ec5f5bbc1f4e39cf20a30fefe04d496c42f1408c73e75616ebbd531a714f0 |
JSONL |
| Source-preserving dictionary | 843,498 | d1929dab4cb5cd9b8399f2a63e2567a836a6cb40463ad70afefa4cc6f698e074 |
JSONL |
| Complete corpus and process database | 35,815,424 | b9db281cccfa50ced087747d812b5712ceac22dc7b0cd1f3227d7c2e36988703 |
SQLite |
The model-specific package is the shortest path to the exact final v21.2 treatment. The larger process release is the audit path: it contains the sentence corpus, dictionary, canonical splits, governed v21.1 splits, reviews, revisions, lessons, event logs, audit code, and living books.
3.3 Integrity verification
Verify before extraction or loading:
sha256sum kuku-yalanji-v21.2-guarded-atlas-lora-bundle.tar.gz
sha256sum kuku-yalanji-v21.2-guarded-merged.tar.gz
sha256sum kuku-yalanji-v21.2-training-20260711.tar.gzAfter extracting the Atlas bundle:
sha256sum base/model.safetensors base/tokenizer.json base/config.json
sha256sum adapter/adapter_model.safetensors adapter/tokenizer.jsonThe expected internal identities are recorded in section 5 and in
HOSTING-MANIFEST.json.
4. Loading and serving
4.1 Proven software versions
The completed RunPod lane used:
Python 3.11
PyTorch 2.4.1+cu124
Transformers 4.48.3
PEFT 0.19.1
Accelerate 1.14.0
Datasets 4.8.5
SacreBLEU 2.6.0
GPU NVIDIA A40
The training requirements allowed
transformers>=4.46,<4.49,
peft>=0.18,<1, torch>=2.4, and
sentencepiece>=0.2,<1. A host may use a newer
compatible stack, but it must pin its environment and rerun both smoke
tests before accepting the deployment.
4.2 Dynamic LoRA loading
Extract the Atlas bundle into an empty directory. The archive has
base/ and adapter/ at its root.
import torch
from peft import PeftModel
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
device = "cuda" if torch.cuda.is_available() else "cpu"
tokenizer = AutoTokenizer.from_pretrained(
"./adapter",
src_lang="eng_Latn",
tgt_lang="gvn_Latn",
)
base = AutoModelForSeq2SeqLM.from_pretrained("./base").to(device)
model = PeftModel.from_pretrained(base, "./adapter").to(device)
model.eval()
text = "<translate> The woman saw the water."
inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=200).to(device)
target_id = tokenizer.convert_tokens_to_ids("gvn_Latn")
with torch.inference_mode():
output = model.generate(
**inputs,
forced_bos_token_id=target_id,
num_beams=1,
max_new_tokens=208,
no_repeat_ngram_size=4,
repetition_penalty=1.10,
length_penalty=1.0,
)
print(tokenizer.batch_decode(output, skip_special_tokens=True)[0])Expected output:
Jalbu-ngku bana nyajin.
Atlas or another dynamic host must support:
- encoder-decoder
M2M100ForConditionalGenerationmodels; - PEFT task type
SEQ_2_SEQ_LM; - custom tokenizer files and the added
gvn_Latntoken; - LoRA rank 64 on attention and feed-forward modules;
- forced target BOS configuration;
- visible research-only and non-speaker-certified labeling.
If a platform accepts only a stock upstream model ID plus an adapter,
do not point it at stock NLLB. Upload or mount the included frozen
base/ as the base model.
4.3 Merged-model loading
The merged archive extracts with model files at its root:
import torch
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
device = "cuda" if torch.cuda.is_available() else "cpu"
tokenizer = AutoTokenizer.from_pretrained(
"./kuku-v21.2-merged",
src_lang="eng_Latn",
tgt_lang="gvn_Latn",
)
model = AutoModelForSeq2SeqLM.from_pretrained("./kuku-v21.2-merged").to(device)
model.eval()
inputs = tokenizer(
"<translate> The children are sitting here.",
return_tensors="pt",
truncation=True,
max_length=200,
).to(device)
with torch.inference_mode():
output = model.generate(
**inputs,
forced_bos_token_id=tokenizer.convert_tokens_to_ids("gvn_Latn"),
num_beams=1,
max_new_tokens=208,
no_repeat_ngram_size=4,
repetition_penalty=1.10,
length_penalty=1.0,
)
print(tokenizer.batch_decode(output, skip_special_tokens=True)[0])Expected output:
Karrkay-karrkay yaluy bundanday.
4.4 Minimal HTTP contract
A host-facing translation service should accept a UTF-8 JSON request and identify the immutable model version in every response. A minimal contract is:
{
"text": "The woman saw the water.",
"source_lang": "eng_Latn",
"target_lang": "gvn_Latn",
"model_version": "v21.2-claude-balanced-replay-guarded-20260714"
}The service must prepend <translate> exactly
once, reject empty or over-limit input, force gvn_Latn as
BOS, serialize or bound concurrent generation, and return a retryable
error when capacity is exhausted. It should also record input/output
hashes, model version, decoding configuration, latency, and guard
decisions. Do not log private user text indefinitely without an explicit
retention policy.
5. Model identity and lineage
5.1 Architecture
Upstream family facebook/nllb-200-distilled-1.3B
Transformers class M2M100ForConditionalGeneration
Task conditional sequence-to-sequence translation
Frozen project base v12.0 tagged direct+reference Bible continuation
Source language eng_Latn
Target language gvn_Latn
Vocabulary size 256,205
gvn_Latn token ID 256,204
Input prefix <translate>
Adapter type PEFT LoRA / SEQ_2_SEQ_LM
Merged weight bytes 2,741,395,552
Earlier MobTranslate experiments did use the 600M NLLB model. The v21 family did not: v21.1 and v21.2 both start from the frozen v12 continuation derived from the 1.3B upstream checkpoint. This distinction matters for compatibility, memory planning, attribution, and license records.
5.2 Frozen base hashes
| File | SHA-256 |
|---|---|
base/model.safetensors |
26bb5fc5ca75eca215081699e5d8a1dc64f69153431a4e78d6e87ebaee131a0f |
base/tokenizer.json |
357daed13ffd789678235cb2e8e25a1e57800f569fa59b99339e165e8860db0e |
base/config.json |
846f3f6eeed33aeb5c6260fb0a1316cc19b9888ae6c3a8da5ec61c9239ff1534 |
5.3 v21.2 adapter and merge hashes
| File | SHA-256 |
|---|---|
adapter/adapter_model.safetensors |
aaf155997d3c7b9ccea53d477a59a93f0460dd8d26226b50831233a42db9e5e6 |
adapter/tokenizer.json |
9041375d4d92d6b87628b57b64103d0ce1974559b7ccf146e871656b754fc8ed |
merged model.safetensors |
7f9d0fe325e9e4568e45f13179adb336b93bbd53e83ddab2826e999eba3c76f7 |
5.4 LoRA configuration
rank 64
alpha 128
dropout 0.0
target modules q_proj,k_proj,v_proj,out_proj,fc1,fc2
bias none
weight tying requested true
task type SEQ_2_SEQ_LM
PEFT version 0.19.1
6. Training corpus
6.1 The 20,047-sentence synthetic corpus
The governed synthetic corpus contains 20,047 English-Kuku Yalanji pairs, 135,547 Kuku Yalanji word tokens, and no open review rows at closure. Its final disposition was 19,136 accepted and 911 revised. Whole-corpus lint passed 20,047/20,047 rows. Normalized Kuku-side duplicate count and normalized pair duplicate count were both zero.
The complete process release also preserves:
| Record family | Count |
|---|---|
| Dictionary entries | 2,688 |
| Working lexemes | 1,870 |
| Reviews | 101,235 |
| Revisions | 1,555 |
| Lessons | 8,035 |
| Process events | 4,745 |
Rows were authored and reviewed against the project's source-preserving dictionary, Elisabeth Patz's grammatical description, explicit morphology and frame records, batch review, lint, phantom-lexeme discipline, revision logs, and closure audits. Read the Operator's Guide, Grammar Cheatsheet, and Dictionary Errata for the living linguistic method.
This is process verification, not community certification. The label
project_approved_synthetic_pending_elder_verification must
not be shortened to speaker verified.
6.2 Canonical split and deterministic quarantine
The canonical corpus split was:
train 16,820
development 1,609
test 1,618
total 20,047
The shared v21 leakage treatment preserved 190 excluded rows in a quarantine ledger rather than deleting them:
effective synthetic train 16,642
validation 1,609
effective synthetic test 1,606
quarantine 190
Source keys use NFKC, case folding, normalized quotation marks, and collapsed whitespace. Target-surface keys use NFKC, case folding, and Unicode letters only. After quarantine, train/validation/test intersections were zero for normalized source, target surface, and exact pair.
6.3 v21.2 balanced-replay treatment
The final training mixture per epoch was:
| Component | Rows |
|---|---|
| Leakage-audited synthetic train | 16,642 |
| Bible direct replay | 2,047 |
| Bible reference-conditioned replay | 2,047 |
| Dictionary-usage replay | 1,428 (357 base rows x4) |
| Elder evaluation rows | 0 |
| Total | 22,164 |
The intended canonical replay mix had 22,198 rows. Ten replay
collision records were found before training and preserved in
quarantine/replay_overlap.jsonl: one synoptic Bible surface
and eight dictionary-usage base rows, corresponding to 34 would-be
training rows after paired Bible and x4 dictionary expansion. The
effective replay mixture had zero normalized source, target-surface, or
exact-pair intersections with validation, synthetic test, or the locked
external evaluations. Bible canonical references and dictionary word IDs
were also checked.
6.4 Public and locked data
The public model-specific archive contains the exact 22,164-row train file, 1,609-row validation file, 1,606-row synthetic test, 128-row pipeline gate, both quarantine ledgers, checksums, and the complete construction manifest.
The 43-row elder-shared control and other locked external evaluation files are deliberately excluded. They had zero training rows and remain unavailable as training material so that future evaluation is not invalidated. The full internal evaluation battery also included 84 held-out dictionary-usage rows and two 325-row Bible conditions.
7. How v21.2 was trained on RunPod
7.1 Experimental ownership
Claude in the run ID means the balanced-replay lane was
prepared and operated by Claude under the frozen paired experiment. It
does not mean Claude itself is the translation model. The trained
network is the NLLB-derived model described above. v21.1 was the
Codex-operated synthetic-only lane; v21.2 changed the training
mixture.
7.2 Data builder
The fail-closed builder was:
training/translation/build_v21_2_claude_balanced_replay.py
SHA-256 542b1209e814966a4488620d3d86f297bb5d2c47b1374c6b6688cf2b0cee2930
It refused a nonempty output directory, verified the final corpus audit, pinned input and frozen-base hashes, reproduced the v21.1 quarantine, checked replay against every held-out family, stable-sorted by a recorded seed, asserted row counts and unique IDs, and emitted JSONL plus SHA-256 manifests.
The builder invocation was:
python3 training/translation/build_v21_2_claude_balanced_replay.py \
--output-dir /workspace/v21.2-claude/dataFor external reproduction, download the already built exact training archive. Rebuilding from raw sources requires the governed internal source roots and locked controls; it is intentionally not equivalent to concatenating the public 20,047-row file with arbitrary replay data.
7.3 Mandatory gate
The full run could not start until a deterministic 128-row mixed-domain gate passed:
gate composition 96 synthetic, 13 Bible direct, 12 Bible ref, 7 dictionary usage
gate epochs 100
gate learning rate 1e-4
acceptance 128 rows, 0 empty, >=115 exact, chrF >=97, length ratio 0.90-1.10
observed 128/128 exact, chrF 100.0, mean length ratio 1.0
The gate establishes pipeline and optimization viability. It is deliberate memorization and is not a generalization score.
7.4 Full optimization contract
train / validation / test 22,164 / 1,609 / 1,606
epochs 3
learning rate 2e-5
batch size 8
gradient accumulation 2
effective batch 16
warmup 0
weight decay 0
source / target cap 192 / 208 tokens during training
LoRA rank / alpha / dropout 64 / 128 / 0.0
seed 42
optimizer steps 4,155
selection validation chrF
archived evaluation decoding beam 1, no repetition constraints
The completed A40 run recorded:
training runtime 3,431.87 seconds (57.20 minutes)
training samples/second 19.375
training loss 1.199454
recorded training FLOPs 2.609e16
peak GPU memory 30,383 MiB
7.5 Full driver
The completed orchestration driver was:
training/translation/run_v21_2_claude_balanced_replay.sh
SHA-256 81beaf25d64b72cfce1a1aef14ff1566a04ffaa0b1df634587505b4bd071ef67
Its logical invocation was:
PYTHON=/opt/mobtranslate-mt-venv/bin/python \
DATA_ROOT=/workspace/v21.2-claude/data \
BASE_MODEL=/workspace/v21.2-claude/base_model/merged \
OUT_ROOT=/workspace/v21.2-claude/models \
LOG_ROOT=/workspace/v21.2-claude/logs \
REPORT_ROOT=/workspace/v21.2-claude/reports \
bash training/translation/run_v21_2_claude_balanced_replay.shThe driver verified checksums, ran tokenizer-length preflight,
executed the mandatory gate, trained, merged the adapter, evaluated all
frozen sets, retained row-level predictions, sampled GPU/CPU/RAM/disk
resources every five seconds, and wrote RUN_COMPLETE only
after every stage succeeded.
7.6 Reproducibility boundary
The sealed record includes the exact data manifest, data checksums, model manifests, adapter, merged model, row-level predictions, metric outputs, resource observations, and final archive checks. It did not copy and hash every Python helper into the v21.2 run directory. The builder and driver above have pre-run timestamps and recorded hashes; shared trainer/evaluator files were modified later for subsequent work. Therefore this guide does not claim that the current repository contains a byte-identical snapshot of every executed helper. This is a provenance gap, not a reason to alter the historical run. Future runs must seal all executed source files and environment locks in their archive before pod deletion.
8. Evaluation
8.1 Frozen automatic results
| Set | Rows | v21.2 BLEU | v21.2 chrF++ | Exact | Interpretation |
|---|---|---|---|---|---|
| Synthetic development | 1,609 | 26.20 | 52.85 | 396 | Same authored process as training |
| Synthetic test, tagged | 1,606 | 26.84 | 53.84 | 426 | Primary synthetic held-out |
| Synthetic test, untagged | 1,606 | 26.77 | 53.81 | 429 | Prefix-dependence control |
| Elder-shared sentence pairs | 43 | 6.52 | 29.36 | 0 | Natural-register transfer remains weak |
| Dictionary usage | 84 | 11.58 | 47.14 | 3 | Replayed-domain retention |
| Bible direct | 325 | 12.36 | 43.78 | 0 | Replayed-domain retention |
| Bible reference-conditioned | 325 | 12.41 | 43.91 | 0 | Replayed-domain retention |
All fourteen model/set combinations in the v21.1-v21.2 comparison had zero empty outputs. BLEU and chrF++ measure surface overlap; they do not establish adequacy, grammaticality, morphology, pragmatics, dialect fit, cultural appropriateness, or community acceptance.
8.2 v21.2 versus v21.1
Relative to the synthetic-only v21.1 ablation, v21.2 changed corpus chrF++ by:
synthetic tagged test -0.87
synthetic untagged test -0.58
elder-shared +0.56
dictionary usage +12.74
Bible direct +15.23
Bible reference +15.15
On the primary tagged synthetic set, paired mean sentence chrF++ was +0.082 for v21.2 with 95% bootstrap interval [-0.225, +0.396]. That is synthetic parity, not a clean win. The dictionary and Bible retention gains were broad and large, which is why v21.2 is the better practical recipe. v21.2 had 4,155 optimizer steps versus v21.1's 3,120, so the comparison does not isolate replay composition from update budget.
Read the independent paired comparison and the broader translation-systems benchmark for bootstrap methods, linguistic slices, LLM post-edit comparisons, and the routing decision.
8.3 Guarded decoder transfer
The v22 step-matched run did not produce a promotable checkpoint, but its development-only decoder selection yielded a transferable serving policy. A separate protocol froze that policy before any v21.2 transfer output was generated. The exact published v21.2 merged model hash was verified before inference.
| Set | Greedy chrF | Guarded chrF | Exact greedy / guarded | Repeated-segment rows greedy / guarded |
|---|---|---|---|---|
| Synthetic dev | 52.8471 | 55.1631 | 396 / 394 | 39 / 1 |
| Synthetic tagged | 53.8374 | 55.7097 | 426 / 426 | 37 / 2 |
| Synthetic untagged | 53.8142 | 55.7278 | 429 / 422 | 36 / 2 |
| Elder diagnostic | 29.3558 | 29.3565 | 0 / 0 | 0 / 0 |
| Dictionary usage | 47.1391 | 47.9648 | 3 / 1 | 0 / 0 |
| Bible direct | 43.7751 | 43.7197 | 0 / 0 | 2 / 0 |
| Bible reference | 43.9093 | 43.8353 | 0 / 0 | 1 / 0 |
All seven sets had zero empty outputs and all 14 frozen transfer
checks passed. In 50,000 paired bootstraps, the mean per-sentence chrF++
interval was positive on tagged synthetic test
[+0.097, +0.560]; intervals for untagged, elder, usage, and
both Bible controls included zero. The exact-match decreases are visible
in the table and were not used as tuning criteria.
The policy is an inference intervention on unchanged weights. Kuku
Yalanji has linguistically meaningful reduplication, so
no_repeat_ngram_size=4 remains subject to speaker review
for false suppression of valid morphology. Read the full v22 and decoder-transfer
report.
8.4 Isolated lexicon and elder version probe
A separately preregistered diagnostic compared the reconstructed step-2,770 checkpoint, the failed v22.0 step-3,120 checkpoint, and the current step-4,155 v21.2 weights. All three models used the same guarded decoder: one beam, four-token no-repeat guard, repetition penalty 1.10, length penalty 1.0, and maximum 208 new tokens. There were no empty outputs in any of the 1,020 model-row evaluations.
The lexicon builder began with 1,640 JSON-LD entries, retained 395 lowercase single-gloss lexical entries, and grouped synonymous headwords into 297 unique English prompts. A response counted as normalized exact when it matched any canonical Kuku Yalanji headword attached to that gloss. None of the 297 English prompts occurred as an isolated training input, but the dictionary informed earlier corpus work, so this is a coverage probe rather than an untouched test set.
| Model checkpoint | Accepted exact | Mean maximum sentence chrF++ | Mean minimum CER | Empty |
|---|---|---|---|---|
| Step 2,770 | 42/297 (14.14%) | 28.06 | 0.807 | 0 |
| Step 3,120 | 43/297 (14.48%) | 28.83 | 0.811 | 0 |
| Step 4,155 guarded | 48/297 (16.16%) | 30.38 | 0.782 | 0 |
On identical rows, step 4,155 exceeded step 3,120 by 1.68 exact-match percentage points; the 10,000-sample paired bootstrap interval was [+0.34, +3.37]. Its mean sentence chrF++ difference was +1.55 with interval [+0.37, +2.95]. This is evidence that continued exposure improved canonical lexical retrieval on this probe. It does not establish that step 4,155 generates better free translations.
The exposure split is the more important result. Step 4,155 retrieved 46/192 (23.96%) accepted forms that appeared as full tokens somewhere in training, but only 2/105 (1.90%) forms not observed that way. Excluding 14 identity/loanword cases, exact retrieval was 43/283 (15.19%). A lookup-first dictionary route is therefore not just an optimization: it is required for reliable known-word answers.
The same run evaluated the 43 rights-cleared elder sentence pairs, which were excluded from training but had been observed in earlier evaluations:
| Model checkpoint | Exact | Corpus chrF++ | Mean sentence chrF++ | Empty |
|---|---|---|---|---|
| Step 2,770 | 0/43 | 29.18 | 29.63 | 0 |
| Step 3,120 | 0/43 | 29.26 | 29.66 | 0 |
| Step 4,155 guarded | 0/43 | 29.43 | 29.89 | 0 |
Every paired elder interval included zero. For step 4,155 minus step 3,120, the mean sentence chrF++ difference was +0.23 with interval [-0.42, +0.99]. There is no evidence here of improved natural elder register.
The public audit package includes the preregistration, run contract, 340-row combined input, row-level predictions and paired statistics, concise results, and package checksums.
8.5 Sealed RunPod reproduction and fast feedback loop
The complete three-checkpoint probe was rerun twice on an exact NVIDIA A40 runtime with deterministic float32 inference, sampling and TF32 disabled, and all behavior-affecting model files pinned. The final A40 outputs matched the frozen local CPU baselines on all 1,020/1,020 rows. The two A40 executions also matched each other on all 1,020/1,020 rows. The final run's output inventory passed remotely and again after download.
The portable kit verifies input/model hashes and runtime versions,
evaluates all manifest entries, computes the paired analysis, compares
GPU rows with local baselines, records real GPU/process resources, and
writes RUN_COMPLETE only after its own checksum inventory
verifies. All three inference passes took about 3.5 minutes; a new
checkpoint already present on a training pod should return a sealed
result in roughly one to two minutes.
Read the complete RunPod evaluation and fast-loop manual, or download the sealed kit and A40 result bundle.
8.6 Known failure modes
- Both v21 models are 0/43 exact on the elder-shared control.
- The current model retrieves only 48/297 accepted dictionary forms on the isolated-gloss probe, falling to 2/105 when the target form did not appear as a full training token.
- v21.2's mean elder output length is about 68% of reference length; multi-clause content is often compressed.
- The archived greedy policy produced 37 tagged-test repeated-segment rows; the guarded policy reduced this to 2 but does not guarantee that every novel output is loop-free.
- The no-repeat guard can in principle suppress valid reduplication and needs fluent-speaker error analysis.
- Replay improves adjacent-domain retention but does not prove unseen-domain or natural-register transfer.
- The synthetic test reflects the same authoring process and has strong internal difficulty stratification.
- One seed and one run per treatment do not estimate run-to-run variance.
- No reverse-direction, calibrated uncertainty, arbitrary-input robustness, or complete speaker evaluation is claimed for this release.
9. Production safeguards
A responsible integration must use the model as one component in a routed translation system:
- Return an exact rights-cleared sentence/resource match when one exists.
- Return a unique exact dictionary match for a clearly lexical query.
- Generate only for novel text within the supported direction and length.
- Reject empty output, target-token failure, length-limit termination, severe under-translation, severe over-generation, and repeated-segment loops.
- Label every accepted generation as a research draft and expose model version and provenance.
- Preserve the source, draft, guard result, and any speaker correction as separate versioned records.
- Never promote a generated form into the dictionary or training corpus without adjudication, rights metadata, dialect/register metadata, and a supersession trail.
LLM post-editing remains experimental. On the frozen 43-row control, Codex and Claude post-editors improved point scores, but neither route was speaker-reviewed and their mutual difference did not cleanly exclude zero. Do not silently add a hosted LLM post-editor and present the result as verified language.
10. Rights, attribution, and cultural responsibility
The upstream NLLB checkpoint is distributed under CC-BY-NC-4.0. This release is therefore for noncommercial research use. The complete process corpus does not assert a blanket Creative Commons or public-domain license over community language knowledge, dictionary material, grammatical descriptions, or third-party sources. The download does not relicense those sources.
Each training row carries its own provenance and rights fields.
Synthetic rows are labelled
project_approved_synthetic_pending_elder_verification;
replay rows retain their source rights metadata. Public availability is
not speaker endorsement and not an unrestricted commercial grant. Before
redistribution, production deployment, or derived commercial work,
conduct an independent rights review and consult the relevant Kuku
Yalanji community and source rights holders.
Do not use this model for high-stakes communication, legal or medical interpretation, cultural authority, language teaching presented as certified, or replacement of fluent-speaker consultation.
11. Versioning and update procedure
The immutable version string, manifests, checksums, and archives define this release. Do not overwrite an archive in place. A future model must receive a new version, new artifact URLs, new checksums, a new model card/guide, and a registry entry that keeps v21.2 available as historical evidence.
For each future run:
- Freeze and checksum data before training.
- Seal every source file, dependency lock, container image ID, environment variable contract, and command.
- Keep train, validation, test, locked external controls, and quarantine explicitly separated.
- Record failed gates and provisioning failures as evidence.
- Retain adapter, merged model, row-level predictions, metrics, resources, and all checksums.
- Run speaker evaluation before any promotion claim.
- Build immutable host bundles and verify full archive reads.
- Add the release to
apps/web/public/models/registry.json. - Rerun
render-docs.shso the live books and raw Markdown update together. - Verify API JSON, CORS, complete downloads, byte ranges, checksum values, smoke outputs, and responsive pages.
12. Related records
- Model registry and downloads
- v22 step-matched replay and decoder-transfer experiment
- Independent v21.1-v21.2 model comparison
- Translation systems research and benchmark
- Synthetic corpus operator guide
- Mega grammar cheatsheet
- Dictionary errata and phantom watchlist
- Sample production pages
- Mi'gmaq model training handoff
- Mi'gmaq v1 model card
- Mi'gmaq evaluation and promotion report
- RunPod evaluation and fast feedback loop
- MobTranslate Next 16 build incident and deployment runbook
- Raw Markdown for this guide
13. Live distribution deployment verification
The public distribution, inference, documentation, and
activity-logging surfaces were rebuilt and verified end to end on
2026-07-14 UTC. This is an operational verification of delivery and
metadata integrity; it is not an additional linguistic evaluation or a
promotion of the model beyond research-only.
13.1 Build and service gate
- Next.js 16.2.9 completed an optimized Turbopack build for all 361 static and dynamic routes. The final build compiled in 74 seconds, completed strict TypeScript in 3.1 minutes, and generated 361/361 pages in 55 seconds. End-to-end wall time was 402 seconds while the shared data disk was under independently measured contention.
- The immutable deployed build ID is
ZS7-SuKDwqjovoz_tBnyE. It was built into the isolated.next.release-turbopack-final-20260714T1347Ztree, started on port 3310, and became ready in 500 ms before the atomic production swap. The immediately precedingLUCL6-Xz-yvnyYPvmgJJatree remains at/mnt/donto-data/workspace/.build-archive/mobtranslate.com/20260714/.next.rollback-LUCL6-Xz-yvnyYPvmgJJa-20260714T135544Z. - Thirteen focused model-distribution, bilingual Discord, saved-result, and contact-link tests passed across four files. The production build supplied the strict TypeScript and prerender gates.
mobtranslate-web.serviceis active on Caddy's configured upstream127.0.0.1:3300, has zero restarts, and became ready in 299 ms after the final swap. Both custom CPU inference services,kuku-v2-infer.serviceandmobtranslate-migmaq-v1-infer.service, are inactive and disabled; ports 7955 and 8765 have no listeners.- The homepage translation route now uses the complete selected
dictionary with
gpt-5.4-miniand a required, Zod-validated tool call. The custom-model lab and versioned inference routes fail closed with explicit 503not_configuredresponses while model serving is paused. - The final browser has no application console errors. Vercel
Analytics is now rendered only when
VERCELis set, avoiding a failed/_vercel/insights/script.jsrequest on this self-hosted deployment. - The only retained notices are Next's deprecated
middleware.tsconvention and Node's experimental SQLite API.
The build incident was not a single compiler defect. The previous
command forced Webpack on a Next 16 application, an obsolete custom
webpack alias duplicated native monorepo support, Tailwind v4 could
discover retained .next.* rollback trees inside the source
repository, and broad runtime filesystem access inflated Next file
traces. Retained builds now live outside the repository,
.next.*/ is ignored, the app uses native
transpilePackages plus a pinned Turbopack root, and runtime
filesystem boundaries are explicit. All 207 final route trace manifests
were audited; none retained an unscoped project or
prior-.next trace. The full evidence and recovery procedure
are in the build
incident runbook.
13.2 API contract gate
The live assertions covered all of the following:
| Check | Observed result |
|---|---|
GET /api/v1/models |
200; two model families |
| Registered releases | 21 total; 20 Kuku Yalanji and one Mi'gmaq |
Kuku Yalanji versions/latest |
v21.2-claude-balanced-replay-guarded-20260714 |
Mi'gmaq versions/latest |
1.0.0-rc1 |
| Rejected v22.0 record | Public negative result; no hostable model download |
| Rejected v23.0 record | Public attested-adaptation report; all candidate weights deleted |
| Historical v21.2 record | Original greedy serving-policy release retained immutably |
| Version benchmark metadata | 48/297 exact; 16.1616%; frozen row-level audit artifacts |
| OpenAPI | 200; OpenAPI 3.1.0; canonical server
https://mobtranslate.com |
OPTIONS |
204 with Access-Control-Allow-Origin: * |
HEAD |
200 with the public CORS contract |
| Unknown model and version | structured 404 responses with distinct error codes |
| Shared-cache policy | public, s-maxage=300, stale-while-revalidate=3600 |
| Private-path audit | no localPath, /mnt/,
localhost, 127.0.0.1, or port 7955 in public
JSON |
| Homepage prompt translation | 200 with nonempty structured output and English gloss |
| Versioned custom inference | Explicit 503 not_configured; no private endpoint
exposed |
The route resolves public links from the configured
NEXT_PUBLIC_APP_URL=https://mobtranslate.com and validates
every fallback origin. Public model JSON and custom-inference responses
expose neither reverse-proxy links nor a private inference service
location.
13.3 Artifact and metadata gate
- Every registered release was fetched through its literal version endpoint.
- The registry contains 147 artifact records across 21 releases. Of these, 129 records are marked available and resolve to 125 unique public URLs. The current release contributes 33 artifacts: 16 evaluation, six documentation, six metadata, three dataset, one bundle, and one standalone-model record.
- Every benchmark package member reverified against
SHA256SUMS; failures: zero. The public result Markdown, analysis JSON, frozen input, manifest, preregistration, run contract, and prediction hashes match the registry. - The sealed RunPod A40 package evaluated 340 rows at each of three checkpoints, producing 1,020 predictions. Both A40 executions matched each other and the frozen local CPU baselines on all 1,020 rows.
- The v21.2 Atlas and merged archives passed full local archive
traversal. Their live endpoints expose the exact advertised
Content-Length; the Atlas endpoint also returned 206 with a correct 1,024-byte range and total size. - The exact training archive passed full traversal and its live endpoint reported the advertised size. The raw Markdown guide and experiment report downloaded byte-identically to their rendered source copies.
- Safe live Discord webhook probes showed English and Indigenous-language fields for translation and TTS. Unit coverage verifies the same bilingual labelling for recording uploads; chat and translation routes pass both sides into the shared logger.
13.4 Browser gate
The homepage, registry, translation lab, documentation index, and this guide were inspected in real Chromium sessions, including the final live build at 1280x720 and 390x844:
/modelshad no horizontal overflow at either width; long model/version identifiers wrapped without overlap.- The model API, hosting guide, v22 negative-result report, OpenAPI, model archive, benchmark inputs, predictions, checksums, dataset, and evaluation links were present with canonical public URLs.
- Light and dark themes were legible on the registry and guide.
- Responsive checks found no horizontal overflow or incoherent overlap on the tested registry, lab, or docs pages.
- At 1280 px the compact navigation remained available without clipping. At 390 px the menu exposed all nine destinations -- About, Atlas, Dictionaries, Map, Spread, Translate, Models, Learn, and Community -- without horizontal overflow or obscured controls.
/docs/linked the rendered guide and v22 report together with their raw Markdown sources.
14. Citation record
When reporting this release, identify at least the model ID, immutable version, direction, upstream lineage, training-mixture count, evaluation scope, status, and access date:
MobTranslate. 2026. Kuku Yalanji NLLB-LoRA,
v21.2-claude-balanced-replay-guarded-20260714 (byte-identical weights from
v21.2-claude-balanced-replay-v2-candidate). English (eng_Latn) to Kuku Yalanji
(gvn_Latn). Noncommercial research release, not speaker-certified.
https://mobtranslate.com/api/v1/models/kuku-yalanji-nllb-lora/versions/
v21.2-claude-balanced-replay-guarded-20260714