MobTranslate Kuku Yalanji Model Program: Technical Audit, Attempt History, and v24 Training Plan

Prepared for: independent technical, linguistic, statistical, and operational review
Prepared: 2026-07-15 UTC
Revision: 2, after independent-review disposition; revision 1 SHA-256 was 565cb8d70aecec2095ba4da9b9324595f61c0b447b759b7a84dee43a8ebb18b1
Public canonical URL: https://mobtranslate.com/trainignplan.md
Rendered edition: https://mobtranslate.com/docs/trainignplan.html
Review disposition: https://mobtranslate.com/docs/kuku-v24-review-disposition.html
Language direction covered: English (eng_Latn) to Kuku Yalanji (gvn_Latn)
Current model of record: v21.2-claude-balanced-replay-guarded-20260714
Current deployment decision: downloadable research artifact; custom Kuku Yalanji CPU inference is stopped and disabled
Next proposed experiment: v24 lexical reconstruction, lexicon uptake, and sentence-retention study with fixed-token treatment comparisons
Certification: none. No model described here is speaker-certified, community-approved, or suitable for authoritative translation.

Revision note: the independent review was treated as advice, not a specification. The project adopted corrections that improve scientific validity or remove unsupported public claims, modified recommendations that were too broad, and deferred new architectures and review infrastructure that are not needed for the narrow screen. Automatic scores remain engineering measurements, not evidence of community authority.

1. Executive assessment

MobTranslate has built a substantial and unusually well-documented low-resource translation research program, but it does not yet have a reliable free-form Kuku Yalanji translator.

The strongest retained custom model is an NLLB-derived 1.3B sequence-to-sequence model, not a chat LLM. Its v21.2 weights were trained on a leakage-audited 20,047-pair synthetic corpus treatment plus controlled replay. Under the current guarded decoder it performs reasonably on held-out material produced by the same synthetic authoring process, and replay preserved dictionary-usage and Bible domains. It still fails the decisive lexical and natural language tests:

At 297 benchmark rows, the deployment policy requires at least 252 accepted exact outputs. That is an observed rate of 84.85%, because the two-sided Wilson 95% lower endpoint must itself be at least 80%. A bare point estimate of 80% would not pass. The current result is 204 exact answers short of the first passing count.

The central training-design error is now clear. The project has 2,688 dictionary entries and 1,870 working lexemes, but v21.2 did not train every eligible dictionary mapping as an isolated English-to-Kuku Yalanji sequence pair. It replayed only 357 dedicated dictionary-usage examples four times. Most dictionary vocabulary appeared only incidentally inside sentences. The frozen benchmark then showed exactly the expected exposure dependence:

Target headword appeared as a complete training token: 46/192 exact (23.96%)
Target headword not observed as a complete training token: 2/105 exact (1.90%)

The next model should therefore include explicit lexical pairs under a task shape distinct from sentence translation. In a training JSONL record, the source and target are separate fields:

{
  "id": "lexeme:dictionary-123:canonical",
  "direction": "eng-gvn",
  "input_text": "<lexeme> woman",
  "unconditioned_input_text": "woman",
  "output_text": "jalbu",
  "pair_kind": "dictionary_lexeme",
  "source_lang": "eng_Latn",
  "target_lang": "gvn_Latn"
}

The conceptual pair is woman -> jalbu. The literal text woman : jalbu should not be put into the source field. Training and serving shapes must match, and <translate> remains reserved for sentence translation.

This lexical objective must not be confused with sentence translation. Memorizing dictionary entries can improve closed-set reconstruction and may help sentence vocabulary, but deterministic lookup remains safer for known words. Lexical reconstruction does not by itself teach case marking, agreement, derivation, tense/aspect/mood, argument structure, discourse continuity, word order, dialect choice, or idiomatic usage. v24 therefore reports separate tasks:

1. direct lexical reconstruction after an explicit <lexeme> pair;
2. contextual lexical acquisition where a form occurs in sentences but not as an isolated pair;
3. lexicon uptake where a held-out mapping is supplied at inference under <glossary>;
4. morphological generalization only where a governed lemma/form analysis exists; and
5. sentence retention under the unchanged decoder.

A fully hidden arbitrary mapping is not a productive-generalization test: jalbu cannot be derived from the English word woman. Exact success there mostly reflects prior exposure, leakage, cognacy, or chance. The all-lexicon fit is therefore scored as closed-set reconstruction and never as unseen-language generalization.

For known dictionary queries, deterministic database lookup remains the technically correct first route and can be near-exact without asking a neural model to regenerate stored facts. A model may be evaluated on sentence drafts, but it cannot enter that route unless a separate natural-text, degeneration, and speaker-review gate passes.

2. Review questions and disposition

The original dossier asked an external reviewer the following questions. The resulting adopt/modify/defer/reject decisions are preserved in the independent-review disposition:

1. Is the evidential separation among synthetic, dictionary, Bible, usage-example, and native-speaker narrative material linguistically defensible?
2. Are the normalization, grouping, leakage, and multi-reference rules for isolated lexical evaluation valid?
3. Can the v24 tasks distinguish closed-set reconstruction, contextual acquisition, glossary uptake, morphology, and sentence retention without claiming an arbitrary hidden mapping is inferable?
4. Is NLLB-200 distilled 1.3B plus LoRA still a defensible architecture, or should a controlled alternative be added without discarding the established baseline?
5. Are the proposed task prefixes, replay ratios, fixed-step controls, and three-seed design sufficient to estimate the effect of lexical supervision?
6. Are chrF++, BLEU, exact match, CER, length, and repetition diagnostics being interpreted within their limits?
7. What speaker-review instrument should be used for semantic adequacy, grammaticality, morphology, variety, register, discourse, and cultural acceptability?
8. Are data rights and community governance strong enough for the proposed research use and public artifacts?
9. Which separate gates should govern deterministic lookup, model lexical reconstruction, and model sentence generation?
10. Which assumptions below are weak enough that v24 should not start until they are resolved?

3. Claims, non-claims, and terminology

3.1 Claims supported by the record

• The governed synthetic corpus contains exactly 20,047 English-Kuku Yalanji pairs, not 210,000 pairs.
• The corpus closure audit passed 19 checks with no failed checks.
• v21.2 is the best retained custom research recipe under the existing comparison battery.
• v21.2 retained adjacent replay domains much better than the synthetic-only v21.1 treatment.
• v21.2 does not meet the project's dictionary reliability requirement.
• v21.2 and v23 do not establish natural elder-register translation competence.
• Guarded decoding materially reduced severe repetition on unchanged v21.2 weights.
• The v23 attested-narrative adaptation failed its preregistered research gate and lexical-model gate.
• v23 candidate weights and checkpoints were deleted after compact evidence was verified.
• Current live homepage translation uses a dictionary-guided hosted LLM route, while custom Kuku CPU services are stopped, disabled, and blocked by a model-bound gate.

3.2 Claims not supported

• The 20,047 synthetic pairs are not 20,047 independently speaker-attested utterances.
• Process review is not elder review or community certification.
• A high synthetic chrF++ score is not a population estimate for natural Kuku Yalanji.
• Exact dictionary recall is not sentence-level grammatical competence.
• Claude in the v21.2 name does not mean that Claude is the deployed translation network. It identifies the operator/preparation lane. The network is NLLB-derived.
• The custom gvn_Latn token is a project tokenizer extension, not evidence that upstream NLLB natively supports a complete Kuku Yalanji model.
• No reported result applies automatically to Kuku Yalanji-to-English translation.
• Public downloadability is not a commercial license or speaker endorsement.

3.3 Language and variety scope

The project uses ISO 639-3 gvn and the custom NLLB token gvn_Latn as engineering identifiers. Those identifiers are coarser than the linguistic variation represented in the sources. Patz narrative material includes Yalanji and Nyungkul labels, with different speakers and observable form differences. The 43 elder-shared sentences, dictionary, Bible, synthetic corpus, and Patz clauses should not be pooled under an assumption of homogeneous speaker, community, topic, register, orthography, or variety.

Every future row should carry, where known:

• source work and exact location;
• speaker and recorder/editor;
• variety or dialect label as given by the source;
• orthographic system and normalization history;
• genre, register, and discourse context;
• rights and approval state;
• whether the text is attested, reconstructed, synthetic, translated, or mechanically extracted;
• review and uncertainty flags;
• supersession history rather than destructive replacement.

3.4 Governance and release boundary

The dossier does not establish a formal Kuku Yalanji authorizing body, negotiated governance agreement, or scope of community approval. That absence is not proof that no relationships exist; it means the public evidence is insufficient for platform-wide claims of community ownership, certification, or authority.

The project will not invent an approval record. Unresolved governance blocks public model release and transfer of speaker, restricted, or uncertain material to third-party providers. It does not block private builder, evaluator, and method work over material whose research use is already documented. Uncertain records are excluded or quarantined. Before public release, permissions for training, redistribution, derived weights, hosted-provider transfer, publication, withdrawal, and culturally sensitive material must be recorded at the applicable source and scope level.

4. Current operational state, 2026-07-15 UTC

The current root-disk margin is too small for casual model duplication. New research artifacts must remain on /mnt/donto-data; large candidate weights should remain on RunPod until their compact evaluation result passes. The deprecated RunPod list command still exposes an old EXITED v20 record, but the current pod-list endpoint has no active pod.

The live model API is:

https://mobtranslate.com/api/v1/models
https://mobtranslate.com/api/v1/models/kuku-yalanji-nllb-lora
https://mobtranslate.com/api/v1/models/kuku-yalanji-nllb-lora/versions/latest
https://mobtranslate.com/api/v1/models/openapi.json

The latest alias currently resolves to:

v21.2-claude-balanced-replay-guarded-20260714

Hosts should pin the literal version and SHA-256, not the mutable latest alias.

5. Data inventory

5.1 Governed synthetic corpus

The completed process corpus contains:

The canonical split is:

train        16,820
development   1,609
test          1,618
total        20,047

The v21 leakage treatment preserved rather than deleted 190 collisions:

effective synthetic train   16,642
validation                   1,609
effective synthetic test     1,606
quarantine                     190

After quarantine, train, validation, and test had zero intersections under the recorded normalized source, target-surface, and exact-pair keys. The corpus content identity recorded by the closure audit is:

cb32b539e0bf9763a707afe22d702c9520f5b41ec244ea8e63013fee2798763a

The synthetic corpus was authored against the source-preserving dictionary, Elisabeth Patz's grammar, explicit morphological and frame records, batch review, lint, phantom-lexeme controls, revision ledgers, lessons, and closure audits. That is a strong process record. It does not remove model-assisted authoring bias, shared-template effects, possible analysis errors, or dependence between training and synthetic evaluation.

5.2 Dictionary

The working dictionary has 2,688 entries and 1,870 working lexemes. It includes canonical headwords, parts of speech, phonemic forms where available, English glosses and definitions, semantic domains, commentary, loanword notes, and source information. Not every dictionary entry is a valid one-to-one training pair:

• some English glosses map to multiple canonical Kuku Yalanji headwords;
• one headword may have multiple senses;
• names, abbreviations, particles, bound forms, and multiword explanations need separate treatment;
• variant, dialect, register, and uncertainty information must not be collapsed;
• an English definition can be descriptive rather than a substitutable translation;
• isolated lemma identity does not encode inflected or derived forms.

The existing frozen lexical probe deliberately used a narrow, reproducible subset. It began with 1,640 JSON-LD graph entries, retained 395 entries where both headword and English definition were a single lowercase Unicode word optionally containing an internal hyphen/apostrophe, and grouped them into 297 unique English prompts. Every canonical headword sharing a prompt was accepted. This is the origin of the current 297-row model lexical-reconstruction gate.

The v24 builder should begin from all 2,688 source entries, emit an eligibility and exclusion ledger, and construct more than one lexical task class. It must not silently claim that all entries are simple bilingual equivalents.

5.3 Database usage examples

The original database export contains 449 English-Kuku Yalanji usage examples. The v21.2 effective training mixture used 357 non-colliding training rows, replayed four times for 1,428 row presentations per epoch. An 84-row held-out set was grouped by dictionary word ID. These examples are high-value because they are short, relatively literal, and closer to everyday dictionary use than Bible verses, but they are not a substitute for broad connected speech.

5.4 Elder-shared material

There are 43 rights-cleared connected-narrative sentence pairs used as a frozen diagnostic. They have been observed through many iterations, so they are no longer a blind final test. They remain valuable as a stable regression set. v18 was trained directly on them and reached 43/43 exact, proving memorization; later models that excluded them returned 0/43 exact. Neither outcome estimates performance on unseen elder speech.

5.5 Patz narrative clauses

Clause-level extraction from Patz supplied the v23 natural-text experiment:

The split was speaker-disjoint, but speaker, story, topic, variety, and split were confounded. All texts share one published grammar's transcription/editorial tradition. One text is one cluster; 56 clauses are not 56 independent speakers. The v23 row bootstrap is therefore descriptive over clauses, not a community-level uncertainty estimate.

5.6 Bible corpus

The eBible asset contains 8,297 unique Kuku Yalanji verses aligned to two English editions, producing 16,597 English-to-Kuku rows. It was critical for early pipeline development and target-language form learning, but it is a free, interpretive, domain-specific translation. It can restructure clauses, expand content, and introduce biblical register. Early mixtures were approximately 94% Bible by row count, and small literal/narrative sets were washed out.

The v24 positive objective should give Bible rows zero weight. A frozen Bible held-out may remain only as a broad catastrophic-forgetting diagnostic. It must not select the model, compensate for failed natural-text endpoints, or be described as everyday-language competence.

5.7 XIGT, historical material, and audio

• About 795 automatically extracted interlinear examples exist in XIGT form. They mix paradigms, words, phrases, clauses, interleaved glosses, OCR substitutions, and uncertain segmentation. They are audit material until clause-level linguistic reconciliation is complete.
• Historical mission papers and dictionaries are valuable diachronic sources but need orthographic, temporal, genre, rights, and provenance labels. They must not be normalized into contemporary training without review.
• Synthetic TTS and human recordings are separate speech assets. They do not become text-translation supervision merely because audio exists.

5.8 Public data artifacts

6. Training architecture and harness

6.1 Current model lineage

Upstream family       facebook/nllb-200-distilled-1.3B
Transformers class    M2M100ForConditionalGeneration
Project base          frozen v12 Bible continuation
Source token          eng_Latn
Target token          custom gvn_Latn, ID 256204
Task                   conditional sequence-to-sequence translation
Adapter                PEFT LoRA, SEQ_2_SEQ_LM
Serving input prefix  <translate> 

The v21.2 adapter cannot safely be attached to untouched stock NLLB. The frozen project base includes the added target token and project-specific weights. Dynamic hosts must load the distributed exact base and adapter together, or use the merged model.

6.2 v21.2 LoRA and optimization

LoRA rank / alpha / dropout   64 / 128 / 0.0
Target modules                q_proj,k_proj,v_proj,out_proj,fc1,fc2
Epochs                        3
Learning rate                 2e-5
Batch / accumulation          8 / 2
Effective batch               16
Warmup / weight decay         0 / 0
Source / target cap           192 / 208
Seed                          42
Optimizer steps               4,155
Checkpoint selection          validation chrF++

The exact merged weight hash is:

7f9d0fe325e9e4568e45f13179adb336b93bbd53e83ddab2826e999eba3c76f7

6.3 v21.2 effective mixture

Across three epochs this is 66,492 row presentations. It is not 66,492 unique sentence pairs. The intended canonical mix was 22,198 rows per epoch; one synoptic Bible collision and eight database base rows were quarantined, excluding 34 expanded rows while preserving their records.

6.4 Executed trainer behavior

The trainer reads structured JSONL, filters the requested direction, tokenizes input_text and output_text separately, applies LoRA, evaluates with SacreBLEU, saves adapter and merged forms, and writes a model manifest with data hashes, environment, training arguments, metrics, and artifact paths. It also contains an NLLB merge canonicalization fix so serialized shared/encoder/decoder embeddings cannot silently diverge after PEFT merge.

Core executed preprocessing is equivalent to:

def preprocess(batch):
    tokenizer.src_lang = "eng_Latn"
    tokenizer.tgt_lang = "gvn_Latn"
    model_inputs = tokenizer(
        [" ".join(text.split()) for text in batch["input_text"]],
        max_length=192,
        truncation=True,
    )
    labels = tokenizer(
        text_target=[" ".join(text.split()) for text in batch["output_text"]],
        max_length=208,
        truncation=True,
    )
    model_inputs["labels"] = labels["input_ids"]
    return model_inputs

This is why the colon notation matters. input_text and output_text are distinct model sequences; a colon is not an abstract separator unless it is literally present in input_text.

6.5 Current guarded decoder

output = model.generate(
    **inputs,
    forced_bos_token_id=tokenizer.convert_tokens_to_ids("gvn_Latn"),
    num_beams=1,
    max_new_tokens=208,
    no_repeat_ngram_size=4,
    repetition_penalty=1.10,
    length_penalty=1.0,
)

The no-repeat guard is an engineering safety intervention. Kuku Yalanji has meaningful reduplication, so fluent speaker review must check whether the tokenizer-level four-gram ban suppresses valid morphology.

7. Complete material experiment history

Scores across different rows, preprocessing, decoders, and objectives are not directly rankable. The table records what each run established within its own contract.

7.1 Initial engineering line

The early line proved the RunPod, LoRA, merge, evaluation, download, and CPU-serving pipeline. It also showed that larger base size and higher overlap scores did not cure clause compression or semantic drift.

7.2 Diagnostic and curriculum line, v8-v16

7.3 Narrative and full-corpus line, v17-v20

v20 falsified the hypothesis that simply adding all available material would produce a faithful translator. The small high-value sets were overwhelmed by heterogeneous, Bible-heavy supervision. Row count was not the right proxy for linguistic value.

7.4 v21.1 versus v21.2

v21.1 trained only the 16,642 leakage-audited synthetic rows. v21.2 used the same synthetic treatment plus Bible and dictionary-usage replay. Both started from the same v12 base and used the same seed and LoRA configuration, but v21.2 had 4,155 optimizer updates versus 3,120 for v21.1 and ran on different hardware. The result identifies the better practical recipe, not a perfectly isolated causal replay effect.

On tagged synthetic rows, the paired mean sentence chrF++ difference for v21.2 minus v21.1 was +0.082 with 95% interval [-0.225, +0.396]. The correct conclusion is synthetic parity with large replay-domain retention gains. Neither model solved elder/natural-register transfer.

7.5 v22 step-matched replay and decoder transfer

v22 reconstructed the v21.2 trajectory at step 2,770 and continued to exactly step 3,120 to test whether v21.2 had been over-trained. The checkpoint failed. Under greedy decoding it retained 37 tagged repeated-segment rows and was worse than step 4,155 on synthetic tests and dictionary usage.

The development-only decoder search did produce a useful secondary result. The chosen guarded policy was frozen before transfer to exact v21.2 weights. On unchanged v21.2 weights:

The current edition is therefore a serving-policy release over unchanged v21.2 weights, not a v22 model.

7.6 v23 attested-narrative adaptation

v23 was a preregistered low-rate second-stage LoRA over exact v21.2 weights. It used three seeds and no Bible rows:

156 reconciled Text 51/12 clauses x4     624
DB usage direct                            356
DB usage glossary                          356
Synthetic retention                      1,024
Total per epoch                           2,360
Epochs                                        6
Seeds                                  17,42,73

Seed 73 was selected on Text 36 development only. The held-out Text 3 result was:

v21.2 corpus chrF++                   31.6512
v23 seed 73 corpus chrF++             32.2926
corpus delta                           +0.6414
mean sentence delta                    +0.4494
paired 95% interval             [-0.2448,+1.1397]

The preregistered natural-test minimum was +1.0 corpus chrF++, the interval lower endpoint had to be positive, and untagged severe loops could not increase. All three failed: the point delta was +0.6414, the interval crossed zero, and untagged loop rows rose from 2 to 5.

The separate model lexical-reconstruction result was 46/297 exact, 15.49%, Wilson 95% 11.82%-20.04%. The research and lexical gates both failed. Candidate merged-weight identity before deletion was:

8ad3d986d8a3d869d4ab1013a7828e1086db1123dc8d80593cc1fd4ec30c7254

All three seeds' adapters, merged weights, checkpoints, optimizer states, and trainer caches were deleted after the compact evidence was verified: 17,360,903,656 bytes. The retained 181-file archive manifest hashes to:

63261bf0139a075ed5292525bda6f704b35680b902526895bbd8b2da9112912d

The observed A40 pod cost was approximately USD 0.9693 at the then-advertised USD 0.44/hour rate. The pod was deleted and the post-deletion active list was empty.

8. Evaluation framework

8.1 Frozen automatic endpoints

The current battery separates:

• synthetic development and tagged/untagged synthetic test;
• isolated dictionary glosses with multiple accepted references;
• dictionary usage examples;
• elder-shared connected sentences;
• speaker-disjoint Patz narrative development/test where applicable;
• Bible catastrophic-retention controls;
• empty, source-copying, length, truncation, and repeated-segment failures.

8.2 Exact lexical scoring

One row is one normalized English gloss. Every canonical Kuku Yalanji headword grouped under that gloss is an accepted reference. The primary score uses NFC/casefold/spacing normalization and exact full-output equality. Approximate spelling, token containment, semantic plausibility, and chrF++ do not count as exact. They remain secondary diagnostics.

Executed scoring logic is conceptually:

accepted = {normalize(form) for form in row["accepted_references"]}
is_exact = normalize(prediction) in accepted

This prevents an arbitrary synonym from being treated as the only reference while maintaining a strict deployment criterion.

8.3 Confidence-adjusted lexical-reconstruction rule

The gate computes a two-sided Wilson interval:

def wilson_interval(successes, rows, z=1.959963984540054):
    rate = successes / rows
    denominator = 1 + z * z / rows
    center = rate + z * z / (2 * rows)
    margin = z * ((rate * (1 - rate) / rows + z * z / (4 * rows * rows)) ** 0.5)
    return ((center - margin) / denominator, (center + margin) / denominator)

The model lexical-reconstruction gate passes only when all are true:

rows >= 297
normalized accepted-reference exact rate >= 0.80
Wilson 95% lower endpoint >= 0.80
empty outputs == 0

For 297 rows the first passing count is 252, not 238. This is a deliberately conservative operational rule on a mechanically constructed frozen benchmark. The prompts are not a random sample of future requests, so the interval must not be presented as a population estimate or as proof of general translation reliability.

8.4 Statistical limits

• Paired row bootstrap intervals describe the frozen rows, not new speakers or communities.
• Sentence rows inside one narrative are dependent through discourse, speaker, topic, and transcription.
• Multiple exploratory slices increase false-positive risk; they are diagnostic unless multiplicity is controlled.
• Corpus chrF++ is nonlinear. Corpus deltas and mean sentence-score deltas answer different questions.
• BLEU and chrF++ reward reference overlap and can both reward wrong propositions or penalize valid variants.
• Exact match is appropriate for a stored canonical headword objective but too narrow for open sentence adequacy.
• Three training seeds estimate some optimization variability, not speaker, dialect, or corpus variability.

8.5 Portable RunPod benchmark

The sealed kit contains code, protocols, the 340 frozen rows, local prediction baselines, runtime locks, and hashes. Model weights are separate pinned payloads. On an NVIDIA A40, two deterministic executions matched each other and local CPU output on all 1,020 predictions across three checkpoints.

cd /workspace/kuku-eval-kit
./bootstrap_and_run.sh
test -f RUN_COMPLETE
sha256sum -c OUTPUT_SHA256SUMS

All three checkpoint passes took about 3.5 minutes together. A candidate already present on a training pod should return a compact evaluation in roughly one to two minutes.

9. Forensic diagnosis of the current model

9.1 Isolated lexical supervision was underweighted

v21.2 saw 16,642 synthetic sentences and 4,094 Bible replay rows per epoch, but only 357 unique dedicated dictionary-usage examples. It did not receive an explicit isolated pair for every eligible canonical headword. A word occurring inside a sentence is not the same objective as mapping an isolated English gloss to an exact lemma.

9.2 The existing dictionary probe is exposure-sensitive

The 46/192 versus 2/105 seen/unseen split indicates that training exposure is strongly associated with exact retrieval. It does not prove causality by itself, because frequency and lexical difficulty are confounded, but it is strong enough to justify a controlled lexical-replay experiment.

9.3 Synthetic evaluation shares the authoring process

The synthetic test has strong source-order and template difficulty structure. Later rows are shorter and more regular; early rows include longer and more varied clauses. A corpus chrF++ around 55 on this distribution is useful for regression control, but it cannot be read as 55 chrF++ on spontaneous or connected natural language.

9.4 Natural material is too small and too clustered

v23 added only 156 unique native-speaker narrative clauses from one training speaker, selected on one development speaker, and tested on one speaker/text. The design was cleaner than prior work but underpowered at the cluster level. More epochs or another seed cannot manufacture speaker diversity.

9.5 Historical Bible dominance distorted objectives

Bible data taught useful target-language surface patterns but dominated early mixtures and rewarded a free religious translation style. v20 showed that even oversampling small sets did not guarantee preservation in a large heterogeneous mixture. v24 should not use Bible rows as a positive training objective.

9.6 Repetition is partly a decoding problem and partly a model problem

The guarded decoder greatly reduced loops on exact v21.2 weights. v23 still increased untagged loop rows under that guard, showing that training can alter degeneration risk. Decoder guards must be frozen on development data and cannot replace model-level retention checks.

9.7 Exact lookup and generation were previously conflated

v9.9 and v16 achieved exact answers on known stored resources through lookup. Asking a model to regenerate the same fact adds error. The product should route known words and known sentence resources deterministically, then use a model only for novel composition. Component scores and whole-system scores must both be reported so lookup success is never attributed to model weights.

10. Current served translation architecture

The homepage currently loads the complete selected dictionary, constructs a glossary, and calls gpt-5.4-mini. Translation mode requires a structured tool call validated by Zod. The system prompt treats user text as untrusted data rather than instructions. The response includes both a Kuku Yalanji draft and a literal English gloss. This is the temporary live path while custom models remain unmounted.

The intended long-term router is:

request
  -> exact rights-cleared sentence lookup
  -> exact/unique dictionary lookup for lexical queries
  -> custom model only if that exact version passed its model gate
  -> optional dictionary-grounded hosted LLM fallback
  -> generation guards and research-draft label
  -> bilingual event log with model/version/provenance

The whole route should expose which branch answered. An exact dictionary lookup may be called dictionary_exact; it must not be logged as model_exact.

11. Why dictionary pairs belong in training

The user-proposed intuition is correct: if a production requirement asks the model to translate isolated known words, the training objective should include isolated words. A sentence-only model has no reason to learn that its entire output for woman must be exactly jalbu.

Explicit pairs provide:

• direct loss on canonical headword retrieval;
• repeated exposure to low-frequency lexical items;
• a production-shaped one-word objective;
• a measurable lexical learning curve;
• a controlled way to test catastrophic forgetting against sentence tasks.

They do not automatically provide:

• sense disambiguation in context;
• inflected forms such as case-marked nominals;
• transitivity and argument structure;
• verb conjugation or tense/aspect/mood;
• derivational morphology;
• discourse anaphora or participant tracking;
• idiomatic collocations;
• dialect and register selection;
• semantic composition for unseen sentences.

The correct conclusion is therefore include the dictionary, but give it its own task design and its own claim.

12. Proposed v24 hypotheses

12.1 Primary lexical hypothesis

Adding source-audited isolated dictionary mappings under a visible <lexeme> task will materially increase closed-set accepted-reference exact reconstruction over a token- and step-accounted continuation control without increasing blank, degeneration, or sentence-retention failures.

12.2 Exposure-response hypothesis

At a fixed token and optimizer-update budget, lexical exact reconstruction will show a positive but saturating response across low, medium, and high lexical exposure. The best recipe will be selected across seeds on lexical development plus sentence-retention gates, not on the final lexical test.

12.3 Acquisition and uptake hypotheses

A model may reconstruct explicitly trained mappings while remaining weak at learning isolated lemmas from sentence context or applying a newly supplied glossary mapping in a novel sentence. v24 measures those three behaviors separately. Morphological generalization is measured only where the lemma is observed and a source-governed inflected or derived form can be held out.

12.4 Natural-language noninferiority hypothesis

Lexical adaptation must not materially degrade held-out natural narrative, elder regression, dictionary usage, or synthetic retention. No claim of improved natural translation will be made unless a multi-text, multi-speaker natural endpoint improves under cluster-aware analysis and speaker review.

13. Proposed v24 data construction

13.1 Freeze source identities first

Before generating a row, freeze and hash:

• dictionary YAML, JSON-LD, and public JSONL;
• source schema and renderer/export code;
• v21.2 model and tokenizer files;
• synthetic retention split;
• database usage split;
• narrative training, development, and test sources;
• current 297-row probe;
• builder, evaluator, trainer, environment lock, and RunPod image digest.

The builder must refuse a nonempty output directory and any input hash drift.

13.2 Emit an eligibility ledger for all 2,688 entries

Every source entry receives one decision record, even when no training pair is emitted:

{
  "entry_id": "dictionary-123",
  "headword": "jalbu",
  "decision": "include_canonical_lexeme",
  "reasons": [],
  "source_sha256": "...",
  "review_state": "project_reviewed_pending_speaker_verification"
}

Exclusions and special handling must be additive and inspectable, for example:

{
  "entry_id": "dictionary-456",
  "decision": "defer_descriptive_definition",
  "reasons": ["english_definition_not_substitutable_translation"]
}

This is a classification output, not a hand-maintained list of particular words. Mechanical schema rules should handle obvious structure; ambiguous semantic cases require recorded linguistic adjudication.

13.3 Task-visible row classes

1. Direct lexical-reconstruction rows

   <lexeme> woman -> jalbu

   These train the closed-set reconstruction objective. They do not share the sentence prefix.

2. Sense-qualified rows

   <lexeme> bank <sense> river edge -> [canonical headword]

   Use only where the source supplies a defensible sense distinction. Do not invent English sense labels from a model without review.

3. Part-of-speech qualified rows

   <lexeme> light <pos> noun -> ...
   <lexeme> light <pos> adjective -> ...

   POS qualification can reduce conflicting targets, but the unqualified production row still needs a documented preferred answer or deterministic lookup policy.

4. Ordinary sentence rows

   <translate> The woman returned. -> [Kuku Yalanji sentence]

   These remain the sentence task and provide the retention endpoint.

5. Glossary-conditioned sentence rows

   <translate> The woman returned. <glossary> woman = jalbu -> [Kuku Yalanji sentence]

   These test whether supplied lexical evidence is used in context. A final uptake mapping is absent from all training and supplied only at evaluation.

6. Attested usage rows

   Preserve existing sentence pairs with their word ID and source. They teach context, not just lemma identity.

7. Attested inflection/derivation rows

   Extract only from reconciled examples and narratives. Record lemma, surface form, morphological analysis, and evidence span. Do not generate a paradigm mechanically and call it attested.

8. Governed synthetic morphology rows

   These may be generated from verified grammar rules when clearly labelled synthetic, checked against lexical and morphological constraints, reviewed, and held separate from attested material.

13.4 Multi-reference policy

For evaluation, every canonical headword attached to one normalized English gloss may be accepted. For training, blindly duplicating one unqualified source with many unrelated target senses creates a conflicting conditional distribution. Use the following order:

1. retain a documented preferred canonical form for the unqualified production query;
2. emit sense/POS-qualified rows for distinguishable alternatives;
3. retain true synonyms as multi-reference metadata;
4. if synonymous targets are deliberately sampled, keep equal, recorded weights and evaluate all as accepted;
5. never discard alternatives from the source dictionary merely to make exact scoring easier.

13.5 Task-specific split and exposure design

One seen/unseen flag is inadequate. Every lexical endpoint records four exposure states:

direct isolated-pair exposure in v24
sentence-context exposure in v24
documented exposure in earlier MobTranslate model lineage
upstream NLLB exposure unknown

Stage A: research fit

• For direct reconstruction, evaluate governed closed-set development mappings that received explicit lexical supervision; this is a retention/memorization endpoint and is labelled as such.
• For contextual acquisition, withhold the isolated pair while allowing governed sentence-context exposure.
• For lexicon uptake, withhold the mapping from all documented project-lineage training and supply it only in the inference glossary. Upstream NLLB exposure remains unknown.
• For morphology, expose the lemma but hold out the governed surface form and context family.
• Cluster by normalized Kuku headword family, English gloss/sense, source entry, and variant relationship so related rows cannot straddle development and final test accidentally.
• Use development to select the exposure recipe and open each final task only after the recipe is locked.

Stage B: all-lexicon product fit

• Refit the selected recipe with every eligible dictionary cluster.
• Evaluate the 297 production probe as closed-set model reconstruction and enforce only the lexical-model gate.
• Do not call this an unseen lexical test.
• Keep deterministic database lookup ahead of the model even if this gate passes.

13.6 Natural-language splits

The next natural claim requires more reconciled source material before training. Development and test should each contain multiple speaker/text clusters. If that data is not yet available, v24 may proceed as a lexical experiment, but natural sentence performance must be a noninferiority claim only.

At minimum:

• no speaker in more than one split;
• no story/text in more than one split;
• variety labels balanced where possible rather than perfectly confounded with split;
• exact and high-similarity source/target leakage checks;
• discourse segments kept intact rather than splitting neighboring clauses across sets;
• cluster-level intervals or randomization analyses in addition to row diagnostics;
• blind fluent-speaker review before any natural-translation promotion.

14. Proposed v24 experiment design

14.1 Base model

Primary controlled experiment: continue from the exact v21.2 merged weights. This isolates the effect of lexical supervision and reuses a known stable baseline. A stock-base or alternate architecture may be added as a separate arm, but it must not replace the controlled continuation in the same comparison.

14.2 Arms

The primary causal question is substitution at fixed compute: does replacing scheduled retention tokens with lexical-task tokens improve the intended endpoint without unacceptable retention loss? Use fixed tokens and fixed optimizer updates, not merely fixed epochs:

C0 and each treatment arm must use the same exact optimizer horizon and planned non-padding token budget. The manifest reports actual examples and source/target tokens consumed by task. One additive-compute treatment may be run later, but it answers a different question and is not part of the primary screen.

14.3 Efficient two-stage search

To control RunPod cost without weakening the confirmatory result:

1. Run a preregistered one-seed screening study for C0/L1/L2/L4/G using development endpoints only; B0 is evaluated without training.
2. Eliminate arms only by frozen rules, not informal sample reading.
3. Carry C0 and the top one or two treatment recipes into paired seeds 17, 42, and 73.
4. Select the recipe from aggregate development behavior across seeds; do not tune the seed.
5. Open final acquisition, uptake, morphology, and natural-retention tests for every confirmatory seed once.
6. If Stage A passes, perform one all-lexicon product refit with the locked recipe.

Screening results are exploratory; the three-seed confirmation supplies the reported effect. Five seeds are added only if the screen shows material variance that three seeds cannot characterize.

14.4 Initial retention mixture

Exact counts depend on the eligibility audit and must be frozen before the run. The recommended categories are:

lexical rows                  none / low / medium / high by C0/L1/L2/L4
glossary-conditioned rows    G arm only
database usage rows          retained, source-disjoint from tests
synthetic retention          fixed stratified sample, not all 16,642 by default
reconciled narrative train   only attested, rights-cleared clauses
Bible rows                   0 in training
elder diagnostic rows        0 in training
contextual-acquisition test  no isolated-pair exposure; governed sentence context permitted
lexicon-uptake test          mapping absent from documented project lineage; supplied only at inference
morphology test              lemma observed; governed surface form/context family held out
held-out speakers/texts      0 in training

The synthetic retention sample should be stratified by grammar, frame, domain, source-order quartile, sentence length, and quality tier. Do not use an ad hoc handful of easy templates.

14.5 Hyperparameter starting point

Use the conservative v23 continuation settings as a starting proposal, not as a frozen decision:

LoRA rank / alpha / dropout   32 / 64 / 0.05
Target modules                q_proj,k_proj,v_proj,out_proj,fc1,fc2
Learning rate                 1e-5
Batch / accumulation          8 / 2
Effective batch               16
Warmup / weight decay         0.05 / 0.01
Decoder                       frozen v21.2 guarded policy
Seeds                         17, 42, 73

The final preregistration must set an exact positive max_steps; epoch count is descriptive only. Checkpoint cadence should align to complete passes through the smallest high-value component, and early stopping must be based on the frozen development composite, not training loss. v23 showed validation loss continuing to fall after natural chrF++ peaked.

14.6 Development selection function

Do not collapse every objective into a weighted average that permits one domain to buy off a failure in another. Use lexicographic/all-checks logic:

1. zero blanks and hard degeneration ceiling;
2. natural and synthetic noninferiority floors;
3. contextual-acquisition or glossary-uptake development improvement, according to the arm;
4. closed-set lexical-reconstruction development improvement;
5. usage-example retention;
6. among eligible checkpoints, choose the lowest update count or highest primary lexical result as preregistered.

Bible is not a positive selector. It is only a catastrophic floor.

15. Proposed v24 evaluation and promotion gates

15.1 Research-task gates

• Direct reconstruction: ambiguity-aware normalized accepted-reference exact after explicit pair exposure.
• Contextual acquisition: isolated reconstruction after sentence-context exposure but no direct pair.
• Lexicon uptake: supplied held-out mapping used correctly in a novel sentence, including governed morphology where the source supports that analysis.
• Morphology: required feature and surface-form accuracy over lemma-observed, form-held-out records.
• Secondary diagnostics: grapheme-cluster and code-point CER, multi-reference chrF++, copying, tokenization class, orthographic validity, and error class.
• Report all documented-lineage exposure strata and mark upstream exposure unknown.
• Compare systems with paired, cluster-aware analysis at the lexeme/sense/source-entry level.

No numeric minimum should be invented after looking at results. The preregistration should set a meaningful effect floor after a baseline power/sensitivity analysis on the frozen row count.

15.2 Model lexical-reconstruction gate

The existing rule remains:

at least 297 rows
at least 252 accepted exact at 297 rows
point exact >= 80%
Wilson 95% lower bound >= 80%
zero blank outputs
exact model ID and exact merged-weight hash bound into the gate

This gate is a conservative rule on the frozen benchmark, not an estimate of future-query reliability. It authorizes only a model lexical-reconstruction route. Add these operational requirements:

• no source copying;
• no output outside accepted orthographic shape for isolated words;
• no generation-limit termination;
• service smoke and concurrency test;
• model-only score and whole-router score reported separately.

15.3 Model sentence-generation gate

• Synthetic tagged and untagged retention within a frozen noninferiority margin.
• Dictionary-usage retention within a frozen margin.
• Elder regression reported with no positive promotion weight unless speaker review supports it.
• Multi-speaker natural endpoint required for any improvement claim.
• Zero blanks.
• Repeated-segment rows no worse than v21.2 guarded.
• Severe under- and over-translation no worse than baseline.

A lexical PASS cannot satisfy this gate. Public sentence generation additionally requires multiple independent speaker/text clusters, fluent-speaker semantic and grammatical review, no critical participant/negation/safety errors, and documented release authority for that scope. Every current Kuku model is FAIL on this gate.

15.4 Dictionary-router gate

The deterministic route is tested separately for exact retrieval, sense/POS handling, ambiguity display, source and variety labels, route metadata, latency, and absence of accidental model attribution. It is not authorized by either model gate and does not require a neural model to be mounted.

15.5 Human linguistic review

Blind reviewers should not see model names. At minimum, annotate:

Corrections should become versioned evidence with reviewer role, confidence, reason, and rights, not silent overwrites.

16. RunPod execution plan

16.1 Preflight on the local box

PROGRAM=/mnt/donto-data/donto-resources/research/translation-training/kuku-yalanji-runpod-2026-06-30
REPO=/mnt/donto-data/workspace/mobtranslate.com

# Build into a new empty directory; never overwrite a prior edition.
python3 "$REPO/training/translation/build_v24_lexicon_grounded.py" \
  --output-dir "$PROGRAM/prepared/v24.0-lexicon-grounded"

cd "$PROGRAM/prepared/v24.0-lexicon-grounded"
sha256sum -c SHA256SUMS.v24-inputs

build_v24_lexicon_grounded.py is proposed work and does not exist at the publication time of this report. The command defines the intended contract, not a claim that v24 data has already been built.

Preflight must also run unit tests, validate row and sense counts, verify disjoint groups, scan tokenizer fertility and lengths without truncation, audit exposure against the full documented model lineage, verify the exact v21.2 base and custom gvn_Latn embedding/output path, and produce the preregistration before a pod starts. Data whose research use or third-party transfer is unresolved is excluded or quarantined; no approval state is invented.

16.2 Proposed exact trainer invocation

The executed trainer already supports the necessary controls. A candidate invocation will look like:

python training/translation/train_nllb_lora.py \
  --base-model /workspace/v24/base/merged \
  --train-file /workspace/v24/data/train.eng-gvn.jsonl \
  --validation-file /workspace/v24/data/validation.eng-gvn.jsonl \
  --output-dir /workspace/v24/models/seed-42 \
  --model-id mobtranslate/kuku-yalanji-nllb-lora \
  --model-version v24.0-lexicon-grounded-seed42 \
  --direction eng-gvn \
  --source-lang eng_Latn \
  --target-lang gvn_Latn \
  --max-source-length 192 \
  --max-target-length 208 \
  --learning-rate 1e-5 \
  --max-steps "$FROZEN_MAX_STEPS" \
  --batch-size 8 \
  --gradient-accumulation-steps 2 \
  --warmup-ratio 0.05 \
  --weight-decay 0.01 \
  --lora-r 32 \
  --lora-alpha 64 \
  --lora-dropout 0.05 \
  --lora-target-modules q_proj,k_proj,v_proj,out_proj,fc1,fc2 \
  --generation-num-beams 1 \
  --generation-no-repeat-ngram-size 4 \
  --generation-repetition-penalty 1.10 \
  --generation-length-penalty 1.0 \
  --seed 42 \
  --full-determinism

The exact update horizon, token budget, task schedule, and save cadence must come from the frozen v24 preregistration. The trainer fails if its observed optimizer step differs from --max-steps and records actual examples and post-truncation source/target tokens consumed by task. This snippet is a reviewable starting contract, not permission to launch with unfrozen counts.

16.3 Live verification while running

A started process is not a working process. For every seed and arm, monitor:

nvidia-smi
ps -eo pid,etime,rss,pcpu,args | rg 'train_nllb_lora.py'
find /workspace/v24/models -name trainer_state.json -print
tail -n 50 /workspace/v24/logs/*.log
df -h /workspace

Verify optimizer steps advance over wall-clock time, GPU activity is real, output files grow, RSS remains bounded, and no empty provider/service dependency is silently treated as success.

16.4 Evaluate before downloading weights

cd /workspace/kuku-eval-kit
./bootstrap_and_run.sh \
  /workspace/v24/models/selected/merged

test -f RUN_COMPLETE
sha256sum -c OUTPUT_SHA256SUMS

The current kit accepts manifest-registered checkpoints; a v24 wrapper must add the candidate and its hashes without mutating v1 evidence. Download compact predictions, metrics, manifests, logs, and gates first. Pull multi-gigabyte weights only after promotion remains possible.

16.5 Cost containment and cleanup

• Register automatic stop and termination timers before training.
• Keep a local operations ledger with UTC timestamps.
• Download compact evidence incrementally after each seed.
• On failed screening, delete all adapters, merged weights, optimizer checkpoints, and caches after compact evidence is verified.
• For confirmatory runs, retain compact adapters when source rights permit independent reproduction; delete merged models, optimizer states, redundant checkpoints, and caches after the decision.
• On pass, retain only the selected final adapter/merged model plus compact evidence; delete unselected bulky weights.
• Verify deletion and the active pod list.
• Record actual billed cost rather than extrapolating from a stale price.

17. Proposed v24 lexical builder excerpt

This is proposed code for review. It illustrates structured handling, not a finished implementation:

from collections import defaultdict
from dataclasses import dataclass
import hashlib
import json
import unicodedata

@dataclass(frozen=True)
class LexicalPair:
    entry_id: str
    english: str
    kuku: str
    part_of_speech: str | None
    sense: str | None
    source: str

def norm(text: str) -> str:
    return " ".join(unicodedata.normalize("NFC", text).casefold().split())

def stable_id(pair: LexicalPair, task: str) -> str:
    payload = "\0".join((pair.entry_id, pair.english, pair.kuku, task))
    return hashlib.sha256(payload.encode("utf-8")).hexdigest()[:24]

def lexical_reconstruction_row(pair: LexicalPair) -> dict:
    english = " ".join(pair.english.split())
    kuku = " ".join(pair.kuku.split())
    return {
        "id": f"lexeme:{stable_id(pair, 'reconstruction')}",
        "direction": "eng-gvn",
        "input_text": f"<lexeme> {english}",
        "unconditioned_input_text": english,
        "output_text": kuku,
        "pair_kind": "dictionary_lexeme",
        "source_lang": "eng_Latn",
        "target_lang": "gvn_Latn",
        "lexicon": {
            "entry_id": pair.entry_id,
            "part_of_speech": pair.part_of_speech,
            "sense": pair.sense,
            "source": pair.source,
        },
    }

def group_references(pairs: list[LexicalPair]) -> dict[str, list[str]]:
    grouped: dict[str, list[str]] = defaultdict(list)
    for pair in pairs:
        key = norm(pair.english)
        if pair.kuku not in grouped[key]:
            grouped[key].append(pair.kuku)
    return dict(grouped)

def write_jsonl(path, rows):
    with path.open("w", encoding="utf-8") as handle:
        for row in rows:
            handle.write(json.dumps(row, ensure_ascii=False, sort_keys=True) + "\n")

The finished builder must add source hash validation, eligibility/exclusion ledgers, sense qualification, separate contextual-acquisition and glossary-uptake records, group splitting, full-lineage exposure audits, deterministic sorting and schedules, quarantine, rights checks, token budgets, tests, manifests, and fail-closed assertions.

18. Split deployment interlocks

The lexical gate generator is invoked after a candidate's closed-set reconstruction audit:

python training/translation/verify_dictionary_cpu_deployment.py \
  --lexicon-audit analysis/version_probe_analysis.json \
  --model-label candidate \
  --model-id v24.0-lexicon-grounded-product \
  --model-sha256 "$MERGED_MODEL_SHA256" \
  --minimum-rows 297 \
  --minimum-exact-rate 0.80 \
  --minimum-wilson-lower 0.80 \
  --output analysis/cpu-deployment-gate.json

Systemd checks the gate before loading any model:

python3 training/translation/check_dictionary_cpu_deployment_gate.py \
  --gate /etc/mobtranslate/kuku-cpu-deployment-gate.json \
  --model-id v24.0-lexicon-grounded-product \
  --model-file /path/to/merged/model.safetensors

The lexical checker refuses startup unless the gate type is model_lexical_reconstruction, its decision is the matching allow value, every required check is true, the model ID matches, the model file exists, and its full SHA-256 matches. A passing report for one model cannot authorize another model.

Sentence services have a second independent preflight:

python3 training/translation/check_model_bound_release_gate.py \
  --gate /etc/mobtranslate/kuku-v24-sentence-generation-gate.json \
  --gate-type model_sentence_generation \
  --allowed-decision SENTENCE_GENERATION_ALLOWED \
  --model-id v24.0-selected \
  --model-file /path/to/merged/model.safetensors

Both existing Kuku sentence-service units carry this second interlock with a model-bound FAIL gate. Therefore a future 297-word lexical PASS cannot mount a free-form sentence service. The deterministic dictionary router has its own route tests and does not depend on either model service.

19. Model hosting after a pass

Only a candidate whose applicable governance, rights, and release scope are documented can become a public model release. That release should include:

• exact frozen base plus adapter bundle for dynamic LoRA hosts such as Atlas;
• standalone merged Transformers model;
• tokenizer and custom gvn_Latn metadata;
• guarded decoding policy;
• immutable version and all SHA-256 values;
• model card, training report, data manifest, rights, and limitations;
• model-only and whole-router benchmark results;
• smoke inputs and expected outputs;
• OpenAPI links and a minimal HTTP request/response contract.

Dynamic LoRA loading remains:

from peft import PeftModel
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained(
    "./adapter", src_lang="eng_Latn", tgt_lang="gvn_Latn"
)
base = AutoModelForSeq2SeqLM.from_pretrained("./base")
model = PeftModel.from_pretrained(base, "./adapter")

The host must use the distributed exact base, not stock NLLB. Every response must identify immutable model version, decode policy, and route. Public output remains a research draft unless the applicable rights and release authority are documented for that scope.

20. Artifact and retention policy

The program keeps scientific evidence while avoiding a warehouse of weak model weights:

No archive is overwritten. A new model receives a new immutable edition. Negative results remain registered as reports without hostable model downloads. Deletion is recorded by file count, byte count, reason, and post-delete verification.

21. Risks and mitigations

22. Decision tree for the next run

Is the 2,688-entry sense/eligibility ledger complete, with rights and uncertain records quarantined?
  no  -> do not train; finish data decisions
  yes -> freeze task-specific clusters, full-lineage exposure, token schedules, source hashes, and preregistration

Do all split/leakage/token-length/unit tests pass?
  no  -> quarantine/fix additively; do not launch
  yes -> run fixed-token and fixed-step screening arms on RunPod

Do any lexical arms pass development safety/noninferiority?
  no  -> retain compact negative result; delete weights
  yes -> run control + finalists over seeds 17/42/73

Does a treatment improve its declared reconstruction/acquisition/uptake endpoint without sentence regression?
  no  -> no model edition; preserve evidence
  yes -> lock recipe and run all-lexicon product fit

Does product fit achieve >=252/297 exact, Wilson lower >=80%, zero blanks, and lexical safety gates?
  no  -> do not mount lexical model; dictionary lookup remains first route
  yes -> generate model-bound lexical gate; this still does not authorize sentence generation

Do multi-cluster natural evaluation, fluent-speaker review, critical-error rules, rights, and release authority pass?
  no  -> keep sentence-generation gate FAIL and service unloaded
  yes -> issue a separate model-bound sentence gate and consider a research-draft sentence route

23. Immediate implementation order

1. Preserve the independent-review disposition and revise this dossier. Done.
2. Correct public ownership/licensing claims, registry status semantics, and add the v23 negative result. Done in source; static docs and the dynamic registry API are live, while the compiled Next.js copy awaits I/O quiescence.
3. Add exact max_steps, actual task/token exposure accounting, and separate lexical/sentence interlocks. Done and locally tested; a real trainer smoke test in the RunPod image remains.
4. Build build_v24_lexicon_grounded.py with tests and a 2,688-entry sense/decision ledger.
5. Produce counts for canonical, ambiguous, sense-qualified, variant, deferred, restricted, and excluded records.
6. Freeze reconstruction, contextual-acquisition, glossary-uptake, morphology, and retention groups; audit exposure through every documented model ancestor.
7. Audit tokenizer fertility, the custom language-token embedding/output rows, and base+adapter versus merged parity.
8. Write the v24 preregistration with exact row/token counts, max_steps, arms, seeds, gates, and retention rules.
9. Extend the sealed RunPod kit to accept v24 manifest entries without changing v1 evidence.
10. Run one-seed fixed-token screening, then paired three-seed confirmation only for selected recipes.
11. Pull compact evidence first, apply frozen task gates, and refit all eligible lexical rows only after recipe lock.
12. Package weights only for a passed, rights-permitted scope; rerender living books and update public checksums.

24. Reviewer reproducibility map

Canonical mounted-drive root:

/mnt/donto-data/donto-resources/research/translation-training/
kuku-yalanji-runpod-2026-06-30/

Important internal records:

docs/KUKU-YALANJI-V21-2-MODEL-AND-HOSTING-GUIDE.md
docs/KUKU-YALANJI-V22-STEP-MATCHED-REPLAY-AND-DECODER-TRANSFER-2026-07-14.md
docs/KUKU-YALANJI-V23-ATTESTED-NARRATIVE-PREREGISTRATION-2026-07-14.md
docs/KUKU-YALANJI-V23-ATTESTED-NARRATIVE-ADAPTATION-2026-07-14.md
docs/KUKU-YALANJI-RUNPOD-EVALUATION-LOOP-2026-07-14.md
reports/KUKU-YALANJI-V21-INDEPENDENT-MODEL-COMPARISON-2026-07-12.md
reports/kuku-yalanji-v20-third-opinion-technical-report-2026-07-02.md
reports/kuku-yalanji-training-resource-catalogue-2026-07-02.md
prepared/v21.2-claude-balanced-replay/
runpod/v21.2-claude-balanced-replay-20260711T050900Z/
runpod/v22.0-step-matched-replay-20260714T023516Z/
runpod/v23.0-attested-narrative-20260714T143510Z/

Reusable code:

/mnt/donto-data/workspace/mobtranslate.com/training/translation/
  train_nllb_lora.py
  evaluate_nllb_lora.py
  build_v21_2_claude_balanced_replay.py
  build_v23_attested_adaptation.py
  build_kuku_lexicon_probe.py
  analyze_kuku_version_probe.py
  compare_v21_models.py
  verify_dictionary_cpu_deployment.py
  check_dictionary_cpu_deployment_gate.py
  apply_v23_retention.py

Public reports:

• v21.2 model, training, data, and hosting guide
• v21 independent comparison
• v22 step-matched replay and decoder transfer
• v23 preregistration
• v23 negative result
• RunPod evaluation loop
• Operator's guide
• Grammar cheatsheet
• Dictionary errata and phantom watchlist
• Model API
• Human model browser

25. Final recommendation

Do not run another broad, heterogeneous continuation and hope that aggregate scale solves the problem. The next experiment should be narrow enough to answer separable questions cleanly: does explicit, source-audited lexical supervision improve closed-set reconstruction, and can the model acquire a lemma from context or use newly supplied glossary evidence without damaging sentence behavior?

Build visible task objectives, match update and token exposure, use paired seeds, audit the full model lineage, then refit all eligible entries for the closed-set model gate. Keep Bible out of the positive objective. Treat natural sentence improvement as a separate claim that requires more multi-speaker, multi-text attested data and blinded fluent-speaker review.

Until the independent gates pass, keep exact dictionary and sentence resources lookup-first, keep custom Kuku model CPU services unloaded, and label hosted-LLM output as an unverified draft. A lexical PASS may justify a research reconstruction route but never a sentence route. Public release additionally remains bounded by source rights and documented authority rather than a governance state invented by the technical team.

Document status: public post-review revision with an explicit disposition record. Corrections are additive and versioned. The raw Markdown at https://mobtranslate.com/trainignplan.md is the requested shareable edition; the spelling trainignplan is preserved intentionally to match the requested URL.