# Kuku Yalanji Synthetic-First Training Plan

Date: 2026-07-02 UTC

Prepared for: MobTranslate Kuku Yalanji model-training third-opinion review

Canonical local root:

```text
/mnt/donto-data/donto-resources/research/translation-training/kuku-yalanji-runpod-2026-06-30/
```

Repository root:

```text
/mnt/donto-data/workspace/mobtranslate.com
```

Public artifact root:

```text
https://mobtranslate.com/model-artifacts/kuku-yalanji-nllb-lora/
```

Related public artifacts:

```text
Grammar cheat sheet:
https://mobtranslate.com/model-artifacts/kuku-yalanji-nllb-lora/kuku-yalanji-working-grammar-cheatsheet-2026-07-02.md

v21s0 synthetic pilot:
https://mobtranslate.com/model-artifacts/kuku-yalanji-nllb-lora/v21s0-grammar-grounded-synthetic-candidates/RUN_REPORT.md

v21s0 synthetic candidate JSONL:
https://mobtranslate.com/model-artifacts/kuku-yalanji-nllb-lora/v21s0-grammar-grounded-synthetic-candidates/v21s0_synthetic_candidates.eng-gvn.jsonl

v21s0 review summary:
https://mobtranslate.com/model-artifacts/kuku-yalanji-nllb-lora/v21s0-grammar-grounded-synthetic-candidates/review_summary.json
```

## 0. Current Operating Premise

The project owner clarified on 2026-07-02 that the available resources and generated
synthetic rows are approved for project training and research use.

The correct status vocabulary is therefore:

```text
approved_for_training = true
rights_status = project_approved_synthetic_pending_elder_verification
```

This does not mean every generated row is linguistically perfect. It means the project has
approval to use the materials and generated rows for training/research now. Elder review is
the next quality layer, not a blocker.

The practical target is:

```text
Build a faithful English -> Kuku Yalanji translation system for MobTranslate, using all
available approved resources, with synthetic grammar/dictionary-driven data as the main
scalable source of everyday sentence pairs.
```

The model should eventually support:

```text
1. Everyday translation drafts.
2. Dictionary/example-style sentence translation.
3. Bible text lookup or Bible-domain generation where useful.
4. Public versioned model/evaluation pages so the evolution is visible.
```

The immediate target is not a perfect public release model. The immediate target is a
diagnostic synthetic-data curriculum that proves whether grammar-grounded synthetic rows
improve everyday translation and reduce Bible-style bleed in short ordinary sentences.

## 1. Plain-English Summary

The core problem is no longer "can we find 20,000 approved real everyday sentence pairs?"
We cannot assume that exists. The current non-Bible everyday supervision is tiny:

```text
DB usage examples:              449 rows
elder-shared sentence pairs:     43 rows
total everyday-ish real rows:   492 rows
```

The large resource is Bible-domain parallel text. It is useful for morphology, recurring
function words, and Kuku Yalanji surface statistics, but it is not the same product as
ordinary translation.

The scalable path is therefore:

```text
grammar PDF/MD + dictionary YAML/MD + grammar examples + Bible surface statistics
-> constrained synthetic sentence-pair generator
-> automatic review/linting
-> frozen route-specific eval suites
-> small diagnostic RunPod training runs
-> expand synthetic rows only when eval improves
```

This is deliberately not freeform "make up a language" generation. It is controlled
construction from known dictionary words, known suffixes, known sentence patterns, and
attested examples.

The first pilot already exists:

```text
v21s0 synthetic rows:             930
accepted for project training:    930
held for missing-base review:       0
dictionary entries loaded:       2287
reference examples loaded:      14154
```

The pilot is intentionally conservative. It uses simple nominal, intransitive, transitive,
locative-motion, wawu/want-like, yinil/afraid-like, and loan-control templates.

## 2. What We Are Trying To Build

MobTranslate needs a practical translator, not a language-theory paper. But because this is
a low-resource language-revival setting, the engineering must stay academically careful.

The desired system is probably not one monolithic neural model. The evidence so far points
toward a routed system:

```text
Known Bible reference?      -> lookup approved corpus text first
Known dictionary example?   -> lookup or retrieval-first response
Everyday English sentence?  -> route-specific translation adapter
Model comparison page?      -> show frozen eval outputs for each model version
```

For exact known text, retrieval is better than asking a neural model to memorize a database.
For new sentences, a model is needed. Synthetic data is the bridge between a dictionary and
an everyday translator.

The actual training objective should be explicit:

```text
Input:  <translate> The woman sees the child.
Output: Jalbungku karrkay nyajin.
```

And separate from:

```text
Input:  <bible_ref> ACT.17.14 <eng> ...
Output: approved Kuku Yalanji Bible corpus text
```

And separate from:

```text
Input:  <usage_example> ...
Output: dictionary/example style sentence
```

The v21 synthetic path is focused on the first objective: ordinary English -> Kuku Yalanji
draft translation using words we can ground in the dictionary and grammar.

## 3. Resources Available Now

All paths in this section are local to the MobTranslate repo or the mounted research drive.

### 3.1 Grammar

```text
dictionaries/kuku_yalanji/resources/grammar.pdf       22 MB
dictionaries/kuku_yalanji/grammar.md                  7,377 lines
experiments/pdftomd/grammar_markdown/*.md             27 page-chunk files
experiments/pdftomd/extract/output/examples.xigt.json 795 extracted grammar examples
experiments/pdftomd/extract/output/grammar_features.csv 862 lines
```

The working grammar cheat sheet distilled from those resources is public:

```text
https://mobtranslate.com/model-artifacts/kuku-yalanji-nllb-lora/kuku-yalanji-working-grammar-cheatsheet-2026-07-02.md
```

The cheat sheet is not a replacement for the source grammar. It is an operational guide for
generation and linting.

### 3.2 Dictionary

```text
dictionaries/kuku_yalanji/resources/dictionary.pdf  3.1 MB
dictionaries/kuku_yalanji/dictionary.md             21,364 lines
dictionaries/kuku_yalanji/dictionary.yaml           31,173 lines
dictionaries/kuku_yalanji/lexemes.json
dictionaries/kuku_yalanji/accepted_words.json
```

The v21s0 generator loaded 2,287 dictionary entries after filtering/normalization.

The dictionary is the main lexical constraint:

```text
English sentence concepts should mostly come from known dictionary words.
Kuku Yalanji target bases should be dictionary bases, grammar-example bases, Bible-attested
bases, or explicitly marked loan controls.
```

### 3.3 Bible Corpus

The v20 full-candidate corpus manifest reports:

```text
Bible train source rows:       16,623
Bible validation source rows:   2,153
Bible test source rows:         2,135

Train tiers:
  high_confidence_verse:       13,270
  sentence_candidate:           3,353

Validation tiers:
  high_confidence_verse:        1,670
  sentence_candidate:             483

Test tiers:
  high_confidence_verse:        1,657
  sentence_candidate:             478
```

The prepared v20 task rows are:

```text
train.eng-gvn.jsonl:      35,394 rows
validation.eng-gvn.jsonl:  4,433 rows
test.eng-gvn.jsonl:        4,397 rows
```

Why train rows exceed Bible source rows:

```text
Bible direct task rows:              16,623 train
Bible reference-conditioned rows:    16,623 train
DB usage oversampled rows:            1,460 train
elder-shared oversampled rows:          688 train
```

So yes, there are "20k-ish" Bible-derived source rows, but not 20k everyday translation rows.
The everyday signal remains hundreds of rows, not tens of thousands.

### 3.4 DB Usage And Elder-Shared Rows

```text
DB usage examples:             449 total
DB usage train split:          365
DB usage heldout split:         84
elder-shared sentence pairs:    43
```

These are high-value because they are closer to the actual product target than Bible rows.
But they are too few to train a general translator by themselves.

The 43 elder-shared rows are best used as:

```text
1. a washout canary,
2. a probe of vocabulary/register retention,
3. a seed for templates,
4. a future human-reviewed eval set,
```

not as the main optimization target. If the model merely memorizes 43 rows, that does not
prove it can translate new sentences.

## 4. What Previous Model Runs Have Shown

This section is short because the detailed v20 report already exists:

```text
https://mobtranslate.com/model-artifacts/kuku-yalanji-nllb-lora/kuku-yalanji-v20-third-opinion-technical-report-2026-07-02.md
```

The important conclusions are:

```text
v12 was the strongest Bible-focused model in the previous ladder.
v18 showed route-specific continuation could memorize/retain usage and elder-shared rows.
v19 balanced replay improved some mixed behavior but had suspiciously high validation scores.
v20 trained on the broadest corpus but did not become a faithful general translator.
```

v20 metrics:

```text
test_all:                  BLEU 11.92, chrF++ 39.39, rows 4,397
test_bible_direct:         BLEU 11.90, chrF++ 39.41, rows 2,135
test_bible_ref:            BLEU 11.93, chrF++ 39.39, rows 2,135
heldout_usage:             BLEU  7.84, chrF++ 43.26, rows 84
elder_shared_all:          BLEU 12.64, chrF++ 36.24, rows 43
train_sample:              BLEU 23.35, chrF++ 47.10, rows 2,456
```

The high-level v20 reading:

```text
The model learned Kuku-Yalanji-shaped text better than it learned faithful everyday
translation.
```

It fixed some under-generation, but introduced or preserved other problems:

```text
1. Bible high-confidence rows still scored low.
2. Everyday heldout rows remained weak.
3. Elder-shared train-row exact reproduction collapsed.
4. Some outputs repeated Bible-register patterns for short ordinary inputs.
```

## 5. Claude Feedback: What Held Up And What Changed

Claude's feedback was useful. Several recommendations are now part of the plan:

```text
1. Use a frozen eval suite.
2. Do leakage audits before trusting validation scores.
3. Treat elder exact match as a canary, not a product target.
4. Prefer route-specific adapters over a universal monolith.
5. Audit whether the custom gvn_Latn embedding row is actually trainable.
6. Do not compare model versions on different eval sets without rescoring.
```

One hypothesis did not hold after checking the v20 prediction file.

Claude suggested v20 might only look worse because the v20 Bible test set includes 478
sentence-candidate rows. I decomposed `eval_test_bible_direct_predictions_greedy.json` by
the stored `tier` field using the same SacreBLEU chrF++ setting (`word_order=2`) used by
the project evaluator.

Exact split:

```text
high_confidence_verse:
  rows:                1,657
  BLEU:                9.11
  chrF++:             37.73
  mean length ratio:   0.96
  median length ratio: 0.86

sentence_candidate:
  rows:                  478
  BLEU:                32.07
  chrF++:              55.79
  mean length ratio:    1.21
  median length ratio:  1.00

all bible_direct:
  rows:                2,135
  BLEU:                11.90
  chrF++:              39.41
```

That means the sentence-candidate rows are not dragging down `test_bible_direct` in this
file. The high-confidence rows are the weaker slice. So the v20 Bible problem is not merely
a test-composition artifact.

This changes the diagnosis:

```text
v20's Bible score is genuinely weak on high-confidence rows.
But that still does not prove "more Bible data" is the answer.
It supports separating Bible reproduction from everyday translation, and building a better
everyday corpus rather than relying on Bible rows to teach everyday register.
```

## 6. Important Technical Audit: gvn_Latn Embedding Trainability

The current code creates or registers a custom target language token:

```text
target_lang = gvn_Latn
target_lang_init_from = tpi_Latn in several gvn-token runs
```

The training script supports:

```text
--modules-to-save
```

and the older full gvn token scripts used:

```text
--modules-to-save model.shared,lm_head
```

But v12 and v20 route scripts defaulted `MODULES_TO_SAVE` to empty or did not pass it:

```text
training/translation/run_v12_tagged_multitask_gvn_token.sh:
  export MODULES_TO_SAVE="${MODULES_TO_SAVE:-}"

training/translation/run_v20_full_candidate_corpus.sh:
  LORA_TARGET_MODULES=q_proj,k_proj,v_proj,out_proj,fc1,fc2
  no modules_to_save argument in the train invocation
```

The train script then builds LoRA with:

```text
target_modules = q_proj,k_proj,v_proj,out_proj,fc1,fc2
modules_to_save = None
```

This strongly suggests the custom `gvn_Latn` embedding row may have stayed frozen after
being initialized from `tpi_Latn`, unless PEFT or embedding resizing made it trainable in a
way not obvious from the script.

This must be audited before the next serious GPU run:

```text
1. Load base model/tokenizer.
2. Add gvn_Latn initialized from tpi_Latn.
3. Save initial gvn embedding row.
4. Train a tiny 1-row or 8-row run with the planned LoRA config.
5. Compare final gvn embedding row and lm_head row to the initial copy.
6. Repeat with --modules-to-save model.shared,lm_head.
```

Decision rule:

```text
If gvn_Latn stays unchanged without modules_to_save, all previous gvn-token A/B conclusions
are weaker than we thought.

If modules_to_save changes the embedding row and improves tiny overfit reproduction, every
future gvn-token run should save/train model.shared and lm_head, or else we should use the
tpi_Latn proxy deliberately.
```

## 7. Grammar Cheat Sheet Summary

The full working cheat sheet is public. This section summarizes the operational rules used
by the v21s0 generator.

### 7.1 Simple Nominal Sentences

The safest first template is a nominal subject plus adjective/predicate without English
"is/are" as a separate word.

Pattern:

```text
NP ABS + adjective/predicate
```

Generated style:

```text
The child is sick. -> Karrkay bambay.
The food is good.  -> Mayi ngulkurr.
```

This is a high-value template because English has many "X is Y" sentences, and the Kuku
Yalanji target does not need a copula in these simple cases.

### 7.2 Case Marking And Word Order

The generator should not rely on English word order alone. It should mark roles using the
case frames in the grammar cheat sheet.

For early synthetic rows, use conservative patterns:

```text
Intransitive:
  S + V

Transitive:
  A-ERG + O-ABS + V

Motion to place:
  S + PLACE-LOC/ALL + motion-verb
```

Where the exact allomorph is not yet robustly modeled, the generator should prefer attested
forms from dictionary/grammar examples and avoid mass-producing uncertain variants.

### 7.3 Tense/Aspect

The pilot uses only simple, high-confidence tense/aspect endings already seen in grammar
examples or corpus rows.

The first expansion should remain conservative:

```text
past/perfective rows
nonpast/present rows
motion/location rows
want/like rows
afraid rows
```

Avoid generating a broad tense/aspect grid until the suffix/allomorph lint has stronger
coverage.

### 7.4 Dative Experiencer Frames

The pilot includes:

```text
wawu / want-like or like-like frames
yinil / afraid-like frames
```

These are useful because everyday translation needs feelings, desires, and preferences.
They also help reduce the model's dependence on Bible-style discourse.

### 7.5 Loan Words

The user explicitly asked that English sentences use known dictionary words and sometimes
English loan words.

The generator therefore keeps a small loan-control template family. Loan words must be
explicitly marked in metadata so later scoring can separate:

```text
dictionary-grounded native-base rows
grammar-example-grounded rows
Bible-attested rows
loan-control rows
```

Loan words should not dominate the corpus. They should appear as realistic controlled
examples, especially for modern nouns where the corpus already shows borrowing patterns.

## 8. Synthetic Data Policy

The synthetic-data policy is now:

```text
Synthetic rows are approved for internal project training/research.
Synthetic rows are not automatically elder-verified reference gold.
Every synthetic row must carry provenance and generation metadata.
Every training run must report synthetic-vs-real mix and eval by route.
```

The distinction matters because model training can use noisy or synthetic rows, but product
claims should not pretend every row has been elder-verified.

The v21 JSONL rows carry:

```json
{
  "pair_kind": "synthetic_candidate",
  "rights_status": "project_approved_synthetic_pending_elder_verification",
  "approved_for_training": true,
  "synthetic": {
    "method": "...",
    "template": "...",
    "confidence": "...",
    "review_status": "auto_reviewed_pending_elder_verification",
    "bases": ["..."],
    "evidence": ["..."],
    "notes": ["..."]
  }
}
```

This lets us train now while preserving reviewability later.

## 9. v21s0 Synthetic Pilot

Generated artifact root:

```text
/mnt/donto-data/donto-resources/research/translation-training/kuku-yalanji-runpod-2026-06-30/synthetic/v21s0-grammar-grounded-candidates-2026-07-02/
```

Public root:

```text
https://mobtranslate.com/model-artifacts/kuku-yalanji-nllb-lora/v21s0-grammar-grounded-synthetic-candidates/
```

Files:

```text
candidates/v21s0_synthetic_candidates.eng-gvn.jsonl
candidates/v21s0_synthetic_candidates.accepted_for_review.eng-gvn.jsonl
review/v21s0_synthetic_candidate_review.jsonl
review/v21s0_synthetic_candidate_review_summary.json
reports/v21s0-grammar-grounded-synthetic-pilot-report-2026-07-02.md
manifests/v21s0_manifest.json
```

Summary:

```text
candidate_rows:                         930
accepted_for_manual_review_pool_rows:   930
held_for_missing_base_review_rows:         0
dictionary_entries_loaded:             2287
reference_examples_loaded:            14154
```

Reference examples used for automatic comparison:

```text
dictionary_example:     89
grammar_xigt:          795
bible_train:        13,270
```

Template counts:

```text
nominal_subject_adjective:       147
intransitive_subject_past:        30
intransitive_subject_nonpast:     30
intransitive_motion_to_locative:  32
transitive_agent_object_past:    496
wawu_dative_want_like:           140
yinil_dative_afraid:              45
attested_loan_word_pattern:       10
```

Confidence labels:

```text
medium_high: 224
medium:      706
```

The pilot intentionally did not generate thousands of rows in the first pass. It proves the
format, metadata, review summary, and public artifact path. The next generator version can
scale the same method.

## 10. Why Synthetic Data Is Not Optional

The project currently has:

```text
large Bible-domain supervision
small dictionary/example supervision
very small elder-shared everyday supervision
rich grammar and dictionary documents
```

That means a model trained only on real parallel rows will tend to learn:

```text
1. Bible register.
2. Repeated Bible phrase shapes.
3. Some dictionary/example phrases.
4. Very little everyday compositional behavior.
```

The product needs everyday compositional behavior:

```text
The child went to the house.
The woman saw the fish.
The man is hungry.
I want water.
She is afraid of the dog.
We cooked food.
They went to the river.
```

Those are exactly the rows the grammar+dictionary can generate in large numbers if we keep
the templates conservative and grounded.

The expected failure mode of not using synthetic data is:

```text
The model remains Bible-shaped, because Bible is the only large training signal.
```

The expected failure mode of careless synthetic data is:

```text
The model learns invented grammar errors at scale.
```

The solution is not to avoid synthetic data. The solution is to make synthetic data
traceable, linted, route-specific, and eval-gated.

## 11. Synthetic Expansion Plan

### 11.1 v21s1: 5,000-row controlled expansion

Goal:

```text
Generate about 5,000 project-approved synthetic English/Kuku Yalanji sentence pairs.
```

Allowed template families:

```text
1. nominal subject + adjective/predicate
2. intransitive subject + verb
3. transitive ergative-agent + absolutive-object + verb
4. motion to location
5. location/state at place
6. possession/with frames only where examples are clear
7. desire/like/want frames
8. fear/feeling frames
9. simple negation where grammar examples are strong
10. controlled loan-word rows
```

English generation policy:

```text
Use known dictionary concepts.
Keep sentences short and ordinary.
Occasionally include English loan words.
Avoid generating cultural facts, sacred content, or long narrative claims.
Avoid English words with no grounded Kuku target unless explicitly marked as loans.
```

Metadata policy:

```text
Every row stores template, bases, suffixes, evidence examples, confidence, and generator
version.
```

### 11.2 v21s2: 20,000-row broader curriculum

Only after v21s1 passes linting and a small train/eval run:

```text
Expand to roughly 20,000 synthetic rows.
Balance across nouns, verbs, adjectives, pronouns, locatives, number/person, and tense/aspect.
Downweight uncertain morphological frames.
Keep low-confidence rows separate.
```

The 20,000-row target should not be a blind count target. It should be a curriculum:

```text
core grammar rows
dictionary coverage rows
minimal-pair rows
loan-control rows
negation/question rows
heldout synthetic probes
```

### 11.3 v21s3: model training run

Train only after:

```text
1. gvn_Latn embedding trainability is audited.
2. frozen eval suite exists.
3. synthetic train/eval split is fixed.
4. leakage audit is run.
5. corpus mix is documented.
```

Recommended starting mix:

```text
Bible morphology/base model: use existing v12/v19 knowledge, but do not let Bible dominate
the everyday adapter.

Everyday adapter train mix:
  synthetic everyday rows: majority
  DB usage train rows: oversampled but not leaked into heldout
  elder-shared rows: light replay/canary, not the main objective
  selected Bible rows: light morphology replay only
```

## 12. Training Architecture Recommendation

The evidence argues against a single universal monolith as the next serious goal.

Recommended architecture:

```text
base NLLB model or existing strongest merged checkpoint
  + Bible/reference adapter
  + everyday synthetic adapter
  + usage/dictionary adapter
  + router/lookup layer
```

Reasons:

```text
1. Bible exact reproduction is different from everyday translation.
2. Elder/usage washout has already appeared when mixtures are poorly balanced.
3. Exact known examples should be retrieved, not memorized in weights.
4. Route-specific scorecards are easier to reason about.
5. Adapters are cheaper to train and compare than full merged monoliths.
```

The public page should show:

```text
model version
training mix
adapter route
evaluation files
sample outputs
known regressions
download links
```

The user has also clarified that the website page does not need an interactive model box.
It should show eval outputs and generated examples. That is a better fit for the current
stage.

## 13. Evaluation Plan

A frozen eval suite is mandatory now.

Suggested frozen suite:

```text
eval_bible_high_confidence
eval_bible_sentence_candidate
eval_db_usage_heldout_by_word_id
eval_elder_shared_canary
eval_synthetic_holdout_core_grammar
eval_synthetic_holdout_dictionary_coverage
eval_probe_everyday_english_only
```

Metrics:

```text
BLEU
chrF++ with word_order=2
length ratio
empty prediction count
repeat-token share
content-word recall where available
dictionary-base coverage
English leakage
exact match only for lookup/canary rows
```

Human review:

```text
Elder review later should focus on small, high-information probe sets first:
  100 everyday core sentences
  100 dictionary-coverage sentences
  50 negation/question sentences
  50 loan-control sentences
  50 model-error samples
```

The first elder review should not be thousands of rows. It should be a compact error-finding
set that tells us which templates are safe and which are wrong.

## 14. Leakage And Split Rules

For DB usage examples:

```text
split by word_id or lemma
never let the same lexical item appear in both train and heldout if the goal is lexical
generalization
```

For synthetic examples:

```text
split by template plus lexical base
hold out entire verbs/nouns/adjectives where possible
hold out whole template families for harder probes
```

For elder-shared rows:

```text
do not optimize directly for exact match
use as canary and qualitative probe
```

For Bible:

```text
separate high_confidence_verse and sentence_candidate tiers
separate direct and reference-conditioned tasks
use lookup-first for exact reference output
```

## 15. Next Run Recommendation

Do not make the next GPU run a broad v21 monolith.

Recommended order:

```text
v21s1-generate-5k
  Generate 5,000 grammar-grounded synthetic candidates.
  Publish report and JSONL.
  Lint morphology, dictionary bases, duplicate rows, and train/eval split.

v21a-embedding-audit
  Tiny train, no meaningful model quality target.
  Verify gvn_Latn embedding/lm_head trainability with and without modules_to_save.

v21b-1k-synthetic-overfit
  Train a tiny everyday adapter on 1,000 clean synthetic rows.
  Goal: prove the model can reproduce synthetic grammar rows and not collapse.

v21c-5k-synthetic-everyday-adapter
  Train route-specific adapter on synthetic + DB usage + light Bible replay.
  Evaluate on frozen suite.

v21d-20k-synthetic-curriculum
  Only if v21c improves everyday heldout/probes without catastrophic Bible/usage regressions.
```

Hard gates before scaling:

```text
1. Synthetic holdout chrF++ beats baseline by a meaningful margin.
2. DB usage heldout does not regress.
3. Elder-shared canary does not collapse to nonsense.
4. Bible high-confidence does not degrade beyond the route-specific tolerance.
5. Outputs do not become long Bible-register rambles on short everyday inputs.
```

## 16. RunPod Cost Expectations

The v19 run was reported around:

```text
17m15s, about $0.68
```

The v20 run was still small enough to be a low-dollar experiment. Synthetic everyday
adapter runs should remain cheap if they use LoRA and modest row counts.

Rough expectation:

```text
v21a embedding audit:      <$1
v21b 1k overfit:           <$1
v21c 5k adapter:           ~$1-3 depending GPU/checkpoint
v21d 20k curriculum:       ~$3-10 depending epochs/GPU/checkpoint
```

These are planning estimates, not invoices. The cost control strategy is:

```text
1. generate/lint locally;
2. train small on RunPod;
3. stop pod immediately after artifacts are copied;
4. publish evals only, no always-on model server;
5. scale only after a diagnostic win.
```

## 17. What A Third Reviewer Should Check

Please review these points first:

```text
1. Are the grammar templates in the cheat sheet valid enough for synthetic generation?
2. Are the v21s0 target forms plausible, especially case suffixes and verb suffixes?
3. Is the distinction between project-approved synthetic data and elder-verified gold clear?
4. Should the next model use gvn_Latn with trainable embeddings, or tpi_Latn proxy until the
   custom token path is proven?
5. Is route-specific adapter training the right path, or should a monolithic curriculum still
   be tested?
6. Which templates should be deleted before scaling to 5,000 rows?
7. Which dictionary semantic classes should be prioritized for everyday translation?
8. Should synthetic rows be weighted lower than real rows, or should the everyday adapter
   intentionally let synthetic rows dominate?
```

## 18. Known Risks

### 18.1 Synthetic Grammar Error Amplification

If a suffix rule is wrong, 5,000 rows can teach the model the wrong rule.

Mitigation:

```text
small templates first
metadata for every suffix/base
heldout probes
manual spot review
delete bad template families quickly
```

### 18.2 Bible Register Bleed

If Bible rows dominate replay, short everyday inputs may receive long Bible-like outputs.

Mitigation:

```text
everyday adapter
short-output length gates
route-specific eval
limited Bible replay
lookup-first Bible route
```

### 18.3 Custom Token Freeze

If `gvn_Latn` is not trainable, the language-token experiment is partly fake.

Mitigation:

```text
embedding diff audit
modules_to_save model.shared,lm_head
adapter-vs-merged equivalence checks
```

### 18.4 False Confidence From Synthetic Heldout

Synthetic heldout can be too similar to synthetic train.

Mitigation:

```text
hold out lexical bases
hold out templates
score real DB usage heldout separately
use human review on small probes
```

### 18.5 Overweighting Elder-Shared Train Rows

The 43 elder-shared rows are precious, but optimizing exact match on them teaches
memorization.

Mitigation:

```text
use them as canary/probe
derive templates carefully
do not give them 25% of a model-selection composite
```

## 19. Concrete Implementation Checklist

Already done:

```text
[x] Inventory grammar/dictionary/Bible/DB/elder-shared resources.
[x] Write working grammar cheat sheet.
[x] Publish grammar cheat sheet.
[x] Build v21s0 synthetic generator.
[x] Generate 930 synthetic candidates.
[x] Mark synthetic rows approved_for_training=true.
[x] Publish v21s0 synthetic pilot artifacts.
[x] Decompose v20 Bible direct eval by tier.
[x] Identify likely gvn_Latn embedding trainability risk.
```

Next:

```text
[ ] Add exact suffix/allomorph lint pass.
[ ] Generate v21s1 5,000-row controlled corpus.
[ ] Create frozen eval suite manifest.
[ ] Run leakage audit for DB usage and synthetic splits.
[ ] Run gvn_Latn embedding trainability audit.
[ ] Train v21b 1k synthetic overfit adapter on RunPod.
[ ] Train v21c 5k everyday synthetic adapter only if v21b passes.
[ ] Update public model/eval page with v21s0/v21b/v21c outputs.
```

## 20. Bottom Line

The synthetic approach is not a side experiment. Given the current resource situation, it is
the main data-acquisition path.

The correct way to do it is:

```text
Use grammar and dictionary as the generator.
Use Bible and real examples as surface-statistical checks, not as the product target.
Keep synthetic rows project-approved and metadata-rich.
Use elder verification later as a quality layer.
Train route-specific adapters, not a blind universal monolith.
Audit gvn_Latn embedding trainability before spending more GPU time.
Publish every version with evals and examples so progress is visible.
```

The next technical goal should be:

```text
v21s1: expand from 930 to about 5,000 grammar-grounded synthetic everyday rows, then run a
small RunPod adapter train that proves whether synthetic rows improve everyday translation
without causing route collapse.
```

## 21. References For External Review

Useful background papers already identified during the project:

```text
NLLB:
https://arxiv.org/abs/2207.04672

eBible corpus:
https://arxiv.org/abs/2304.09919

Low-resource translation from grammar books / Aycock et al.:
https://arxiv.org/abs/2409.19151

AmericasNLP Indigenous MT shared task:
https://aclanthology.org/2024.americasnlp-1.28/

Mambai retrieval-augmented low-resource MT:
https://aclanthology.org/2024.eurali-1.1/

LoRA:
https://arxiv.org/abs/2106.09685

ByT5:
https://aclanthology.org/2022.tacl-1.17/

Character-level MT comparison:
https://aclanthology.org/2024.tacl-1.22/
```