mobtranslate.com / docs › Wajarri source: wajarri-v3-research-plan.md

Wajarri v3 Research Plan and Work Log

Program: English to Wajarri (eng_Latn to wbv_Latn)
Canonical source: /mnt/donto-data/donto-resources/research/language-programs/wajarri-v1/WAJARRI-V3-RESEARCH-PLAN-AND-WORK-LOG.md
Public HTML: https://mobtranslate.com/docs/wajarri-v3-research-plan.html
Public Markdown: https://mobtranslate.com/docs/wajarri-v3-research-plan.md
Started: 2026-07-31
Current phase: v3 is complete as a narrowly bounded controlled release. Three independently seeded A40 runs, CPU float32 parity, immutable Hugging Face publication, shared-Space serving, public model-registry publication, production homepage routing, PostgreSQL cache identity, and desktop/mobile browser acceptance have passed. The release supports six dictionary-bound subject-slot constructions only; it is not a free-form Wajarri translator. No new Wajarri target sentence was invented for this release.
RunPod policy: Wajarri worker 77tes61jw5a024 was deleted only after complete result recovery and independent local checksum verification. The post-deletion provider census proves it absent; unrelated Alpha worker wtupxv15debnvh was not touched. No Wajarri RunPod worker remains active.

This is the single living research plan and work log for the next Wajarri model. It replaces scattered progress notes. New evidence, corpus decisions, experiments, failures, and promotion rulings are appended here.

Executive decision

Do not train for more epochs on the present corpus. Publish and serve v3 only through its fail-closed six-construction contract; keep dictionary lookup outside the model and route every unsupported sentence to the ordinary prompt-based path.

The final experiment succeeded by narrowing the task instead of pretending that sparse evidence could support unrestricted generation. The model receives a masked English construction, the decoder is seeded with the target-language and <copy> control tokens, and the model generates one of six complete source-bound predicates. A deterministic renderer replaces the single first-position slot with an exact Wajarri dictionary headword. This produced 35/35 exact templates and rendered clauses for all three independent seeds and exact GPU-to-CPU parity. It does not license novel inflection, arbitrary syntax, or a speaker-validated naturalness claim.

Wajarri v2 is a strong closed-set lexical reconstruction artifact, but it is not a competent sentence translator. The selected checkpoint reconstructs 1,076 of 1,179 one-target prompts exactly (91.26%), yet scores 0 of 5 exact and 9.62 chrF++ on the small complex historical diagnostic. The selected checkpoint consumed only the first 640 presentations of a planned 3,712-presentation recovery schedule: 509 lexical presentations, 111 synthetic sentence presentations, 14 historical sentence presentations, and six fixed-utterance presentations. The remainder of that schedule was never presented to the selected checkpoint and must not be described as its training trajectory.

The paired source-record audit also rules out continuing the current raw definition-context objective. On the same 1,684 records, direct prompts reproduce the source target 1,094 times while definition-context prompts do so 878 times. Definition context breaks 227 direct successes and rescues only 11 direct failures. This does not prove that definition text is intrinsically harmful; it proves that the present ungoverned prompt is harmful for this checkpoint. v3 will test explicit POS/sense labels and sentence-conditioned lexical use instead.

The checkpoint-level retention diagnosis is now complete. It found one safe control checkpoint, no safe treatment checkpoint, and a monotonic tradeoff in which composition fitting overwrote governed lexical forms. The immediate task is therefore source-backed contrast acquisition and a task-separated sentence-adapter design, followed by a new controlled experiment. More raw epochs, repeats of the present 36-row matrix, or undifferentiated sentence volume are not justified by the evidence.

Current controlling result

The controlling result is corrected confirmation run wbv-v3-forced-slot-confirmation-runpod-20260802-a2. It passes only the controlled subject-slot route. The first apparent three-seed result was invalidated because Transformers silently used Trainer seed 42 in all three runs; the replacement run explicitly bound both seed and data_seed, failed unless the runtime values matched, and required three distinct selected adapter hashes.

Seed Selected update Raw template exact Rendered clause exact Batch 1/16 mismatches Retention Synthetic exact Synthetic chrF++ One-target lexical exact Adapter SHA-256
17 40 35 / 35 35 / 35 0 54 / 55 22 / 36 75.4246 813 / 1,179 a4a91d4e6a6ba49509a8ab432c0674686c96b4611f58db8b63089f26707ed3d6
42 40 35 / 35 35 / 35 0 54 / 55 22 / 36 75.3028 816 / 1,179 d59a4a276d4968e6d4e453dd534ebca6edfaacf936e1bd97cf7c3eb4be1c8801
73 40 35 / 35 35 / 35 0 54 / 55 22 / 36 75.1763 824 / 1,179 8574727bbf9cfb3e5032e9d922d7af61d6eb62f4b6b0976d6fc06b18b7dd82f7

Every seed had zero blank, source-copy, repeated-four-gram, unresolved-slot, and unresolved-bracket faults on the authorized route. The winning seed-17 adapter then produced byte-identical normalized predictions under A40 bfloat16 and CPU float32 at batch sizes 1 and 16. The prediction SHA-256 was b17ab8d44ccaaa8dba962ecf46486d731d7aaf8d4c6a7adeed430044612cf25f in all four conditions.

The lower one-target lexical scores are not hidden. They demonstrate why v3 must not replace deterministic dictionary lookup or the public lexical route. The model is authorized only for this composition contract:

public English:       The bilby is running.
dictionary binding:  bilby = marruwa
model input:          <translate> The <copy> is running.
decoder prefix IDs:   [2, 256204, 256208]
raw model output:     <copy> jamarnimanha.
rendered result:      marruwa jamarnimanha.

The other complete predicates are yanajimanha (coming towards the speaker), yanmanha (going away from the speaker), nyinamanha (sitting), nyinarangamanha (sitting down), and garrimanha (standing). The router requires one exact dictionary headword and one exact frozen construction. It refuses malformed slot outputs. Unsupported constructions never invoke this adapter and continue through the ordinary dictionary-prompt route.

Public component Immutable identity
Model ajaxdavis/mobtranslate-wajarri-v3-controlled commit 1a26382648e294f2c102102ef914c7b0c86bec20
Model tag v3.0.0-controlled-subject-slot-s17-step40-20260802
Dataset ajaxdavis/mobtranslate-wajarri-synthetic-corpus-v1 commit 302f3a93eb255a2fc70692b7c429ba5ee8300687
Shared Space ajaxdavis/alpha-v0-historic commit 03b6d4ed9c9ffb8ac2e55676447084af2c7aa5c8
Homepage release 20260802T103148Z-72668fe89dc5

The release claim is therefore positive but narrow: exact controlled reconstruction and rendering over six source-bound predicates and exact dictionary-bound subjects. Free-form translation, productive morphology, historical-text competence, discourse competence, and fluent-speaker validation remain unproven and unauthorized.

First causal screen result

The one-seed B0/C0/T1 A40 screen completed on 2026-07-31. B0 is the untouched public v2 adapter, C0 is a step- and token-matched retention continuation, and T1 is the 36-row composition intervention. The treatment caused measurable sentence learning, but it failed the lexical noninferiority gate and is not a release candidate.

Development measure B0 C0 T1
Exact sentence 0 / 13 0 / 13 3 / 13
Both required slots present 0 / 13 0 / 13 3 / 13
Required subject present 7 / 13 9 / 13 7 / 13
Required predicate present 1 / 13 1 / 13 7 / 13
Mean chrF++ 42.03 43.18 60.56
Full regression measure B0 C0 T1
One-target lexical exact 1,076 / 1,179 1,072 / 1,179 1,061 / 1,179
Controlled-synthetic exact 13 / 36 12 / 36 12 / 36
Controlled-synthetic chrF++ 70.90 71.54 70.02
Fixed-utterance retention 55 / 55 55 / 55 55 / 55

Relative to B0, T1 gained eight exact one-target lexical rows and lost 23, for a net loss of 15. Its 36 unique composition rows were each presented 16 times, concentrating training on only four predicates. This establishes that direct composition supervision can move the model; it does not establish broad Wajarri grammar or sentence reliability. T1 is not promoted, not uploaded, and not served.

The next experiment therefore lowers the composition dose and interleaves broad lexical anchors. It also uses a fresh 32-cell development matrix made from eight source-bound subjects not used in the first matrix. No new run may tune against the consumed 13-cell development set.

Evidence audited on 2026-07-31

Current model

Measure Wajarri v2 result
One-target direct lexical exact 1,076 / 1,179 (91.26%)
Definition-context exact 878 / 1,684 (52.14%)
Controlled-synthetic holdout exact 13 / 36
Controlled-synthetic holdout chrF++ 70.90
Complex historical holdout exact 0 / 5
Complex historical holdout chrF++ 9.62
Mechanical faults under frozen decoder 0

The selected model remains public at ajaxdavis/mobtranslate-wajarri-v2. Its release card correctly states that free-form translation has not passed a natural-language gate.

Final recovery schedule and actual selected-checkpoint exposure

The complete planned context-recovery schedule was:

Schedule role Presentations Share
Definition-context lexical recovery 1,684 45.37%
Direct lexical recovery 1,211 32.62%
Sentence-context retention 760 20.47%
Fixed-utterance retention 32 0.86%
Batch-alignment padding 25 0.67%

The schedule contained 3,712 presentations but only 3,093 unique parent rows. Presentation count is not independent evidence count. More importantly, the public checkpoint is step 20 and consumed only the following prefix:

Consumed role Presentations Share
Definition-context lexical recovery 287 44.84%
Direct lexical recovery 222 34.69%
Synthetic sentence retention 111 17.34%
Historical sentence retention 14 2.19%
Fixed-utterance retention 6 0.94%
Total actually consumed 640 100%

Its complete documented project lineage is:

Lineage stage Consumed presentations Unique exposed pairs or rows
v1 lexical/context/synthetic checkpoint 800 44,800 599
v2 lexical-dose checkpoint 2,220 71,040 1,409
v2 sentence-recovery checkpoint 25 800 421
v2 context-recovery checkpoint 20 640 640

These are optimizer presentations, not independent linguistic observations. Upstream NLLB pretraining exposure is unknown.

Synthetic sentence coverage

There are 211 controlled pairs across only six construction families:

Construction family Rows
Positive stative, small 69
Positive stative, good 42
Positive stative, hungry 30
Positive stative, cold 25
Present intransitive, going away 24
Present intransitive, sitting down 21

The synthetic holdout is not an independent grammar test. It reuses all six construction families, all six predicate forms, and 12 of the 13 training subjects. Its score measures recombination within a small closed inventory.

All 23 selected-model errors on this holdout preserve the expected predicate while substituting the subject noun. Examples include:

English Reference Model
The old woman is cold. Jugarnu janda. Jardu janda.
The turtle is going away. Gandarra yanmanha. Galyba yanmanha.
The sandhill is good. Dungguru barndi. Burlga barndi.
The younger brother is sitting down. Buwa nyinarangamanha. Matarna nyinarangamanha.

This identifies lexical selection inside sentence generation as a primary error, not failure to reproduce the six predicates.

Sentence lineage, task pairing, and split integrity

The 211-row public synthetic corpus contains two record shapes: 57 rows carry an explicit grammar binding inline, while 154 legacy rows carry their sentence and lexical-compatibility decisions in external review ledgers. All 154 legacy rows join to both required ledgers. They are reviewed research synthesis, not speaker-attested references. Every v3 row will carry its binding and review provenance inline so record identity alone is sufficient to audit eligibility.

The private v2 training payload contains 166 synthetic training parents. Of these, 164 have the same record identity in the public corpus. Two older training IDs duplicate public holdout source-target pairs under different newer IDs. One of those two training parents occurs in the selected checkpoint's first 640 presentations. Neither duplicate occurs in the reported 36-row evaluation subset, so this defect does not explain its 13 exact matches; however, those two public holdout identities are contaminated and are retired. A separate evaluated legacy ID was deduplicated to a newer public record with the same pair. v3 must assign splits to normalized source-target groups before record-level deduplication.

Thirty-five of the 36 evaluated sentence rows have an exact join to a local dictionary target for the expected subject. Their paired outcomes are:

Isolated direct task Sentence task Rows
Exact Exact 12
Exact Failed 18
Failed Exact 1
Failed Failed 4

The 18 sentence failures after exact isolated reconstruction are the clearest evidence for sentence-conditioned lexical realization or composition failure. The five isolated-task failures are not five independent lexical phenomena: they collapse to three source-record clusters. Three butterfly rows consistently select a competing surface across both tasks; the sandhill row selects different competing forms by task; and the kookaburra sentence is exact even though isolated reconstruction selects another recorded form. Those clusters require sense, synonym, variety, and orthographic adjudication before accepted references or training overlays change.

Canonical lineage artifacts are under analysis/benchmark-runs/wbv-v3-v2-sentence-lineage-audit-v1.1/. The contract SHA-256 is 9559e794a2f22114a77766e4bc927213d112958f4008459e01ac65f2ab049b56; the report SHA-256 is 7d31914dd85a2e5d554c1804a421e4faa2bbc08ca6c5f873234d468b6b967af3. All artifact checksums pass, and a clean rerun is byte-identical. The earlier v1 audit is retained and explicitly superseded because it conflated an evaluated legacy row's external review provenance with the inline provenance of its deduplicated public replacement.

Lexical failure character

Among the 103 one-target direct failures:

Failure class Count
Wrong known Wajarri target 62
Different surface form 27
Near surface form 11
Over-generation 2
Under-generation 1

Exact recall by reference-token length is 89.6% to 93.8% across every bucket from one token through five-plus tokens. Tokenizer fragmentation may matter, but it does not explain the main lexical failure pattern. The current model usually generates well-formed known vocabulary and selects the wrong item.

Grammar and source frontier

The current grammar edition contains:

Evidence class Count Current use
Douglas numbered examples 269 OCR candidates, not semantically admitted
Visually curated Douglas examples 40 Transcription accepted; alignment unresolved
Table blocks 11 Paradigm review required
Morphotactic statements 16 Historical analysis requiring reconciliation
Source-anchored grammar assertions 69 Mostly not independently accepted for generation
Historical sentence witnesses 29 24 train, 5 development diagnostic
Productive synthetic templates 6 Narrow present/stative scope
Independent natural speaker-validated rows 0 Natural promotion gate remains closed

Seven downloaded ICTV videos total about 20 minutes and contain Wajarri teaching, profiles, oral history, narrative, cooking, and place material. They have no accepted transcript or alignment yet. The 50words review contains 55 source pairings, including eight fixed utterances and 165 audio links. Several bilingual Wajarri books are identified but not yet acquired as aligned text.

Research basis

The revised design follows several results from low-resource machine-translation research:

  1. The current FormosanBank NLLB adaptation guide recommends directional checkpoints, source/document-level split discipline, a single-token language code, correct NLLB decoder configuration, train-only tokenizer adaptation, and corpus improvement before model tuning.
  2. GATITOS finds that curated bilingual lexicons improve low-resource translation and that lexicon quality matters more than indiscriminate noisy scale.
  3. Rare-word data augmentation improves translation by placing low-frequency words in new sentence contexts rather than merely replaying isolated pairs.
  4. A recent Indigenous-language synthetic-data study finds gains from curated plus filtered synthetic data and deterministic language-specific normalization.
  5. The Mambai retrieval study finds gains from combining dictionary entries with retrieved parallel examples, while showing that manual-derived tests can badly overstate performance on native-speaker material.
  6. A 2026 PEFT comparison finds full fine-tuning strongest, OFT statistically close, and LoRA a strong efficiency-quality tradeoff across 13 Indigenous language pairs. Data quality and task coverage remain prior questions.
  7. Published 20,000-pair results used 20,000 high-quality parallel pairs and a 16,000-sentence evaluation set. That scale is not comparable to repeating six deterministic templates.
  8. Wajarri is suffixing and agglutinative, with split-ergative case, bound pronouns, TAM contrasts, and subordinate-clause dependencies. These must be represented explicitly rather than inferred from a word list. See Marmion's Wajarri overview.

Phase 1: benchmark and failure ledger

Status: Complete for the published Wajarri v2 baseline. The ledger is frozen and reproducible; later models must be scored against the same rows plus newly sealed suites.

Lexical suites

Build and freeze:

Suite Population Primary purpose
L-CENSUS 2,511 dictionary source records: 1,684 current and 827 historical Complete source-record accounting
L-DIRECT 1,408 normalized direct prompts Ambiguity-aware reconstruction
L-ONE 1,179 one-target prompts Strict exact reconstruction
L-CONTEXT 1,684 current source-record definition prompts Closed definition-conditioned reconstruction
L-SENSE To be frozen after sense-level review Genuine POS- and sense-qualified selection

The older count of 3,016 was the sum of 1,408 direct prompt groups and 1,608 grouped definition-context rows. It was not a count of dictionary source records. The selected v2 report contains 3,092 lexical outcomes: 1,408 direct prompt outcomes plus one definition-context outcome for each of 1,684 current source records. These are related views over the dictionary, not 3,092 independent lexical facts.

Every model failure receives a stable row ID and the following machine-readable fields:

Semantic groups will be discovered with embeddings and then reviewed. They will not be maintained as a brittle list of English keyword rules.

Frozen v2 baseline findings

The checksum-bound run evaluated 3,188 rows and produced 1,143 failures. Of those, 1,115 are lexical failures, 23 are controlled-synthetic sentence failures, and five are historical sentence failures. There were no blanks, source copies, repeated four-token loops, or retention failures under the selected decoder.

Frozen suite Exact Result
Direct lexical, all prompt groups 1,099 / 1,408 78.05%
Direct lexical, one accepted target 1,076 / 1,179 91.26%
Direct lexical, multiple accepted targets 18 / 224 8.04%
Definition-context, current source records 878 / 1,684 52.14%
Controlled-synthetic holdout 13 / 36 36.11%
Historical diagnostic 0 / 5 0%
Fixed-utterance retention 55 / 55 100%

The direct benchmark also contains five explicitly reconciled cross-source conflicts; all five score exact against the declared union of references. They are retained as their own ambiguity class rather than silently folded into one-target or multi-target rows.

The one-target result is a closed-set reconstruction measurement, not lexical generalization: all 1,179 prompts had an exact input-target exposure somewhere in the documented model lineage, and 103 still fail. In the lexical-dose stage alone, 1,194 of all 1,408 direct prompts had an exact accepted pair exposed; these score 1,091/1,194 (91.37%), while the 214 prompts without that exact exposure score 8/214 (3.74%). This is a post-hoc association, not a causal estimate.

The selected recovery checkpoint saw 287 of the 1,684 definition-context rows once. Those rows score 157/287 (54.70%); the other 1,397 score 721/1,397 (51.61%). One presentation is not a meaningful context-dose experiment. A controlled dose study remains justified, but simply completing the old schedule would not answer whether definition context, explicit sense labels, or sentence use caused an improvement.

The stronger paired result is operationally decisive. Across the same 1,684 source records, direct prompting is source-target exact for 1,094 records and raw definition-context prompting for 878. The paired outcome classes are 867 both exact, 227 direct only, 11 context only, and 579 neither. Raw definition context therefore has a -12.83 percentage-point paired exact-rate difference. The records are clustered by prompt and dictionary structure, so this is a descriptive frozen-benchmark result rather than an independent-binomial population estimate.

Across all lexical failures, the model generated another recorded Wajarri dictionary target 685 times. The existing coarse surface labels contain 691 wrong-known-target outcomes, 306 different forms, 47 near forms, 37 over-generated forms, and 34 under-generated forms. This makes lexical competition and sense selection the main measured failure mode. More isolated replay alone is not the preferred intervention.

All 23 controlled-synthetic failures preserve at least one final reference word, and inspection shows the expected predicate is retained while the subject slot changes. Because all 36 rows come from only six already-trained construction families, this test does not discriminate broad grammar competence. The next sentence suite must hold out complete construction families and morphology combinations.

Canonical Phase 1 artifacts are under analysis/benchmark-runs/wbv-v3-v2-baseline-failure-ledger-v1.1/. The contract SHA-256 is 0f020d91acc724fa949fec3ab1ae940ac05096ce840dfd1df651917b28ddd107; the summary SHA-256 is 4be01f8540fac12d9ad53c245a1f3ffa968ddbf1ae50e361143d8d893ca39fe7. All recorded artifact checksums pass, and a clean second generation produced identical hashes.

Sentence and morphology suites

Create four distinct suites:

Suite Initial target Split rule
Closed-template development Existing 36 Development-consumed; never final evidence
Historical development Existing 29 Source identity retained; five seen diagnostic rows remain development-consumed
Morphology and contrast 400-600 Hold out lexeme families and feature combinations
Compositional sentence test 300 Hold out complete template families and lexical combinations
Natural final test At least 150 At least three independent speaker/text/source clusters

Sentence rows will expose gold slots for subject, predicate, arguments, case, TAM, polarity, clause count, dependency relation, and construction family. Aggregate chrF++ cannot hide failures in negation, participant roles, or clause completeness.

Phase 2: evidence conversion

The frozen pre-generation commission is now materialized at analysis/corpus-requirements/wbv-v3-3000-sentence-commission-v0.1.0/. It specifies work to be reviewed and generated; it is not itself a synthetic corpus.

Commission component Frozen count
Construction families 24
Future unique pairs per family 125
Future train / development / sealed-test pairs 2,400 / 300 / 300
Historical Douglas source witnesses 40
Current SCSA curriculum source witnesses 20
Source-specific lexical failures retained for review 817
Highest-priority lexical items selected for sentence overlay 500
Douglas phenomenon tags assigned to families 49 / 49
Families supported by bounded existing templates 2
Families still requiring productive-rule review 22

The 24 families cover property, identity, possession, location, intransitive and transitive TAM, negation, imperatives, role reversal, questions, spatial cases, cause/purpose, temporal dependency, and object-modifying dependency. Every family receives exactly 100 train rows and 25 held-out rows. Development and sealed-test allocations alternate 12/13 by family so both total 300. The design also requires at least 600 morphology or participant-role contrasts, 500 reviewed lexical-competition rows, and 300 additional glossary-conditioned presentations. Isolated dictionary pairs do not consume the 3,000-sentence budget.

All 60 published sentence witnesses remain review-only. The original SCSA PDF is now archived with SHA-256 ebcd824b6b6679e7c9e875bbff5b3bc27b0f795b56889eb08f76c12868aefe0d; the source-fidelity audit verifies all 86 candidate records and 106 cited spans against it with minimum target- and English-example token recall of 1.0. That resolves source capture, not linguistic acceptance. The 40 Douglas transcriptions still require semantic/alignment and current-orthography adjudication; the 20 commissioned SCSA pairs still require segmentation, morphology, sense, and variety review before they can license productive generation. All 24 families remain generation_authorized=false, and RunPod remains blocked.

  1. Complete semantic alignment and current-orthography review for the 40 visually curated Douglas examples.
  2. Review the remaining 269 numbered examples in construction-priority order.
  3. Reconcile 11 table blocks and 16 morphotactic statements with current Wajarri evidence.
  4. Preserve recorded and source-hypothesized paradigm forms as different states; never train on hypothesized cells as attested forms.
  5. Transcribe and align the seven downloaded ICTV videos.
  6. Integrate the eight accepted 50words fixed utterances and other exact public bilingual witnesses where split and source identity permit.
  7. Acquire and align suitable passages from the identified Wajarri dictionary, teaching books, and bilingual narratives.

The Douglas and Marmion disagreement over switch-reference is not resolved. Switch-reference generation remains disabled until the analysis and surface realizations are reconciled.

Phase 3: controlled corpus expansion

Synthetic scale is conditional:

Stage Unique pair target Stop/go requirement
Pilot 3,000 At least 18 validated construction families and clean audits
Expansion 8,000 Improvement on unseen lexeme/construction development suites
Full 15,000-20,000 Independent gains without family-specific regression

The pilot should cover, when validated by the evidence ledger:

Each pair must carry immutable provenance, grammar-rule IDs, lexeme IDs, semantic slot restrictions, morphology analysis, generator version, split-group ID, split, and audit status. Split groups are assigned from normalized source-target identity plus governed construction and lexeme families before any record-level deduplication. Unique evidence pairs and repeated optimizer presentations are always counted separately.

No macro construction family may exceed 8% of the pilot. No predicate form may exceed 3%. Lexical bindings must obey animacy, valency, semantic-class, and variety restrictions. English prompts must include natural statements, questions, and commands rather than telegraphic template fragments.

Glossary-conditioned task

Train and evaluate sentence use of retrieved dictionary entries separately:

<translate> The old woman is cold.
<glossary> old woman = jugarnu; cold = janda

This asks the model to apply supplied lexical knowledge in grammar, rather than recover arbitrary hidden dictionary mappings. It directly targets the subject-substitution pattern observed in the current synthetic holdout.

Long-tail dictionary words remain lookup-first in production. Context generation will focus on semantically useful core vocabulary and the diagnosed failure population, not force rare names, flora, and fauna into implausible sentences merely to increase a counter.

Phase 4: controlled model experiments

Do one-seed screening under identical token and update budgets:

Arm Treatment
B0 Frozen Wajarri v2 baseline
C0 Step- and token-matched sentence retention continuation
S Balanced grammar-grounded sentence pilot
SG S plus glossary-conditioned sentence use
FSG Same winning data from a fresh frozen Wajarri-compatible base
TOK Winning data with a train-only validated tokenizer extension

The initial mixture target is 65% ordinary grammar-grounded sentences, 15% morphology/contrast rows, 10% glossary-conditioned sentences, 5% natural or historical retention, and no more than 5% isolated lexical replay.

The primary causal question is whether sentence and glossary supervision improve unseen construction and lexical-combination behavior at fixed compute. The secondary question is whether inherited lexical adaptation helps or harms sentence learning.

Only after selecting the data recipe will the program compare LoRA r16, OFT, and one bounded full fine-tune. The winning recipe is then repeated with seeds 17, 42, and 73. Recipe selection uses aggregate development performance, not the luckiest seed.

All weight comparisons use one frozen greedy decoder. Beam search, repetition guards, and reranking are evaluated afterward as serving interventions over fixed weights.

Phase 5: promotion gates

Dictionary lookup, lexical reconstruction, and sentence generation have independent gates.

Dictionary route

Lexical model

Sentence model

Automatic metrics are diagnostics. They do not authorize natural translation by themselves; low-resource learned metrics can correlate weakly with human judgments. See the ACL evaluation study.

Product experiment

Benchmark four routes on the same sealed rows:

  1. NLLB adapter alone;
  2. NLLB with retrieved glossary entries;
  3. retrieval-grounded large-language-model translation;
  4. NLLB draft plus retrieved dictionary, grammar evidence, and constrained post-editing.

The homepage should continue to expose route provenance. Known one-word queries use deterministic dictionary lookup. Sentence output remains a research draft until the sentence gate passes.

Resource estimate

Work Expected scale
Completed C2/T2 one-seed screen About 20 minutes of A40 worker time at USD 0.44/hour; nominal compute about USD 0.15 before provider billing granularity
Next checkpoint-sentinel diagnostic Estimated 20-35 A40 minutes, about USD 0.15-0.26 before startup and billing granularity; not authorized
Source review for the first contrast commission 48-96 new unique pairs only after the evidence gates below pass
Broader factorial expansion 80-160 new unique pairs only after the first contrast screen passes
Multi-seed confirmation Deferred until one recipe passes all checkpoint and full-regression gates

These are planning bounds, not promises. Training begins only after data hashes, splits, token accounting, and evaluation code are frozen.

Work log

2026-07-31: research audit and decision

2026-07-31: Phase 1 started

2026-07-31: Phase 1 completed

2026-07-31: Phase 2 started

2026-07-31: sentence lineage and composition audit completed

2026-07-31: live cluster probe and source-bound triage completed

2026-07-31: curriculum-source fidelity checked

2026-07-31: model-bound 20-sentence pre-census completed

The live census used 20 source-bound sentence previews and nine unique lexical realizations. It produced no blanks and no source copies.

Outcome Count
Direct lexical realizations exact 7 / 9
Sentences exact 2 / 20
Both lexemes exact but sentence failed 10
Lexical failure and sentence failure 8
Expected subject present 8 / 20
Expected predicate present 4 / 20
Both expected slots present 2 / 20
Common-noun family exact 2 / 15
First-person family exact 0 / 5

Representative failures show distinct mechanisms rather than one generic quality problem:

English Source-bound candidate Model Observed failure
The child is running. Mayu jamarnimanha. Mayu yanmanha. Subject retained; predicate collapses to going
The dog is sitting. Duthu nyinamanha. Duthu nyinarangamanha. Specific sitting-down form replaces sitting
I am coming. Ngatha yanajimanha. ngurragi yanmanha Fixed first-person/home-going phrase overreaches
The woman is sitting. Nyarlu nyinamanha. Jurdu nyinarangamanha. Sister/female-relative subject and sitting-down frame substitute
The child is standing. Mayu garrimanha. Jura nyinarangamanha. Competing child form plus wrong posture frame

The census contract SHA-256 is 10426dd564eafd8c83ddaa1963c1714aedf9ac18354521524787db15877d2987; the report SHA-256 is 972394210283ec0d127781b05653b72921cf15d7de1ae3f7d1db0f5463dc62be. Every output checksum passes.

2026-07-31: selected-checkpoint lineage attribution completed

2026-07-31: narrow composition intervention frozen

This section supersedes the earlier “300-pair dry run, then 3,000 pairs” next gate. The 3,000-pair commission remains useful as a long-range coverage plan, but current evidence supports a smaller causal screen first.

The new commission exhaustively classifies 14 subjects by five bounded present-intransitive predicates. It composes only complete source-bound forms; it infers no stem, suffix, conjugation, or unseen surface.

Cell disposition Count Role
Novel training candidate 36 Nine training subjects by four new predicates
Existing clean retention 10 Already-issued going-away pairs
Development candidate 13 Held-out child, dog, and first-person combinations
Blocked reference conflict 1 Child going-away has competing Mayu and Jura targets
Blocked subject adjudication 5 Woman/nyarlu versus observed Jurdu competition
Excluded semantic compatibility 5 Fish is not forced into a terrestrial posture/motion matrix
Total classified 70 Every possible cell has an explicit disposition

Four contrastive requirements are mandatory before screening: coming versus going away; sitting versus sitting down; first-person going away versus going home; and Mayu/Jura plus Nyarlu/Jurdu subject competition.

The original census used four generic-going prompts even though yanmanha is bound to going away from the speaker. It also used four generic-coming prompts for the multi-target English cue coming, while the selected source-scoped form yanajimanha is specifically motion towards the speaker. Those eight prompts are retired from exact comparison and replaced in the intervention by going away and coming towards the speaker. Both original exact sentence results came from the retired generic-going set. The remaining source-qualified live subset is therefore 0/12 exact, while direct reconstruction remains 7/9 exact.

The commission contains zero training-eligible rows. This is intentional: it freezes what must be reviewed without silently converting dictionary concatenation into accepted language. The first v1 commission is preserved with its published hashes and explicitly superseded because its coming prompt was not sense-qualified. The controlling v1.1 contract SHA-256 is b698bca492916fec56dc0f2bdf211f80363652d639fcc9a6b4cf6a366e9b4367. Report SHA-256: 10dfc96e1a98b19aef81d63b4800049d69551311e55956cef783c6ea56bfef22. Manifest SHA-256: 979ec403054352e79561eec98b91b46425feadf6b33b0523e3da21fdb303d048. All checksums pass, and a clean rerun is byte-identical.

Controlling execution order

  1. Review the 36 novel cells against the bound dictionary forms, existing intransitive template, source-specific predicate semantics, and explicit subject compatibility decisions.
  2. Resolve or omit the one child reference conflict and five woman cells. A blocked row cannot delay the clean core experiment and cannot be silently guessed.
  3. Materialize one immutable intervention payload that separates unique evidence rows from optimizer replay. Preserve the ten clean going-away rows as retention; do not count them as new evidence.
  4. Freeze three step- and token-accounted arms from identical v2 weights: untouched B0, retention-only continuation C0, and composition treatment T1. Pair row order, batch boundaries, learning-rate schedule, maximum updates, and decoder policy.
  5. Run one screening seed on RunPod only after payload hashes, token counts, model/tokenizer identity, and development scoring are frozen.
  6. Require improvement on held-out subject-predicate cells, no material loss on the complete lexical census, and no regression on retained sentence families. A lexical score cannot authorize sentence translation.
  7. If the narrow screen fails, inspect cell-level substitutions and revise the task or conflict policy. Do not buy a larger run with 3,000 repetitions of a failed construction.
  8. If it succeeds, confirm the recipe with seeds 17, 42, and 73, then activate additional grammar families from the 3,000-pair commission in evidence-backed batches.

The earlier RunPod block is now satisfied by the source review, paired schedule, corrected model-bound baseline, and sealed kit described below.

2026-07-31: source review issued the bounded internal treatment

2026-07-31: paired fixed-compute schedules frozen

Three rejected designs are preserved because they changed the causal question:

  1. Same-subject control matching was 9.97% shorter in non-padding tokens and consumed five old holdout rows.
  2. Independent nearest-row matching remained outside the 1% token gate.
  3. Restricting controls to only 40 inline-reviewed rows could not reach the treatment target-token budget.

The controlling v1.3 schedule admits externally reviewed legacy rows only when both the sentence-review and lexeme-compatibility ledgers pass. It uses no old holdout row.

Property C0 retention control T1 composition treatment
Distinct rows 36 36
Presentations 576 576
Non-padding tokens 10,592 10,592
Optimizer updates 18 18
Physical batch / accumulation 4 / 8 4 / 8
Checkpoints 3, 6, 9, 12, 15, 18 3, 6, 9, 12, 15, 18

Both arms start from the exact public adapter weight SHA-256 05e6c40172d93b99538b3a21a04bbbd14a6c767180f85b83d5f00064a6858fb4. They share seed 17, learning rate 2e-5, warmup, batch boundaries, LoRA topology, and greedy decoder. The only intended difference is the treatment evidence.

Schedule contract SHA-256: 28103ce918fe123bd22bd17dd5cbe3382de79105dce555d0dd0ea59181185acb. Report SHA-256: c0faa943b45d5313c90fa9b03c71ea86b836af94896ebfae732b621a2e17b9dc.

2026-07-31: corrected development baseline frozen

The immutable public Space was probed on all 13 source-qualified development cells:

Measure Baseline
Exact sentence 0 / 13
Both required slots present 0 / 13
Required subject present 7 / 13
Required predicate present 1 / 13
Blank or source-copy fault 0

Examples make the failure mechanism concrete:

English Reviewed development form Public model
The child is coming towards the speaker. Mayu yanajimanha. mayu yanmanha
The child is running. Mayu jamarnimanha. Mayu yanmanha
The child is sitting. Mayu nyinamanha. Junda nyinarangamanha
The dog is coming towards the speaker. Duthu yanajimanha. Duthu yanmanha
I am going away. Ngatha yanmanha. Ngurragi yanmanha
I am sitting. Ngatha nyinamanha. Ngatha nyinarangamanha

The model often retains a plausible subject while collapsing the predicate to a familiar going-away or sitting-down frame. This is why the first v3 experiment targets controlled composition instead of adding undifferentiated epochs or thousands of unreviewed sentences.

Baseline contract SHA-256: 70f2e0a9e6533b6616c19cc763635ff9edb1a02669090801b464cd44f96e6c64. Report SHA-256: ef2e415f59f70e0e6c825c7e2b24060d5e07cdfda18209a614999053c1c1d19e.

2026-07-31: sealed RunPod screen completed and rejected for promotion

2026-07-31: fresh held-subject development matrix frozen

2026-07-31: next mixed-replay schedules prepared

The next causal comparison is B0 versus C2 versus T2:

Property C2 matched control T2 mixed treatment
Presentations 576 576
Optimizer updates 18 18
Composition presentations 288 288
Lexical presentations 288 288
Unique composition rows 36 36
Unique lexical rows 144 144
Source non-padding tokens 4,548 4,548
Target non-padding tokens 4,066 4,066
Total non-padding tokens 8,614 8,614

2026-08-01: lexical regression localized and static GPU kit passed

The new partition analyzer was run against the recovered B0, C0, and T1 predictions before designing any additional data. The 144-anchor population was defined to contain all 23 B0 to T1 losses, so its role is diagnostic and preventative rather than a random population estimate.

One-target partition B0 exact C0 exact T1 exact
All one-target prompts 1,076 / 1,179 1,072 / 1,179 1,061 / 1,179
Selected lexical anchors 144 / 144 140 / 144 121 / 144
Mandatory prior-regression anchors 23 / 23 19 / 23 0 / 23
Unanchored complement 932 / 1,035 932 / 1,035 940 / 1,035
Matched lexical controls 144 / 144 144 / 144 144 / 144

This changes the failure diagnosis. C0 did not reduce exact reconstruction anywhere in the 1,035-row unanchored complement; its four losses all belonged to the sensitivity set. T1 improved the unanchored complement by eight exact rows while losing all 23 mandatory anchors. The aggregate decline was therefore concentrated in a small continuation-sensitive population rather than distributed across the lexicon. That is evidence for interleaving direct replay of the damaged mappings. It is not evidence that the 23 words are a natural statistical class, because they were selected from the observed losses.

The T2 treatment now receives two presentations of each of 144 anchors while its composition dose is halved from 16 to eight presentations per unique composition row. C2 receives the same number of lexical and sentence presentations and is matched exactly on source, target, and total non-padding tokens. This directly tests whether targeted lexical replay preserves the sensitive mappings while retaining the sentence-composition signal.

Implementation and preflight changes:

The immutable preflight kit is experiments/kits/wbv-v3-mixed-replay-screen-preflight-20260801-a1. It contains 37 checksum-bound files and is 34 MB compressed. Static preflight reports 32 development rows, 3,188 regression rows, 576 presentations per arm, and 18 optimizer updates per arm. All 78 focused Wajarri tests pass, Ruff passes, every kit checksum passes, and the archive passes gzip and tar integrity checks.

Artifact SHA-256
Build contract 9ee5f3045c5610bd2e51886a093dca0da3e19e658435d1f1a1e22cfa6015b2b7
Immutable kit contract 68b311c1b9ea0fd4d65819f6167e62b2f49de276c5a7dda54fe9cf5e93c7f3b0
Kit checksum ledger c3724fc6f133b5f0b5af01aa3e6bab9e44b4264b8a1d89be7db80388b86c627a
Reproducible kit archive 99b0855a6b94fc27f07ed22b7c6618888766a8e70fedb53129ad6f274d5c1a51
B0 lexical partition report 1cbdb71029a4c96a13a0b967eea1c21d61d16a11b2918d2a1272df9f41ac1dab
C0 lexical partition report 2a1bfb65fc38dfa133fd8fb8ac5eac43a5fc2bb6b59db1936c25f0cd0af129cf
T1 lexical partition report 6cb034fe74ff8d33171134bd9a3ef4814d4c0f440f39684b15405a9d70ddf79a

The kit is intentionally not executable on paid compute. Its immutable contract contains paid_compute_authorized_by_user=false, and a tested bootstrap interlock exits before checking CUDA or creating an output directory. No RunPod pod is active. The next operational step, if paid execution is approved, is to reissue an authorized derivative of this exact preflight kit, verify that only the authorization/run identity changed, run one A40 seed, recover every result, and delete the worker immediately.

2026-08-01: C2/T2 mixed-replay A40 screen completed and rejected

The previously prepared comparison was authorized, executed as wbv-v3-mixed-replay-screen-runpod-20260801-a1-r2, recovered, and shut down. Both arms began from the immutable public Wajarri v2 adapter and received 576 presentations over 18 optimizer updates. The selected control was C2 step 18; the selected treatment was T2 step 15.

Fresh 32-cell development measure B0 C2 T2
Exact sentence 0 0 4
Both required slots present 0 0 4
Required subject present 12 12 12
Required predicate present 0 0 11
Mean chrF++ 35.95 36.93 54.42
Repeated-token faults 2 0 0

The treatment effect was real and progressive, rather than a single lucky checkpoint:

T2 update Exact Both slots Predicate present Mean chrF++ Repeated-token faults
3 1 1 2 38.78 2
6 2 2 4 42.42 2
9 3 3 8 49.02 1
12 3 3 9 51.62 0
15 4 4 11 54.42 0
18 4 4 11 53.96 0

The corresponding C2 checkpoints remained at zero exact and zero expected predicates throughout. Direct composition supervision therefore changes sentence behavior; ordinary matched continuation does not explain the gain.

The full regression result nevertheless rejects T2:

Full measure B0 C2 T2
All one-target lexical exact 1,076 / 1,179 1,074 / 1,179 1,067 / 1,179
Trained-anchor exact 144 / 144 141 / 144 129 / 144
Mandatory sensitive-anchor exact 23 / 23 20 / 23 8 / 23
Unanchored-complement exact 932 / 1,035 933 / 1,035 938 / 1,035
Controlled-synthetic exact 13 / 36 11 / 36 13 / 36
Controlled-synthetic chrF++ 70.90 71.06 71.12
Complex historical exact 0 / 5 0 / 5 0 / 5
Complex historical chrF++ 9.62 9.84 10.27
Fixed-utterance retention 55 / 55 55 / 55 55 / 55

T2 gained seven exact lexical rows, lost 16, and retained 1,060, for a net change of minus nine. It passed the narrow sentence-effect test but failed all-one-target, trained-anchor, mandatory-anchor, and retired-diagnostic noninferiority. Public-serving suites had zero mechanical faults. The formal status is NEGATIVE_OR_INCONCLUSIVE_INTERNAL_COMPOSITION_SCREEN; promotion is false. Neither selected adapter was uploaded or served, and the public Space remains on Wajarri v2.

Completed-run artifact SHA-256
C2 selected adapter dd65431ee49d53591d47a00f0ccc5f88a0ac48ab2ebd65e16c82057cdd37a78f
T2 selected adapter b31a785ac28456137a3059f092a4930a9ab7c256b348bd3c81dcfe2fc2dc4bc3
Result 6d5d33b3b2354c42d8c4994d04e784d7d1fd35810a3532d7907ce256347c7ef8
Result manifest 02cb61eff158b0aba8849cd23a87fe8f20f133d150cf8ad570ddbce14342f07f
Run-complete record 1b47965c99f6287eee9567143961a6d7d00102041fb9054b4472f4bcfa82a022
Recovery manifest 3e161119d5becf5c65d8a6605f5002d21e1b40a554fd5dd7dbd9a8ac37954f31
Post-hoc analysis report 496b3e9d08a867f579111145bb570300b66993d386a8139e5ec9aca04d2ae2b2

The A40 worker 1vgczur074a6k9 was deleted after recovery. A fresh runpodctl pod list --output json on 2026-08-01 returned [].

2026-08-01: qualitative failure localization

The lexical losses are not uniform. All 31 one-token anchor targets survived. Losses occurred in 3 of 35 two-token targets, 9 of 44 three-to-four-token targets, and 3 of 34 targets with five or more model tokens. Token length contributes to sensitivity, but it does not explain the wrong-known-word substitutions or derivational flattening.

All 15 T2 losses from the 144 baseline-exact anchors belong to the 23-row mandatory sensitivity set:

English prompt Governed target T2 output Source record
shoes jinabuga jinabu wbv-src-local-001279
small brown venomous snake jidu jirdu wbv-src-local-001356
climbing up or into garlbajunmanha garlbajimanha wbv-src-local-000318
feeling sorry gurninyimanha gurnimanha wbv-src-local-000536
spoiling walhiyimanha walhimanmanha wbv-src-local-001398
bush buuju bariya wbv-src-local-000266
playfully bunthanthu bunthanyimanha wbv-src-local-001099
boil gurraburda gurrgurda wbv-src-local-000214
becoming full ngumulimanha mimigimanha wbv-src-local-000605
swelling yurayimanha yurayimanmanha wbv-src-local-001462
spreading out garrayijunmanha garrimanmanha wbv-src-local-001401
straightening thubarnmanha thubarnimanha wbv-src-local-001432
making a pile of sand budirimanmanha budimanmanha wbv-src-local-001066
sitting something nyinajunmanha nyinamanha wbv-src-local-001314
doing in vain widigunmanha widimanha wbv-src-local-000460

Every one of these 15 source records has a decoded speaker-attributed word recording, but every form is still marked morphologicalAnalysisStatus=unanalysed and none has a governed sentence example in dictionary edition wajarri-dictionary-50words-attested-v1.2.0. It would therefore be unsound to invent sentence contexts or morphological paradigms for them merely to protect a benchmark score. The immediate protection mechanism is checkpoint-level lexical sentinels and direct replay. Sentence generation for these forms remains blocked until a source supplies their syntactic and morphological use.

The fresh composition errors are also structured:

This points to two separable problems: preservation of morphologically complex lexical forms during continuation, and controlled selection of source-qualified predicates and subjects inside a sentence. More repetitions of the current templates would conflate them.

2026-08-01: paired public-Space glossary probe

A deterministic builder derived 36 training and 32 development glossary-input variants from existing reviewed rows. The target side is byte-identical to the corresponding plain row: this adds 68 input variants and zero new Wajarri target sentences.

The public CPU Space was then probed sequentially with each plain development prompt and its glossary-conditioned partner. All 64 responses came from the expected immutable Wajarri v2 model identity; there were no retries, queue waits, blank outputs, or source copies. CPU float32 results are kept separate from the A40 BF16 experiment.

Paired 32-cell measure Plain v2 Glossary-conditioned v2 Net gain
Exact sentence 0 6 +6
Both required slots present 0 6 +6
Required subject present 12 23 +11
Required predicate present 0 9 +9

All six exact gains were sitting clauses: adolescent girl, aunt, cousin, dingo, grandmother, and red kangaroo. The same prompt structure still produced Ngaru nyinamanha for expected Nguurru nyinamanha and jarda nyinamanha for expected Juja nyinamanha. Glossary conditioning is therefore a useful retrieval intervention, not a solved translator. It improved the source slot and one predicate family without teaching the deictic-motion, running, or standing contrasts reliably.

The probe made 64 requests in 306.30 wall seconds and reported 227.885 model seconds. Its contract SHA-256 is 07abcd5d10cd895e5d4280f8e359b21695e4f43db8eae5c504404328e006e464; report SHA-256 is 5af517a2dcb9fe0150b3e31f231030430a5d390b49b920dc168392b7e1cee703; manifest SHA-256 is b71a4c516aeff0d0ea3fa000403a94335bede35c967f17a459a788d7309fbc5e. The glossary-fixture contract SHA-256 is 5c600ff7c9259626c9736c3b0eb49dfaa3d71d1de8b36ff1e9376252d6a3af06; its report SHA-256 is 4f81e8618164197c3bc0b38bbcfceee12ac301c10fb1d3e86d8458f08471102f. Both checksum ledgers pass.

New sentence requirements before the next training stages

The phrase “generate more synthetic sentences” now resolves into four gated stages. Counts below are unique English-Wajarri pairs, not optimizer presentations, prompt mirrors, paraphrase counts, or independent speaker attestations.

Stage New Wajarri target pairs What is required Gate before proceeding
A: checkpoint-sentinel rerun 0 Re-run the existing matched C2/T2 schedules while measuring all 144 baseline-exact anchors and all 23 mandatory anchors at every checkpoint; also measure plain and glossary development conditions An admissible checkpoint must retain all protected anchors and beat B0 and C2 on sentence composition
B: evidence acquisition 0 synthetic pairs Find source-backed usage and morphological analyses for the 15 sensitive forms above; adjudicate yanajimanha versus yanmanha, nyinamanha versus nyinarangamanha, and exact running/subject forms No form may enter sentence generation from an isolated gloss plus an unanalysed headword alone
C: first contrast commission 48-96 16-32 deictic-motion contrasts, 16-32 sitting-versus-sitting-down contrasts, 8-16 exact running contrasts, and 8-16 subject/orthography contrasts Every target must be licensed by an activated source rule and lexeme; split whole subject and morphology families before generation; reject if lexical sentinels or retained sentences regress
D: broader factorial expansion 80-160 Cross 20-40 vetted animate subjects with the four source-qualified predicates; glossary mirrors may be added as input variants but do not count as new target pairs Only after Stage C improves held-family development and passes full lexical, sentence, degeneration, and provenance gates

Standing should remain a lightly represented positive control rather than receive indiscriminate new volume. The previously designed 3,000-pair corpus is retained only as an expansion ceiling after Stages A-D establish that the intervention generalizes. There is currently no evidence-based reason to generate 20,000 Wajarri sentences.

For every issued pair, the repeatable language-program contract requires:

  1. exact source IDs for every lexical form and grammar rule;
  2. a declared construction family, morphology features, subject family, and variety status;
  3. complete-form targets rather than generated suffix strings;
  4. group-level train/development/test assignment before paraphrasing or deduplication;
  5. separate counts for attested, research-synthesis, glossary-input, and speaker-reviewed evidence;
  6. deterministic lint for source binding, slot completeness, contamination, duplicates, orthography, and forbidden extrapolation;
  7. a frozen model, tokenizer, decoder, row, token, update, and learning-rate contract before GPU use;
  8. per-checkpoint lexical sentinels so a later checkpoint cannot be selected after silently erasing known forms.

2026-08-01: checkpoint-sentinel preflight frozen

The next diagnostic driver now evaluates untouched B0 and every C2/T2 checkpoint at updates 3, 6, 9, 12, 15, and 18 on:

A checkpoint is admissible only if it has zero sentinel mechanical faults, loses no baseline-exact anchor, and retains all 23 mandatory anchors. Development score may choose only among admissible checkpoints. If none exists, the run emits NO_ADMISSIBLE_CHECKPOINT_DIAGNOSTIC_ONLY and preserves diagnostic adapters without pretending to have selected a candidate.

The immutable preflight-only kit is experiments/kits/wbv-v3-checkpoint-sentinel-preflight-20260801-a1. It contains 44 checksum-bound files plus the checksum ledger, 41,964,483 uncompressed bytes, and a 34,804,052-byte reproducible archive. Static preflight confirms 32 plain development rows, 32 glossary development rows, 3,188 full-regression rows, 576 presentations per arm, and 18 updates per arm.

Next-kit artifact SHA-256
Build contract 14ba16b81671fc05360c0060efc3da61b0077ec2107b946a8959566db0e8891b
Immutable kit contract 5b6090f9508e1efe543f78cba3ba161a49a4fd09d0aa38f19f6a94d11a8df2eb
Kit checksum ledger bd39a9ef20cebc96cddbfc28970c597f1dba0dff8ae41e3c3d89d6e0c066712a
Reproducible kit archive d74df70cb34b927a27bfc2c1bad84259550c66fde6c6b191491934f75092047d

All 97 focused Wajarri v3 unit tests pass, focused Ruff checks pass, the bootstrap passes bash -n, all 44 bound checksums pass, and both gzip and tar archive checks pass. A local bootstrap test verified every checksum and then refused with Paid GPU execution is not authorized by this immutable contract; it created no output directory. The contract remains paid_compute_authorized_by_user=false, and a final live RunPod check still reports no pods.

This section supersedes the earlier preflight’s next-step wording. The next paid action is not another broad epoch run and is not currently authorized. If separately authorized, it is one bounded A40 checkpoint-sentinel diagnostic from this exact kit, followed by immediate result recovery and worker deletion. Its outcome decides whether Stage C sentence commissioning is warranted.

2026-08-01: authorized checkpoint-sentinel A40 run completed and rejected

The user authorized the bounded next RunPod experiment. An authorized derivative was built without changing the frozen scientific design. Relative to the preflight contract, only creation time, run identity, authorization state, and the paid-compute Boolean changed; normalized remainder hashes matched before execution.

Three secure A40 allocations never reached nonzero uptime or exposed SSH and were deleted without running the workload: igklrbz482bypv, 6vaeouwg0enqxf, and 8fp4wszfenxwms. The successful worker was re80p6zctrxdtv, an A40 in EU-SE-1 started from the official runpod-torch-v280 template and exact image runpod/pytorch:1.0.2-cu1281-torch280-ubuntu2404. Admission verified an NVIDIA A40 with 49,140 MiB VRAM, PyTorch 2.8.0+cu128, CUDA 12.8 availability, more than 160 TB free in the provider workspace, and every one of the 44 kit checksums before model loading. The worker rate was USD 0.44/hour. It was retained for roughly 30 minutes including result packaging and transfer, implying approximately USD 0.22 before provider rounding.

The run evaluated untouched B0, every C2 and T2 checkpoint at updates 3, 6, 9, 12, 15, and 18, the 32-cell plain development matrix, the paired 32-cell glossary-input matrix, all 144 baseline-exact lexical sentinels, all 23 mandatory sensitive sentinels, and the 3,188-row full regression bundle.

Complete checkpoint census

Arm Update Admissible Plain exact Plain predicate Plain chrF++ Glossary exact Glossary predicate Sentinel exact Mandatory exact
C2 3 yes 0 / 32 0 / 32 35.83 6 / 32 10 / 32 144 / 144 23 / 23
C2 6 no 0 / 32 0 / 32 35.86 6 / 32 11 / 32 143 / 144 22 / 23
C2 9 no 0 / 32 0 / 32 36.75 4 / 32 10 / 32 142 / 144 21 / 23
C2 12 no 0 / 32 0 / 32 36.91 4 / 32 10 / 32 140 / 144 19 / 23
C2 15 no 0 / 32 0 / 32 36.91 5 / 32 11 / 32 141 / 144 20 / 23
C2 18 no 0 / 32 0 / 32 36.93 4 / 32 10 / 32 141 / 144 20 / 23
T2 3 no 1 / 32 2 / 32 38.78 7 / 32 12 / 32 141 / 144 20 / 23
T2 6 no 2 / 32 4 / 32 42.42 7 / 32 13 / 32 138 / 144 17 / 23
T2 9 no 3 / 32 8 / 32 49.02 7 / 32 13 / 32 135 / 144 14 / 23
T2 12 no 3 / 32 9 / 32 51.62 7 / 32 13 / 32 130 / 144 9 / 23
T2 15 no 4 / 32 11 / 32 54.42 8 / 32 15 / 32 130 / 144 9 / 23
T2 18 no 4 / 32 11 / 32 53.96 8 / 32 15 / 32 129 / 144 8 / 23

The C2 selector correctly chose update 3, the only admissible control checkpoint. No T2 checkpoint was admissible, so its update-3 adapter is explicitly retained under diagnostic-adapters/T2 with selection status NO_ADMISSIBLE_CHECKPOINT_DIAGNOSTIC_ONLY. The later four-exact treatment checkpoint was not allowed to conceal its 14 sentinel and 14 mandatory-anchor losses.

Selected-checkpoint full regression

Full measure B0 C2 step 3 T2 step 3 diagnostic
One-target lexical exact 1,076 / 1,179 1,076 / 1,179 1,072 / 1,179
Trained-anchor exact 144 / 144 144 / 144 140 / 144
Mandatory-anchor exact 23 / 23 23 / 23 19 / 23
Unanchored-complement exact 932 / 1,035 932 / 1,035 932 / 1,035
Controlled-synthetic exact 13 / 36 13 / 36 13 / 36
Controlled-synthetic chrF++ 70.90 72.67 70.95
Complex historical exact 0 / 5 0 / 5 0 / 5
Complex historical chrF++ 9.62 9.62 9.84
Fixed-utterance retention 55 / 55 55 / 55 55 / 55

Both continued adapters passed public-serving mechanical checks, one-target and unanchored tolerance, controlled-synthetic tolerance, and fixed-utterance retention. T2 failed trained-anchor and mandatory-anchor retention. Both C2 and T2 also failed retired definition-context diagnostic noninferiority: B0 already repeats winyjinyji for the green-cricket prompt, while both continuations additionally copy the English source telephone instead of producing governed wayangunda. Thus C2 is a useful safety-preserving control, not a release.

The treatment’s earliest protected losses were not random short words:

Prompt Governed form T2 step-3 output
bush buuju bariya
boil gurraburda gurrgurda
Sturt's Desert Pea marluguru marlba-marlba in the full 1,408-row lexical run
doing, in vain widigunmanha widimanha

The Sturt's Desert Pea row exposes an additional release blocker. The 144-row sentinel invocation returned correct marluguru, but the full lexical evaluator returned marlba-marlba from the same compact adapter weight SHA-256. The evaluation order and padding context differ even though the decoder is deterministic. This is an observed batch-shape invariance failure until reproduced and explained; all future release candidates must return the same normalized output for every gate row at batch sizes 1, 8, 16, and 64.

Qualitatively, the treatment learns a few cells while leaving the key contrast failures intact:

English Governed reference Public B0 T2 step 3 T2 step 15 diagnostic
The aunt is standing. Maraji garrimanha. Maraji garrayi-garrayi Maraji garrimanha. Maraji garrimanha.
The adolescent girl is coming towards the speaker. Warlugura yanajimanha. Muthura yanmanha. Muthura yanmanha. Muthura yanmanha.
The horse is sitting. Nguurru nyinamanha. Dundulyji nyinarangamanha. Dundulyji nyinarangamanha. Dundulyji nyinarangamanha.
The red kangaroo is standing. Marlu garrimanha. Burun jardu jarlgu nyinya. Burun jardu jarlgu nyinya. Jalgu garrimanha.

This is narrow matrix fitting, not broad sentence translation. At step 15 the four plain exact cells are aunt-standing, dingo-standing, grandmother-running, and grandmother-standing. Coming-towards remains 0 of 8 at every treatment checkpoint. Sitting remains confused with sitting-down in the plain condition. Glossary conditioning helps more than unconditioned generation, but even the most fitted checkpoint is only 8 of 32 exact on the consumed matrix.

Recovery and shutdown evidence

The remote result manifest bound 204 files. Every checksum passed on the worker, the deterministic result archive passed gzip integrity, its local SHA-256 matched the remote value, extraction succeeded on /mnt/donto-data, and all 204 manifest checksums passed again locally. Only compact selected/diagnostic adapters and complete evaluations were retained; optimizer state and unselected checkpoint weights were deleted under the run's deletion ledger.

Artifact SHA-256
Authorized build contract 3d48e09bdeccf228037b42848c8dd189778ca7da8f87ed335ad02fdf68ba97dc
Authorized immutable kit contract 868e992be46ecadb4756a638b4be23d6f01c0c70f8073b19e27841498f27e48a
Kit checksum ledger 258a50238e90e1992738ea4d541cd0472b0f38bf179a48523484e8541bd8ceef
Kit archive ee985a8a52dded9094410989294599aadacf0121122d95a63758c8918b0d2295
C2 compact adapter f78c4bbb129ff74406838187985e02037e2f70ca905ee264517d4003e03660b0
T2 diagnostic adapter 320ce95d175829204586de806fe5dc28112c37fb9d51d5e2c3d9c5e36ee9e155
Result 83206aaa848b4112639c9918190e42307c92df17a65204484041da3512116a75
Result manifest a4b909106c4e3798f94087b2d76caa932104c1b549e70303a572baafc904d53d
Run-complete record 54e3ebb5cc6485b64e7be2b15956b0b1b1f158ff3f3070051672f51bcd55b3fe
Recovered 1,615,084,259-byte archive 2e29604a2c6dea5cfbee84730f7f8bdc58eea42ce2a4dd1a55b0f892f7429d44

The canonical recovered directory is experiments/results/wbv-v3-checkpoint-sentinel-runpod-20260801-a1/. Worker re80p6zctrxdtv was deleted after local verification, and a fresh runpodctl pod list --output json returned [].

Hugging Face ruling and live state

No new Wajarri adapter was uploaded. The public model remains ajaxdavis/mobtranslate-wajarri-v2 at repository commit 357a7e5bde96990a3d5131bca12da6dacce98b3a, model version v2.0-context-guarded-nr4-s17-context-recovery-r16-s20-20260731. The shared Space ajaxdavis/alpha-v0-historic remains on CPU Basic at commit 950b261a7fe6304b94aa63d03b7f0126cc8bbf15 and advertises the same immutable Wajarri identity. A live model-bound request for The aunt is standing. returned Maraji garrayi-garrayi, not governed Maraji garrimanha.. This confirms that hosting works and sentence translation remains unsolved.

What remains before another model can be released

The next work is not another epoch run and not a 20,000-row blind expansion:

  1. Complete source-backed use and morphological analyses for the sensitive forms, beginning with the four update-3 full-regression losses and the earlier 15-row loss set.
  2. Commission 48-96 new governed target pairs: 16-32 coming-towards versus going-away contrasts, 16-32 sitting versus sitting-down contrasts, 8-16 running contrasts, and 8-16 subject/orthography contrasts. Glossary-input mirrors remain input variants and do not count as new Wajarri targets.
  3. Keep the public v2 lexical adapter immutable. Build a separate <translate> sentence adapter under the same frozen base and tokenizer, while dictionary lookup and <lexeme> reconstruction continue through deterministic lookup or v2. One Hugging Face Space can route both adapters under one API without asking one adapter to optimize incompatible tasks.
  4. Train the sentence adapter only after family-level splits are frozen. Compare plain and retrieved-glossary inputs at fixed token exposure, checkpoint every one to two updates, and include batch-size invariance in every checkpoint gate.
  5. Use the entire 1,179-row one-target lexical suite as a diagnostic and preservation/distillation surface, not only 144 matched sentinels. A sentence adapter may be task-routed, but sentence outputs must still preserve required governed lexical slots.
  6. Require a held-family sentence gain, zero deployment faults, stable outputs across batch sizes, synthetic and fixed-utterance noninferiority, and source-backed natural review before opening a final test or uploading a new public adapter.
  7. Only after the first 48-96-pair intervention generalizes may the program commission the 80-160 broader factorial set. The 3,000-pair corpus remains a later ceiling; there is no evidence for generating 20,000 Wajarri pairs now.

The shortest technically defensible path to a useful Hugging Face system is therefore a task-routed lexical-plus-sentence model service backed by new source-governed contrasts. The current experiment proved that additional updates on the existing matrix cannot produce that system safely.

2026-08-02: copy-token screen recovered; subject-slot architecture issued

Subsequent controlled screens did not justify expanding unsupported Wajarri morphology. The strongest sentence continuation, D6, improved the bounded composition matrix but still rewrote copied dictionary forms. T7 then made <copy> a dedicated single tokenizer token and task control. That intervention reduced neither the underlying one- and two-character rewrites nor the lexical regression enough to pass.

Measure Untouched B0 D6 T7 selected update 480
Plain composition exact 0 / 11 9 / 11 9 / 11
Inline held-subject exact not selected 21 / 24 22 / 24
Neutral single-copy exact not selected 48 / 64 48 / 64
Neutral dual-copy exact not selected 24 / 64 25 / 64
Fixed-utterance retention 55 / 55 54 / 55 54 / 55
Controlled-synthetic exact 13 / 36 23 / 36 23 / 36
Controlled-synthetic chrF++ 70.9009 75.7673 75.0276
Direct lexical exact 1,099 / 1,408 not selected 838 / 1,408
Definition-context lexical exact 877 / 1,684 not selected 590 / 1,684

The T7 adapter is an internal starting point, not a release. Its selected weight SHA-256 is 02e04f84ae261bfb7fcfca4a05e221b0e2982036496ef9a8f311a526c7446d9c. It had 13 direct-lexical and 47 context-lexical mechanical faults. The model remained 0 of 5 exact on the historical diagnostic. No T7 weight was uploaded, registered, mounted, or routed to the homepage.

Evidence frontier re-audited

The primary-source audit found no machine-readable Wajarri sentence collection that could responsibly turn the unsupported -manha inventory into hundreds of training clauses:

The complete-predicate frontier therefore remains six source-bound surfaces: yanajimanha, yanmanha, jamarnimanha, nyinamanha, nyinarangamanha, and garrimanha. The program explicitly rejected suffix-based generation over the remaining unanalysed headwords. More row count would not repair missing linguistic evidence.

Deterministic subject-slot intervention

The next intervention removes the part T7 could not perform reliably. The model no longer regenerates a dictionary subject. It emits one literal slot plus one complete governed predicate:

model output:       <copy> jamarnimanha.
dictionary binding: <copy> = marruwa
rendered output:    marruwa jamarnimanha.

The renderer fails unless there is exactly one <copy> marker in the first output position. It performs one deterministic replacement and does not alter the predicate. Two paired arms isolate how the slot is declared:

Arm Model-visible source Raw target
M8 masked source <translate> The <copy> is running. <copy> jamarnimanha.
D8 declared source <translate> The bilby is running. <glossary> bilby = <copy> <copy> jamarnimanha.

The commission contains 103 training parent pairs, 11 composition-development parents, and 24 held-subject-development parents. It creates 206 training representations and 70 slot-development representations but zero new Wajarri sentences. M8 collapses to six unique model-visible pairs, one per complete predicate; D8 preserves 103 distinct English-subject inputs. No sealed-test row was read.

Each arm has exactly 7,680 scheduled presentations and 480 optimizer updates. Every update contains four fixed-utterance retention rows, four plain sentence rows, four inline sentence rows, and four subject-slot rows. Non-slot presentation positions are paired between arms. Token exposure is reported rather than assumed equal:

Arm Source tokens Target tokens Total non-padding tokens
M8 97,597 65,884 163,481
D8 113,290 65,880 179,170

The 140-row consumed development screen has 11/11/24/24 ordinary plain/inline composition and held-subject rows plus 11 masked composition, 11 declared composition, 24 masked held-subject, and 24 declared held-subject slot rows. Frozen checkpoints are updates 20, 40, 60, 80, 120, 160, 240, 320, 400, and 480. A candidate must render all 11 matched composition and all 24 matched held rows exactly, retain at least 9/11 ordinary plain composition and 22/24 ordinary inline held rows, preserve the T7 full-regression tolerances, have zero deployment faults, and return identical raw and rendered outputs at batch sizes 1 and 16.

The evaluation path was hardened before launch. The prior full-regression decoder used skip_special_tokens=True, which could erase an illegal generated <copy> token before scoring. The new evaluator preserves all generated task controls, records unresolved_task_token as a mechanical fault, separately scores raw template exactness and deterministic rendered exactness, and never conflates either with ordinary sentence exactness. Fifteen focused local tests pass.

Immutable A40 launch state

The corrected authorized kit is experiments/kits/wbv-v3-subject-slot-screen-runpod-20260802-a2/. Every bound file and the complete kit checksum ledger passed locally and again on the worker. Static preflight confirmed two arms, 7,680 presentations per arm, 480 updates, ten adapter snapshots, 140 development rows, and no sealed test.

Launch artifact SHA-256
Authorized build contract 8eac140ed0a402b5ba60db52627cb6b01d148a09959237edfec97c547db73336
Immutable kit contract b11057e7101c24eb69b09c2770d35954cbd3413d180ee9fd7dff789cd24edf98
Kit checksum ledger 85edae8a571d6d440d9f693873eeee3fd3d7b723f481a87135dfdf70ec4796c7
T7 initial adapter 02e04f84ae261bfb7fcfca4a05e221b0e2982036496ef9a8f311a526c7446d9c

The first worker, 03bfrytfzbtrx3, reached D8 update 175 before an operator audit found two selector defects inherited from the older experiment runner: fixed-utterance equality would reject a genuine 54-to-55 improvement, and cross-representation diagnostic faults were incorrectly included in the primary arm fault scope. The worker was stopped and deleted before checkpoint selection. Its partial weights are not retained and it cannot produce a model claim.

Two new tests now prove that retention improvements satisfy noninferiority and that primary mechanical selection is scoped to the four ordinary endpoints plus the arm's two matched slot endpoints. Cross-representation exactness remains a secondary diagnostic tie-break, while batch-1/batch-16 invariance remains global and hard. All 17 focused tests pass.

Corrected worker 5ge03ph6aewrki started at USD 0.44/hour on a secure Swedish A40. It has a hard provider-side termination deadline of 2026-08-02 09:51:35 UTC. Every a2 kit checksum passed locally and remotely before execution. No result or model-quality claim has yet been made. The worker will be deleted only after the result archive is recovered, independently hashed, extracted on /mnt/donto-data, and every result-manifest file verifies locally.

2026-08-02: subject-slot screen rejected, decoder-prefix mechanism discovered

The corrected paired screen completed at 2026-08-02 06:29:48 UTC. Under the preregistered greedy decoder, neither D8 nor M8 emitted <copy> once on any matched slot row at any of ten checkpoints. The result is therefore NEGATIVE_OR_INCONCLUSIVE_SUBJECT_SLOT_SCREEN; no adapter is authorized for publication or serving by that run.

Selected result T7 baseline D8 step 80 M8 step 80
Matched rendered slot exact not applicable 0 / 35 0 / 35
Valid generated slot marker not applicable 0 / 35 0 / 35
Plain composition exact 9 / 11 9 / 11 9 / 11
Inline held-subject exact 22 / 24 20 / 24 21 / 24
Batch-1/batch-16 mismatches over 140 rows baseline diagnostic 8 9
Direct lexical exact 838 / 1,408 753 / 1,408 815 / 1,408
Definition-context lexical exact 590 / 1,684 523 / 1,684 593 / 1,684
Fixed-utterance retention 54 / 55 52 / 55 54 / 55
Controlled-synthetic exact 23 / 36 21 / 36 21 / 36
Controlled-synthetic chrF++ 75.0276 76.6330 74.9159
Historical exact 0 / 5 0 / 5 0 / 5

The result does not support more synthetic Wajarri volume. It isolates an interface failure: the complete source-bound predicate was often present, but the added special token was never the greedy argmax. The M8 training audit proves this was not a disconnected embedding row. <copy> received nonzero gradient on all 1,920 backward calls; its tied encoder, decoder, shared-input, and output-head row changed. D8 is additionally rejected for declared-subject interference, retention loss, and batch instability.

Recovered a2 artifact SHA-256
RESULT.json 9056c16b86d1262cb4c21d738a3fdfd1ec61f678d9f4e3a151a8ffa32aa2a633
RUN-COMPLETE.json dbdbcd044b099cecb6ddb1eac389a0e3c7d42007722daec82b0f71776e2da750
RESULT-MANIFEST.json af90e7a5077bb5fa8122317a323015890b68190595450751f1fd631767651276
Selected D8 adapter 18be2f62f36aea65d2f189c7943d20edb7f757599c4264871f74fd3912dc2d92
Selected M8 adapter b924b5cdc2fcd2194a0d97a6e278d4ca075d069603a538c9fba0c26469294679
Recovered 70,996,207-byte archive a351cb81629e81dcc0da6aa4bae6f59f30a3e19dc13c4bf2280ab4041721aa82

The remote manifest bound 142 files. All 142 passed on the worker, the archive hash matched after transfer, gzip integrity passed, extraction completed on /mnt/donto-data, and all 142 files passed again locally. Worker 5ge03ph6aewrki was then deleted. The subsequent pod census contained only unrelated Alpha worker wtupxv15debnvh.

Post-hoc forced decoder-prefix probe

Because <copy> is interface metadata rather than Wajarri, the next diagnostic seeded the decoder with the exact prefix [decoder_start=2, wbv_Latn=256204, <copy>=256208] and allowed M8 to generate the complete predicate. Two attempted invocations produced no model result and are not counted: the first omitted the evaluator's token-extension helper; the second used forced_decoder_ids, which Transformers 4.48 rejects in favour of decoder_input_ids. The supported decoder-prefix invocation then completed.

On only the route M8 was trained to serve, the result is exact:

Forced-prefix M8 endpoint Batch 1 Batch 16
Masked composition 11 / 11 11 / 11
Masked held-subject 24 / 24 24 / 24
Combined raw template exact 35 / 35 35 / 35
Combined rendered clause exact 35 / 35 35 / 35
Marker, position, and predicate correct 35 / 35 35 / 35
Mechanical faults 0 0

The complete 140-row diagnostic had ten batch mismatches, but none belonged to the matched masked route. All 35 intended route outputs were identical across batch sizes. Declared-subject and ordinary rows remain invalid under this decoder and are not authorized routes. Representative exact compositions include:

<translate> The <copy> is running.
<copy> jamarnimanha.
dictionary binding marruwa -> marruwa jamarnimanha.

<translate> The <copy> is sitting down.
<copy> nyinarangamanha.
dictionary binding miginy -> miginy nyinarangamanha.

The probe archive is 48,917 bytes with SHA-256 8936ac4fdf9782cefcfef2df0cbfa6958ad92ccf91086ec32c53c8e1a386f297. Its six files, including exact evaluator code, all pass the included checksum ledger locally. This is a post-hoc architecture discovery, not release evidence by itself.

Frozen three-seed confirmation now running

The confirmation reruns M8 from T7 for seeds 17, 42, and 73. Seed 17 is freshly trained; its post-hoc adapter is not reused as confirmation. Every seed receives the same 7,680-presentation schedule, 480 updates, token exposure, optimizer trajectory, and ten checkpoint positions. Checkpoint evaluation is restricted to the 11 masked composition and 24 masked held-subject rows under the supported decoder prefix. Selection chooses the earliest 35/35, zero-fault checkpoint. The selected adapter is then checked at batch sizes 1 and 16 and against the consumed full regression suites.

A seed passes only if all 35 raw and rendered outputs are exact, all controls and predicates are present, route faults and batch mismatches are zero, fixed-utterance retention is at least T7, synthetic exact loses no more than two rows, synthetic chrF++ loses no more than two points, and no unresolved task token appears. Even a three-seed pass authorizes only a fail-closed controlled route over the six governed predicates. It does not authorize free-form translation.

Confirmation artifact SHA-256
Authorized build contract 50e1d61f6116ba0626f4317e9364c4607cd24af5296689e0e866d9e50246818b
Immutable kit contract a9fcfa3584d12a1da5d10056d3bd7bcda6cd4823eda0dc4fab0b9f00da591521
Kit checksum ledger ed3dd087591dc03c51870c7e5d58e6d4b8172d2d9c162c4271f1cc0784d0b5fd
Reproducible 35,453,128-byte kit archive 4501c0a98e1a9497dacaa75d98e21e42ed4f59053466e361b64a1d9862c1b9f0

All 34 kit files pass locally and remotely; static preflight and 17 focused local tests pass. An initial worker, d75nut7548tuec, exposed only a 20 GB filesystem and was deleted before upload or training because it failed the 30 GiB reserve. Replacement worker 77tes61jw5a024 has an explicit 50 GB mounted workspace, a secure Swedish A40, a USD 0.44/hour rate, and provider auto-termination at 09:00 UTC. The unrelated Alpha worker is not part of this run.

2026-08-02: first confirmation invalidated by a seed audit

The first confirmation run, wbv-v3-forced-slot-confirmation-runpod-20260802-a1, initially reported the same positive route result for declared seeds 17, 42, and 73. Artifact recovery and evaluation were valid, but a post-run trainer audit found that the run was not a valid multi-seed confirmation.

The command-line seed was written into each manifest, but the constructed Seq2SeqTrainingArguments did not receive either seed or data_seed. Transformers therefore reset all three Trainer instances to its default seed 42. The three selected update-40 adapters were byte-identical:

declared seed 17 -> d59a4a276d4968e6d4e453dd534ebca6edfaacf936e1bd97cf7c3eb4be1c8801
declared seed 42 -> d59a4a276d4968e6d4e453dd534ebca6edfaacf936e1bd97cf7c3eb4be1c8801
declared seed 73 -> d59a4a276d4968e6d4e453dd534ebca6edfaacf936e1bd97cf7c3eb4be1c8801

That run proves deterministic repeatability under one effective stochastic state, not robustness across three independent seeds. Its adjudicated status is INVALID_AS_MULTI_SEED_CONFIRMATION; it never authorized a release. The complete invalid-run evidence was retained rather than rewritten:

Invalid a1 artifact SHA-256
RESULT.json 50535a692d4132a81c68b645db1074b519ed0ee5dbc4907a65fe5c71613c33f0
RUN-COMPLETE.json 33407cd0a06579d2a69470ab44440f6c8c021f52deffa603e4b4106459c9f8f7
RESULT-MANIFEST.json 2640d0067a7c88b10b38cacbeb42c575265c10852558d9d2397c56718d2008be
Recovered 105,094,918-byte archive 36a43404c482e06e4703894fa5ce83c50b3e6a1162223bf8cd5d2d82649e33cb

All 200 a1 manifest files still verify locally. The corrective implementation binds both Trainer seed fields to the declared run seed, records the runtime values, fails on a mismatch, and requires three distinct selected adapter hashes. The replacement a2 kit contract SHA-256 is 83788fad2533713c61d6d12e9d212a7c5bfe60dbba6a29efd6fc8ff016791e14; its archive SHA-256 is 71ae31582f1bfe1f726d415db937ebd393f335f9d029db6673130429a0dfa4a8.

2026-08-02: corrected independent confirmation passed the controlled route

Replacement run wbv-v3-forced-slot-confirmation-runpod-20260802-a2 retrained seeds 17, 42, and 73 independently from the same T7 start. Every seed used the same 7,680-presentation M8 schedule, 480 maximum updates, checkpoint positions, optimizer contract, six masked model-visible pairs, and 35-row consumed route screen. The only intended stochastic change was the bound seed and data seed.

The selector chose update 40 for every seed because it was the earliest checkpoint satisfying all route gates. Each adapter generated the exact raw template and exact rendered clause for all 35 rows at batch sizes 1 and 16. Each had zero route faults and identical outputs between batch shapes. The three weight hashes are distinct, proving the stochastic states were no longer collapsed.

Confirmation gate Seed 17 Seed 42 Seed 73
Trainer seed bound to declared seed pass pass pass
Trainer data seed bound to declared seed pass pass pass
Distinct selected adapter hash pass pass pass
Raw slot template exact 35 / 35 35 / 35 35 / 35
Rendered clause exact 35 / 35 35 / 35 35 / 35
Required predicate present 35 / 35 35 / 35 35 / 35
Slot count and first position valid 35 / 35 35 / 35 35 / 35
Mechanical faults 0 0 0
Batch-1/batch-16 mismatches 0 0 0
Fixed-utterance retention 54 / 55 54 / 55 54 / 55
Controlled-synthetic exact 22 / 36 22 / 36 22 / 36
Controlled-synthetic chrF++ 75.4246 75.3028 75.1763
Historical exact 0 / 5 0 / 5 0 / 5

Seed 17 is the released adapter. Its selected weight SHA-256 is a4a91d4e6a6ba49509a8ab432c0674686c96b4611f58db8b63089f26707ed3d6. Seed 42 is d59a4a276d4968e6d4e453dd534ebca6edfaacf936e1bd97cf7c3eb4be1c8801; seed 73 is 8574727bbf9cfb3e5032e9d922d7af61d6eb62f4b6b0976d6fc06b18b7dd82f7.

The route result does not erase the negative evidence. All three adapters remain 0/5 exact on the complex historical diagnostic, and direct one-target lexical reconstruction is below public v2. The positive claim is therefore restricted to exact template generation under the forced control prefix and deterministic subject rendering. Dictionary lookup remains the product authority for known words.

Corrected a2 artifact SHA-256
RESULT.json 6abbad036a146c3a15695f1c0ca31e7427f20f2f4ad7a712350cc261fac92fb1
RUN-COMPLETE.json a76fb09f7e8f46200536fbd1164ec0c7a57c6a18ba158fecdb6914c6ff02895f
RESULT-MANIFEST.json 2f7f5d2d97827c7641c035f64e3fce99a660e3c8f4ec223dee4af8606a742626
Deterministic recovered archive c445e98201cd4dbc484e74a27123aef4de3daeb12393998f8ce398fc9702b700

The a2 manifest binds 200 files. All 200 passed on the worker, archive transfer and gzip integrity passed, the local archive hash matched, extraction succeeded on /mnt/donto-data, and all 200 files passed again locally.

2026-08-02: CPU float32 parity passed

The first automated CPU invocation stopped before model loading because the standalone evaluator package omitted nllb_runtime_token_extension.py. This was a packaging failure, not a model failure, and no parity claim was made. The launcher was corrected to require and checksum the helper before execution. Its SHA-256 is 1c9156464016efc43edf82df6fbc0afa6ffc590133322af6dd9e3c29fc89c695; the runtime helper SHA-256 is cf1637bb17a358391770cb307e326151f63a85a9b2c2f083b531cce5d6aa5e2f.

The corrected run loaded the exact seed-17 adapter on CPU float32 and compared it with the A40 bfloat16 evidence. All 35 rows matched exactly at batch sizes 1 and 16. The normalized prediction SHA-256 was the same in all four conditions:

b17ab8d44ccaaa8dba962ecf46486d731d7aaf8d4c6a7adeed430044612cf25f

Every CPU identity, row-set, raw-template, rendered-clause, predicate, slot-shape, fault, and batch-invariance gate passed. The parity report SHA-256 is 9758ec536ce5500f84f544d03ea67cd7b79edbd9b69d361b1bb061de2be9352f; its result manifest is 8a23898a6d98f703408ab41930e636e69a24f93780ea02231f9e2504e6dbf5b1; the deterministic parity archive is 317fbad1d01547ab5c178ab5f5844a8fbe11f47d2dd33a807da8f23eb1d9daf8. All six parity-manifest files verify locally.

2026-08-02: worker recovery and deletion completed

Wajarri worker 77tes61jw5a024 was kept only through result recovery, local archive hashing, extraction, 200-file result verification, and CPU parity completion. It was then deleted. A fresh provider census proved that worker absent and left unrelated Alpha worker wtupxv15debnvh untouched.

Shutdown evidence SHA-256
Worker record before deletion b1b0c3da5b9bbdd635fee8ea5f6af23f94a80ebb2f7ef5245fa3e2a1f0f0ae4f
Provider census after deletion 44aefc19b19bc35d7f70a51749af394862fe939253a7c5ad2970acd65b00b449
Post-provider deletion ledger 91483094471a24e7abfc46ca2d7fb35226566348d7530e2adf30e46796090c4d

The manifest-bound DELETION-LEDGER.json inside the confirmation output records training-time cleanup and is distinct from the later provider-worker ledger. During release preparation, an attempted overwrite of that manifest-bound file was rejected by the checksum builder. The original was restored from the independently verified archive, all 200 manifest files were reverified, and the provider deletion record remained a separate artifact. This is why the release contains both ledgers under different names.

2026-08-02: immutable Hugging Face dataset published

The controlled dataset release is public at ajaxdavis/mobtranslate-wajarri-synthetic-corpus-v1.

Dataset identity Value
Commit 302f3a93eb255a2fc70692b7c429ba5ee8300687
Immutable tag v3.0.0-controlled-subject-slot-s17-step40-20260802
Files 59 total; 58 checksum-manifest rows
Training parent pairs 103
Unique masked model-visible pairs 6
Scheduled presentations 7,680
Newly invented Wajarri target sentences 0
Sealed-test rows read during release construction 0
Anonymous publication verification 59/59 present; 58/58 checksums exact
Publication verification SHA-256 a590e514cf36a50a6b62566e1dd4d2907d70e9d79da41e1a4fe280d90347db6a

The release preserves source records, schedules, model-visible masked pairs, development evidence, checksums, and the invalid-a1 adjudication. Repeated schedule presentations are not described as unique sentences, and glossary or masked input variants are not counted as new Wajarri targets.

2026-08-02: immutable Hugging Face model published

The controlled adapter is public at ajaxdavis/mobtranslate-wajarri-v3-controlled.

Model identity Value
Model ID mobtranslate-wajarri-v3-controlled
Version v3.0.0-controlled-subject-slot-s17-step40-20260802
Commit 1a26382648e294f2c102102ef914c7b0c86bec20
Immutable tag v3.0.0-controlled-subject-slot-s17-step40-20260802
Base ajaxdavis/mobtranslate-wajarri-v1 at ddae6103913b7cab299d687bee2dc7904fd59459
Adapter SHA-256 a4a91d4e6a6ba49509a8ab432c0674686c96b4611f58db8b63089f26707ed3d6
Release files 31 checksum-bound files plus repository metadata
Release record SHA-256 589d6a8a8b1de0b2c8a64c89abe79c453dd75c84d849179dcedd0db56572a809
Publication verification SHA-256 4f9d35aff3fe14c16c590161b364f785c8821a2501b0c26c2f619fd040eec9a7

Anonymous downloads verified every published file byte-for-byte. Both the branch commit and immutable tag resolve to the same release content. The repository contains the adapter, tokenizer/runtime extension files, frozen templates, GPU evidence, CPU parity evidence, training provenance, invalid-a1 adjudication, worker-deletion evidence, checksums, requirements, and a complete hosting guide.

The release builder exposed and fixed one filesystem trap before publication. adapter_config.json was initially hardlinked from the recovered result; rewriting its portable base reference would therefore have mutated the source artifact. The builder now copies that file before editing and a test proves the source remains byte-identical. The source recovered adapter and all confirmation checksums remain unchanged.

The NLLB-derived model follows the upstream CC BY-NC 4.0 constraint. Application code licensing does not override model or data terms.

2026-08-02: shared Hugging Face Space deployed and tested

The shared adapter service ajaxdavis/alpha-v0-historic is deployed at commit 03b6d4ed9c9ffb8ac2e55676447084af2c7aa5c8. Wajarri runs on CPU float32 under the existing bounded on-demand adapter cache. It is not mounted on the MobTranslate host and does not require a continuously running RunPod GPU.

The Wajarri service advertises only task subject_slot; generic Wajarri translate and lexeme tasks are rejected. Startup binds the exact model repository, version, base revision, <copy> token ID 256208, target-language token ID 256204, runtime special-token initialization, six templates, and checksums. Twenty-nine Space tests pass.

Live English construction Bound subject Exact Space output
coming towards the speaker marruwa marruwa yanajimanha.
going away from the speaker marruwa marruwa yanmanha.
running marruwa marruwa jamarnimanha.
sitting down marruwa marruwa nyinarangamanha.
sitting marruwa marruwa nyinamanha.
standing marruwa marruwa garrimanha.

All six live requests returned HTTP 200 and exact output. An unsupported construction, a missing dictionary binding, and generic Wajarri /v1/translate each returned HTTP 422 invalid_request. The Space had zero failed Wajarri model requests in final acceptance. The live acceptance report SHA-256 is fabf8453b7192c62867e65cebd11af6924321e07e93cf3b4bc7dc577e4596571.

The bounded cache may unload Wajarri when the two warm adapters occupy memory. In that state /health reports Wajarri as available, not failed; the next uncached Wajarri request reloads the exact pinned adapter. MobTranslate's PostgreSQL result cache prevents repeated text from repeatedly paying that load cost.

2026-08-02: production homepage route accepted

Production web release 20260802T091947Z-72668fe89dc5 deployed through the immutable release script. The full web suite passed 775 tests across 93 files, focused Wajarri routing passed 14/14 tests, TypeScript passed, the production build completed under the service environment, the candidate smoke test passed, and runtime checksums remained unchanged before and after public smoke testing.

The homepage route is abstracted through the same multi-language hybrid registry as Kuku Yalanji and Mi'kmaq. Wajarri adds a controlled task contract rather than language-specific endpoint code. The registry binds model ID, model version, dictionary code wbv, task contract, six construction IDs, templates, and the immutable dataset revision.

Supported live request

{
  "text": "The bilby is running.",
  "mode": "translate",
  "stage": "draft",
  "direction": "to_language"
}

The public endpoint POST https://mobtranslate.com/api/translate/wbv returned HTTP 200 and exact marruwa jamarnimanha. through route huggingface_draft. The response identified model mobtranslate-wajarri-v3-controlled, the exact v3 version, contract wajarri-subject-slot-v3-s17-step40-20260802, construction running, model template <copy> jamarnimanha., and dictionary headword marruwa.

The first request took 7.726 seconds and wrote a model-bound PostgreSQL draft cache row. The repeat took 90 ms and reported draft: hit. The complete source-check request preserved the translation exactly, returned route huggingface_grammar_review, and exposed the supporting dictionary records and scope limit under progressive disclosure. Its first request took 4.278 seconds; the repeat took 85 ms with draft, evidence, review, and resolver all reporting cache hits.

Unsupported live request

The bilby is flying. is outside the six-construction contract. It did not invoke the v3 adapter. The public endpoint returned through dictionary_prompt using gpt-5.4-mini, then cached that result separately. The first request took 2.800 seconds and the repeat took 72 ms. This proves that one model's narrow pass does not authorize unsupported Wajarri sentence generation.

Cache identity

The controlled draft cache key binds:

language
HMAC source fingerprint and source length
dictionary revision
model ID and immutable version
draft contract
construction ID
model template
dictionary bindings fingerprint

Evidence, review, and resolution add the evidence fingerprint, reviewer version, and resolver version. The live database contains separate ready rows for hybrid_hf_controlled_draft, hybrid_controlled_dictionary_evidence, hybrid_controlled_llm_review, and hybrid_controlled_resolved_translation. Unsupported prompt output is stored under separate stage dictionary_prompt_translation. A dictionary update, model update, task-contract change, construction change, binding change, reviewer change, or resolver change produces a new identity rather than reusing stale output.

Browser and layout evidence

Real Chromium acceptance selected Wajarri on the public homepage, entered The bilby is running., and observed marruwa jamarnimanha.. The interface shows the translation first. “Why this result?” then reveals the dictionary records, automated source check, limits, model identity, and cache state. The text is ordinary public-facing language rather than training-pipeline jargon.

Browser evidence Viewport Result SHA-256
Desktop first viewport 1440x1000 exact result visible; no horizontal overflow or overlap 687b1cc8b05212667ad59d5ab3f828c5e93f5a1e5332e64abfc52175faec196c
Mobile full page 390x844 exact result present after normal scroll; no horizontal overflow 1eb53e4fd772f89d3f36dd7f48c0323a2f16a318a11b79ea0c26d8bef3a45bb0
Mobile expanded evidence 390x844 progressive disclosure opens and remains readable b5c0c9b44299a98100e5c4523d9ece88729332880f58f919e98ed695c81b269e

The machine-readable homepage and public-registry acceptance record is experiments/results/wbv-v3-controlled-homepage-live-acceptance-20260802.json, SHA-256 7bf05e5b78ae1c05a57589aeb5a1ee320c1cc208af8a1f7fcae563d6f8516b54.

2026-08-02: public model registry reconciled

The final cross-system audit found one publication inconsistency after the model, dataset, Space, and homepage route were already live. GET /api/v1/models listed only Kuku Yalanji and Mi'kmaq, while GET /api/v1/models/mobtranslate-wajarri-v3-controlled returned HTTP 404. This did not break homepage translation, but it meant a third-party host could not discover the released Wajarri adapter through MobTranslate's model-distribution API.

The fix added Wajarri to the existing generic registry rather than creating a Wajarri-specific API. The entry binds the exact controlled task, immutable version, base revision, adapter hash, bounded metrics, noncommercial terms, model and dataset repositories, checksum ledger, CPU parity report, Space endpoint, and this work log. It states free_form_translation_authorized: NO. Public serialization strips the source-only localPath, and a focused test fails if any /mnt/ path becomes visible.

The first registry deployment attempt stopped before cutover because an unrelated one-hour spaced-repetition assertion read the live clock twice and occasionally observed 59 minutes 59 milliseconds. The test now pins time with Vitest's fake clock, matching the deterministic seven-day boundary test already in the suite. Its file passes 83/83 tests. Release 20260802T100136Z-72668fe89dc5 then exposed the correct API, but real-browser inspection found that the catalog rendered the custom status published-controlled as both “published controlled” and “Not published.” A shared status predicate was added so controlled publication remains a distinct state but is displayed and selected as published. Release 20260802T101815Z-72668fe89dc5 fixed that contradiction. Its mobile check then exposed clipped immutable base and dataset IDs, so the model card gained flex shrinking and overflow-wrap:anywhere for long identities.

The final immutable deployment passed all gates and published production release 20260802T103148Z-72668fe89dc5 with source SHA-256 e434ad550ee20b438939dcdf4cbbeca569dff8bb0fccca8c1a0e176a6330790f. Release integrity covers 8,640 files and 37 symlinks. The service is active and /api/health reports that exact release.

Final registry acceptance Result
Catalog GET /api/v1/models HTTP 200; model count 3; Wajarri present
Wajarri model endpoint HTTP 200
versions/latest HTTP 200; resolves exact v3 version
Exact immutable-version endpoint HTTP 200
Published status published-controlled
Task controlled_subject_slot_translation
Model downloads 2: immutable repository and adapter weights
Dataset downloads 1
Documentation downloads 1
Available artifact links 8 / 8 returned HTTP 200
Private path leaks 0
Model-distribution tests 8 / 8 passed
Full web suite 777 / 777 passed across 93 files
TypeScript, release-integrity, build, candidate smoke pass
Catalog publication label Published: controlled; contradictory Not published absent
Mobile base and dataset identity wrapping full text retained; client width equals scroll width
Catalog horizontal overflow none at 1440x1000 or 390x844
Immutable model-version page horizontal overflow none at 390x844

The registry source SHA-256 is 79887d73de776d53ea1c8a728083cf4d2df2187f55be58f29e7eb2a36e4f3c20; its focused test source SHA-256 is 4aa1355039700f16f4723fb8abe84e9a42347fe830d97857757c05b09363b4fc; the shared publication-status helper SHA-256 is dfd0c6304f081722eccd4498cfffa5e8f2203e9d2b146f097608bc8c7c3f9ac7; the final catalog-page source SHA-256 is 183de1561b632a5654da80cc8c90e670f7505c3e3efe33cba653497e329239e1. Final live translation smoke repeated the bounded route: The bilby is running. returned exact marruwa jamarnimanha. through huggingface_draft; complete review preserved the exact translation; unsupported The bilby is flying. remained on dictionary_prompt and did not invoke the adapter.

Final catalog browser evidence Viewport SHA-256
Wajarri catalog card, controlled status 1440x1000 6f56fe749741b9542f6ab62021882daeef9d4f9c317768e76c2bc538fd04d877
Wajarri catalog card, wrapped immutable identities 390x844 07d840248ee0e6ad80d2eb7881365f5985e723675c2413c86c8627474f292ee8
Immutable model-version page 390x844 893beb8e52f39f4dafca9b9755831e20d4fd5dfb555d880ec3b77159a20862c2

The earlier homepage release 20260802T091947Z-72668fe89dc5 remains the first successful controlled-route deployment and is preserved in the chronological record above. The final release supersedes it only to expose the same immutable Wajarri artifact through the public registry and present that record consistently on mobile and desktop. Deployment retention also reported one pre-existing incomplete release from 2026-07-21 that could not be verified for pruning; it did not alter the active release or current runtime integrity.

Final v3 release ruling

The Wajarri v3 program is complete for the bounded release objective:

Required outcome Final state
Source-governed model-visible pairs complete and published
Independent seed confirmation pass, 3/3 distinct adapters
Exact controlled route pass, 35/35 for every seed
Batch-size invariance pass at batch 1 and 16
GPU-to-CPU parity pass, exact prediction hashes
Full result recovery and checksums pass
Wajarri RunPod shutdown pass; worker deleted
Public Hugging Face dataset pass
Public Hugging Face model and immutable tag pass
Shared-Space API pass with fail-closed refusals
Public model registry and immutable version API pass
Public model catalog desktop/mobile presentation pass
Homepage supported/fallback routing pass
PostgreSQL model-bound caches pass
Desktop/mobile browser acceptance pass
Free-form Wajarri translation not authorized
Productive morphology or natural-language reliability not established

The technically important result is architectural. Sparse, source-bound evidence did not justify manufacturing thousands of novel Wajarri clauses. Direct sentence fine-tuning repeatedly traded sentence fitting against dictionary retention. The successful release instead isolates exactly what the evidence supports: deterministic dictionary retrieval for the subject, neural selection/generation of one complete governed predicate under a forced task prefix, deterministic rendering, and refusal outside the frozen contract.

Any later free-form Wajarri model is a new research program. It must begin with additional independently attested clauses or explicit speaker-reviewed morphological paradigms, family-level held-out evaluation, and a separate sentence-generation gate. It must not inherit v3's narrow pass as evidence for unrestricted translation.