MobTranslate Kuku Yalanji v24 Independent-Review Disposition

Date: 2026-07-15 UTC
Status: active design record; not a training authorization
Applies to: the proposed v24 Kuku Yalanji lexical experiment and its public documentation

Provenance and limits

The project operator supplied a long review titled Independent review of the MobTranslate Kuku Yalanji program in the working session on 2026-07-15. The message contained the same review twice; this record treats it as one review. The review cited primary technical, linguistic, ethics, governance, and model sources. MobTranslate has not independently established the reviewer's identity or institutional authority, so it is engineering and research advice rather than community authorization, speaker certification, legal advice, or an ethics approval.

This disposition exists to preserve the reasoning behind the next revision. A recommendation is not adopted merely because it appears in the review. It is adopted when it corrects a real validity problem, removes an unsupported claim, or creates a proportionate operational control.

Decision summary

The review changes the v24 design materially, but it does not justify launching a larger program of work. The next experiment remains narrow: test direct lexical reconstruction and glossary-conditioned lexical uptake while measuring sentence retention. No RunPod job is authorized until the builder, fixed-compute schedule, full-lineage exposure audit, and preregistration exist and pass locally.

Recommendation area	Disposition	Reason
Separate lexical reconstruction, contextual acquisition, morphology, glossary uptake, and sentence translation	Adopt now	These are different estimands and product capabilities. Collapsing them caused overclaiming.
Replace isolated `<translate>` rows with a visible `<lexeme>` task	Adopt now	Prevents short lexical rows from silently sharing the sentence task and potentially reinforcing undertranslation.
Treat arbitrary fully hidden dictionary mappings as productive generalization	Reject	The target form is not derivable from English. A fully hidden mapping mostly tests prior exposure or leakage.
Full documented-lineage exposure audit	Adopt now	A v24-only seen/unseen label is scientifically misleading when v21.2 already has a long project lineage.
Fixed optimizer steps alone	Adopt with correction	Use an exact `max_steps` contract and report examples and source/target/non-padding tokens. Token accounting complements, rather than replaces, step matching.
Fixed-token substitution as the primary causal question	Adopt now	It tests the lexical intervention at comparable compute. One additive-compute arm may follow only after screening.
Untouched v21.2 plus retention-only continuation plus lexical treatment	Adopt now	Both baselines are required to separate continued-training effects from lexical-supervision effects.
Select a recipe across seeds, not a lucky seed	Adopt now	Three paired confirmatory seeds remain proportionate. Five seeds are deferred unless screening shows material variance.
Wilson interval as population reliability	Reject	The 297 prompts are a frozen constructed benchmark, not a random sample of future requests.
Wilson interval as a conservative frozen-benchmark acceptance rule	Adopt now	The 252/297 threshold remains useful when described accurately.
Sense/POS-aware lexical units and variant taxonomy	Adopt incrementally	Required for ambiguous entries; the builder must preserve all source entries and defer uncertain analyses rather than fabricate them.
Grapheme-aware CER and cluster-aware inference	Adopt for confirmatory evaluation	Useful once sense-level groups and enough independent clusters exist; not a blocker for builder unit tests.
Split dictionary-router, model-lexical, and sentence-generation gates	Adopt now	A model can memorize words while failing sentences. A lexical pass must never authorize the sentence service.
Governance as a public-release prerequisite	Adopt with a precise boundary	Unresolved rights or authority block public model release and restricted/speaker-data transfer. They do not require inventing an authority or halt private method development on material already documented as permitted.
Formal community approval manufactured by the technical team	Reject	The relevant people and institutions must define authority and scope. Unknown fields remain unknown.
Correct unsupported public community-ownership and universal-licensing claims	Adopt now	Current copy is stronger than the evidence recorded in the model program. The application code licence must be separated from data and model terms.
Retrieve a small glossary rather than send the complete dictionary to a hosted model	Benchmark before changing production	The complete dictionary is currently public and intentionally used. Compare relevant-entry retrieval against the full-dictionary prompt for quality, provenance, privacy, latency, and cost before replacing it.
Large speaker-reviewed grammatical challenge suite before any experiment	Defer	It is required before a sentence-reliability claim, not before the narrow lexical screening study. No sentence promotion is possible without it.
At least two reviewers for every natural item	Target for promotion review	Sensible when authorized reviewers and independent material are available; unavailable review must produce abstention, not an automatic pass.
Audit tokenizer fertility and the custom `gvn_Latn` embedding/output path	Adopt now	The custom token and PEFT serialization can alter behavior independently of LoRA weights.
New architecture or character/byte model in v24	Defer	It would confound the immediate lexical-treatment question. NLLB v21.2 remains the controlled baseline.
Separate lexical and sentence adapters	Defer	First test visible task separation in the controlled shared model. Deterministic dictionary lookup remains the product route regardless.
Adaptive decoder retry policy	Defer to a separate serving experiment	Decoder changes must not be mixed with the training-effect comparison.
Preserve every failed model iteration	Reject	Conflicts with disk limits and adds little value. Preserve compact evidence for all runs and compact adapters only for confirmatory runs when rights permit. Delete screening weights and bulky merged/optimizer artifacts after decision.
Signed provenance immediately	Defer	Immutable hashes and append-only manifests are adequate for v24. Signing is useful after a maintained key and release process exist.
Repair model-registry status semantics and add v23 negative result	Adopt now	The UI currently renders an executed `negative-result` as “Awaiting training run,” and v23 is absent.

Adopted v24 research questions

The revised study asks four separable questions:

Direct lexical reconstruction: after explicit <lexeme> supervision, can the model reproduce governed, closed-set dictionary mappings?
Contextual lexical acquisition: when a lemma appears in sentence training but not as an isolated pair, can the model reconstruct the lemma?
Lexicon uptake: when a held-out mapping is supplied only at inference under <glossary>, can the sentence model use it in a novel context?
Sentence retention: does lexical adaptation preserve the frozen synthetic, usage, and available attested sentence endpoints under an unchanged decoder?

Morphological generalization is a fifth valid task, but it enters v24 only where a source supplies an attested or reviewed lemma/form analysis. The project will not generate a large artificial paradigm and label it attested.

Adopted task shapes

<lexeme> woman
  -> jalbu

<translate> The woman returned.
  -> [Kuku Yalanji sentence]

<translate> The woman returned. <glossary> woman = jalbu
  -> [Kuku Yalanji sentence using the supplied lexical evidence]

Sense and part-of-speech labels are model-visible only when supported by source records:

<lexeme> light <pos> noun <sense> illumination

Metadata such as pair_kind remains necessary for sampling and provenance but is not treated as model conditioning unless represented in the input.

Adopted comparison contract

The primary screen is a fixed-token substitution comparison:

Arm	Description
B0	untouched v21.2 baseline; no continuation
C0	retention-only continuation with the same planned optimizer/token budget as treatment
L1	low direct lexical exposure, replacing scheduled retention tokens
L2	medium direct lexical exposure, replacing scheduled retention tokens
L4	high direct lexical exposure, replacing scheduled retention tokens
G	glossary-conditioned sentence/lexicon-uptake arm

One seed may screen development endpoints. The control and selected treatment recipe then run with paired seeds 17, 42, and 73. The recipe is selected from aggregate development behavior; final tests are opened for every confirmatory seed. Five seeds are not the default.

Every run must freeze and report:

planned and observed optimizer updates;
examples presented;
source, target, and total non-padding tokens presented;
scheduled presentations per task and unique lexical sense/sentence;
learning-rate schedule and horizon;
exact input order/seed contract;
behavior-affecting tokenizer, generation, evaluator, router, and benchmark hashes.

The first implementation may report planned per-task exposure from a deterministic schedule rather than invent a fragile approximation from epoch counts. Exact per-task loss attribution is useful instrumentation, not a launch prerequisite.

Adopted gate separation

The following decisions are independent:

dictionary_router:          PASS or FAIL
model_lexical_reconstruction: PASS or FAIL
model_sentence_generation: PASS or FAIL

The existing 297-prompt rule becomes the model lexical-reconstruction gate. It means only:

The exact model passes MobTranslate's conservative exact-reconstruction rule on the frozen 297-prompt benchmark.

It does not estimate general request reliability and does not authorize free-form sentence generation. The web dictionary route remains deterministic lookup-first. Sentence generation requires its own model-bound PASS based on natural semantic and grammatical evidence; no current Kuku Yalanji model has that PASS.

Governance and rights boundary

The project will not write an approved governance record without an identifiable authorizing body and documented scope. A machine-readable release state may truthfully say unresolved, list known source rights, and block only the scopes that are not established.

Before a public model release or third-party transfer of speaker/restricted material, the record must establish the applicable permissions for training, redistribution, derived weights, provider transfer, publication, withdrawal, and culturally sensitive material. Public availability or a copyright licence is not silently treated as community authorization.

Private code, builder, evaluator, and public-source method work may continue while governance is unresolved. Any uncertain data class is excluded or quarantined from that work. This is a practical stop boundary, not a claim that governance has been resolved.

Public-copy and licensing correction

Until stronger claims are documented, public copy should use language such as:

Independent open language research built from attributed dictionaries, grammars, recordings, and contributed material. Machine translations are unverified drafts and are not community-certified.

The footer and model pages must distinguish:

application code and its MIT licence;
each model's upstream and derivative terms;
language-data terms and source-specific restrictions;
synthetic-corpus terms and inherited restrictions.

The correction does not deny that individual speakers, linguists, or contributors may have participated. It stops turning undocumented participation into a platform-wide ownership or certification claim.

Explicit non-actions

This review does not authorize the following work now:

launching v24 before the revised preregistration and local preflight pass;
mounting a Kuku Yalanji sentence model because it passes isolated-word recall;
describing direct dictionary reconstruction as unseen lexical generalization;
treating 252/297 as 80% reliability over future user requests;
adding five-seed, multi-architecture, separate-adapter, or byte-model arms to the first screen;
creating a nominal community-approval record without the relevant authority;
deleting or overwriting prior experiment evidence;
retaining every weak screening model merely because storage is available.

Implementation sequence

Correct the public dossier's terminology, comparison design, gate interpretation, and retention policy.
Correct model-registry state semantics and add the missing v23 negative result.
Narrow unsupported community-ownership and universal-licensing copy.
Add exact max_steps and exposure accounting to the trainer contract.
Add a separate model-bound sentence-generation gate and keep it FAIL for all current models.
Build the sense-level eligibility/exposure ledger and deterministic v24 schedules.
Audit the custom language token, tokenizer fertility, and merged/base-plus-adapter behavior.
Freeze the preregistration, then and only then consider the one-seed RunPod screen.

Source references supplied with the review

These links are retained as review provenance. Their inclusion does not change the source-specific rights ledger or constitute community authorization.