# Kuku Yalanji Isolated-Lexicon Version Probe

Status: **frozen before model inference**  
Created: `2026-07-14T07:23:50.679303+00:00`

## Question

Under one fixed decoder, what percentage of isolated English dictionary glosses does each model return as one of
the canonical Kuku Yalanji headwords recorded for that gloss? How does that result differ for target forms observed
or unobserved as complete tokens in v21.2 training outputs, and how do the same versions perform on the separate
43-row elder sentence corpus?

## Fixed construction

- Source lexicon SHA-256: `a461612f8d2df805f4dd9f311e7a96702eee108cab04d752a6cbb7564b8b6993`
- Training split SHA-256: `7cb97c1c1d1afd05d0b7ddd04f64180182bf9de261bf5473d07b28bdb43967f0`
- Frozen probe SHA-256: `38ce15e36a60018d1e9c057f359a68f4c3bf7f8f09b4d027fa7ca82abd88cce7`
- Elder file SHA-256: `29a9bfbfecbe535b56eda3bf0ecf255a17190b438186b60b999c78e1ee4854d2`
- Combined inference file SHA-256: `6e525f16eb7475c6bc5450b978d70db0fbf1a0d71f2af9f143fb80c4bf60c01f`
- Lexicon entries considered: 1640
- Eligible single-gloss entries: 395
- Unique English prompts: 297
- Multi-reference prompts: 68

The mechanical inclusion and grouping rules are recorded in `probe_manifest.json`. No prediction has been inspected
to add, remove, or rewrite a reference. Hyphens and apostrophes remain meaningful characters. Every canonical
headword sharing the same normalized English gloss is accepted; an arbitrary single-reference exact score is not
used as the primary word endpoint.

The combined inference file contains the dictionary rows followed by the unchanged 43 elder rows. The scorer splits
them by their structured `pair_kind`; they are never pooled into one metric.

## Fixed decoding and endpoints

All checkpoints will use: one beam, `no_repeat_ngram_size=4`, repetition penalty `1.10`, length penalty `1.0`.
The primary word endpoint is NFC/casefolded exact agreement with any accepted headword. Raw exact, token
containment, character error rate, chrF++, empty output, and source copying are secondary diagnostics. The elder set
is scored separately with corpus chrF++, exact match, empties, and paired row-level bootstrap intervals.

This probe is post-training and cannot affect checkpoint or decoder selection. It is not blind: the dictionary was
available during corpus construction, and the elder set has been observed in prior analyses. Results therefore
describe frozen regression and dictionary coverage, not independent population-level translation validity.