{ "file": "/workspace/models/kuku-yalanji-nllb-lora/v12.0-tagged-direct-plus-reference-bible-gvn-token-4096row-25epoch-batch16/eval_heldout_direct325_predictions_greedy.json", "metrics": { "bleu": 13.058005039179777, "chrf": 44.370825777267356, "rows": 325, "num_beams": 1, "no_repeat_ngram_size": 0, "repetition_penalty": 1.0, "length_penalty": 1.0 }, "rows": 325, "empty_predictions": 0, "length_ratio": { "mean": 0.9744215668270354, "median": 0.9473684210526315, "min": 0.5, "max": 1.75 }, "max_token_repeat_share": { "mean": 0.1263190400076581, "median": 0.11764705882352941, "min": 0.045454545454545456, "max": 0.5 } } SAMPLES REF: MAT.4.18 SRC: Walking by the sea of Galilee, he saw two brothers: Simon, who is called Peter, and Andrew, his brother, casting a net into the sea; for they were fishermen. HYP: Jesus yikindarrda jalundurr Galileeburr dungan-dungan, nyulu bulanin jambul nyajin, bulal yaba-yabaju, Simon, nyulu burri yindu Peter. Nyungu yaba-yabaju Andrew makarr yilbanya, minya kuyu maninka, sellim-bunganka. TGT: Nyulu Jesus jalundurr Galileemundurr dungan-dungan, nyulu bulangan nyajin, yaba-yabaju. Nyubun burri Simon, nyungu burri yindu Peter. Yabaju burri Andrew. Bula makarrda kuyu manil-manin, yinya bulanga work. REF: REV.4.4 SRC: Around the throne were twenty-four thrones. On the thrones were twenty-four elders sitting, dressed in white garments, with crowns of gold on their heads. HYP: Chair 24-bala yinyangkarr chairmundurr chairmundurr ngulkurr. Jana chairnga bundandan, elder-elder 24-bala, jananga kambi bingaji. Wubulku dukulbu crownji gold-one. TGT: Chair ngulkurr yindu-yindu 24-bala yinyangkarr chairmundurr. Elder-elder 24-bala bundandan yinyaymba chairnga 24-balanga. Jananga kambi bingaji, wandu crown gold-one dukulbu. REF: 1CO.9.17 SRC: For if I do this thing willingly, I have a reward: but if against my will, a dispensation of the gospel is committed to me. HYP: Kaki ngayu ngayku work ngulkurr balkal, Godungku nganya payim-bungal. Kari. Kaki ngayu work ngulkurr balkal, Godungku nganya yungan. TGT: Kaki ngayu wawubu Godumu kuku balkanka, ngayu money maninka yurrandamun. Yamba yala kari. Godungkuku nganya manin, nyungu work balkanka. Ngayu must nyungun kuku nyaka. REF: MRK.10.37 SRC: They said to him, “Grant to us that we may sit, one at your right hand and one at your left hand, in your glory.” HYP: Bula yalaman, Yundu junkurrji bajaku, yundu nganjinin yulmbarril. Nganjin yunu maja-bunganka, yubalin majaji ngulkurr bajaku, yubalin majamanka. TGT: Bula nyungundu yalaman, “Yalada ngalin yununji bundanka? Yundu majamal, ngalin yununji bundanka, yinyamundu majamanka yununji.” REF: ACT.15.19 SRC: Therefore my judgment is that we don’t trouble those from among the Gentiles who turn to God, HYP: Johnungku yalaman baja, Kanbal Gentile-warra Godundu jurrkijin, ngana kari kuli dajil baja jananka. TGT: Jamesangka yalaman baja, “Ngana kari kuku hard-bala yunga bamanda Gentile-warranda Godundu jurrkijinyamunbu. REF: LUK.5.25 SRC: Immediately he rose up before them, and took up that which he was laying on, and departed to his house, glorifying God. HYP: Nyulu dingkar dakan, bamanda wubulku nyajinka. Nyulu bilangkurr manin, God buyay-manin, bayanbaku dungan. TGT: Nyulu waba-buyun jinbalku dakan, bilangkurr manin, bayanbaku dungan, Godkuda buyay-manil-manin. REF: MRK.8.5 SRC: And he asked them, How many loaves have ye? And they said, Seven. HYP: Jesusangka jananda babajin, Yurranga mayi bread wanjarr? Jana yalaman, Bread 7-bala. TGT: Jesusangka jananda babajin, “Yurranga mayi bread wanjarr?” Jana yalaman, “7-bala.” REF: LUK.7.40 SRC: And Jesus answering, said to him, Simon, I have somewhat to say to thee. And he saith, Master, speak. HYP: Jesusangka nyungundu yalaman, Simon, ngayu yunundu kuku dajil. Simonangka yalaman, Mawada, balkada. TGT: Jesusangka Simonanda yalaman, “Ngayu junjuy balkanka yunundu, Simon.” Simonangka yalaman, “Yundu wanyu balkanka ngaykundu?”