Mi'gmaq Dictionary Browse Consolidation

Completed: 2026-07-12
Live dictionary: mobtranslate.com/dictionaries/migmaq
Ruling: canonicalize the browse view without deleting or overwriting historical rows

1. Why this was required

Visual verification of the new translator exposed repeated headwords on the Mi'gmaq dictionary page. The page reported 13,643 word rows, although the archived Mi'gmaq Online source contains 6,974 entries. This was not a harmless layout duplication: two historical imports had created parallel records under two equivalent word-class taxonomies.

The audit found:

Exactly 6,581 older non-managed rows paired with 6,581 newer YAML-managed rows having the same normalized headword and normalized part-of-speech label. The newer rows generally held richer dictionary relations and the mapped recordings. The one three-row group was not fully duplicate: Winpegijuig has a noun record and a source-created verb-inanimate record. The consolidation therefore keys identity by normalized spelling and normalized part of speech, not spelling alone.

2. Data-preservation rule

No word, definition, translation, example, recording, or provenance row was deleted. Before migration, the complete affected state was exported to checksummed JSONL:

The exports, audit SQL, query logs, and checksum inventory live under reports/ in the Mi'gmaq training research root.

3. Canonicalization method

The migration 20260712123000_migmaq_dictionary_canonical_browse.sql runs in one transaction and refuses to commit unless it finds the expected 6,581 exact legacy/canonical pairs.

For each pair it:

1. selects the newer managed row with the same language, normalized headword, and normalized part-of-speech label;
2. copies legacy-only child content onto that canonical row with an explicit consolidation note;
3. preserves definition-to-translation associations where the corresponding canonical definition exists;
4. marks the older word row with metadata.browseSupersededBy, timestamp, and reason;
5. configures the language metadata with the non-destructive browse strategy;
6. verifies the final visible count and checks that no legacy-only definition or example remains absent from the canonical row;
7. commits only if every postcondition passes.

The application query layer excludes rows carrying browseSupersededBy from public lists, search, statistics, related-word lookup, and map browsing. Direct database and ID-based access remains available for audit and recovery.

4. Content moved forward

Reference comparison before migration found 3 legacy-only definitions, no globally legacy-only translation strings, and 111 legacy-only examples. To preserve relation structure as well as text, the migration copied:

• 3 definitions;
• 2,467 definition-linked translations whose association was absent on the matching canonical definition;
• 111 examples.

All copied rows carry a note naming the preserved source row. Idempotent NOT EXISTS checks prevent the migration from adding the same normalized child relation twice.

5. Postconditions

The live first page now contains one row per lexeme/class and reports Showing 1-50 of 7,062. Browser verification opened agase'wa'latl, exposed three attributed word-pronunciation controls plus an example recording, and received HTTP 206 for the archived MP3 media request.

6. Recovery

To restore the former browse behavior without deleting the copied relations:

1. remove or disable the isVisibleInDictionaryBrowse() predicate in the application query layer;
2. rebuild and restart the web service;
3. leave all browseSupersededBy metadata and copied relations intact as audit evidence.

A deeper rollback can remove only the metadata keys and child rows bearing the exact consolidation note, using the pre-consolidation JSONL inventories as the reference. That operation should be performed in a transaction and is not needed for normal recovery.

7. Linguistic non-claim

This is record consolidation, not a linguistic judgement that homographs or polysemous forms are identical. Identity is limited to duplicate import representations with the same normalized spelling and part-of-speech label. Distinct part-of-speech records remain visible. No dialect, grammaticality, semantic equivalence, or community-approval claim is inferred from database identity.