Release Date: 10/2/2026
Format: JSONL
Size: 391.72 KB
# MALOBA — Lingala OCR transcriptions, harmonised text (v1.1) **Access and licence.** Access is granted automatically once requested. However, please note: use of this dataset is governed by NOODL-1.0, which requires organisations based outside Africa and developing countries to expressly inform Congo Digital & Service (CDS) of their specific intended use, in particular any commercial use. Please do so through the Request to Access form. Dataset of Congo Digital & Service (CDS), MALOBA project, with UNDP Republic of Congo. Text only (no images): reference transcriptions of handwritten and printed Lingala documents, annotated in Label Studio, harmonised to the MALOBA spelling convention (Unicode look-alikes fixed, tone marks removed, majority ɔ/ɛ spelling). Monolingual Lingala, intended as source text for back-translation (NLLB-Maloba). - `ocr.jsonl`: 1,282 unique lines (duplicates and unrevised pre-annotations removed), with `texte_brut` (original) and `modifications` (each replacement: step, before, after). - `ocr_pages.jsonl`: 104 pages rebuilt in reading order. - `ocr_nllb.jsonl` / `.parquet`: 535 units for NLLB (67 pages + 468 lines, 22,498 words); exclusions listed in `ocr_nllb_exclusions.jsonl` (lines under 3 words, 2 fragments close to the OCR test set). - The 40-line OCR test set is NOT included and must stay out of any training. - Known limits: two French loanwords lost their accent through the rule (`Thème` → `Theme` ×6, `bénéfice` → `benefice` ×1); 61 accented forms absent from the lexicon are kept as written; a few rare letters (ɑ, ı, ł, š, õ, ĩ) have no rule. Details in `rapport_finalisation_ocr.md`. - No annotator e-mail, name or pseudonym.
Licensing
Nwulite Obodo Open Data Licence 1.0 (NOODL-1.0)
Private dataset: access limited to the CDS community on Mozilla Data Collective. Use under the NOODL 1.0 licence.
Mixing with the MALOBA OCR test set (not included) for training.
Ethical Review
These datasets contain no personal data. The texts come from documents provided by SIL Congo and from prompts written for the project. They were transcribed by Lingala-speaking annotators recruited by CDS, mostly graduates of the Faculté des Lettres et des Langues Vivantes, Université Marien Ngouabi (UMNG), then checked by a lead linguist. Handwritten documents were anonymised before annotation. The identities of annotators and validators are not published.
Intended Use
Monolingual Lingala source text for back-translation and language modelling (NLLB-Maloba).
DATASHEET (template shared by UNDP, after Gebru et al. "Datasheets for Datasets" and Bender & Friedman "Data Statements for NLP")
Core purpose: monolingual Lingala text from document transcriptions, for back-translation and language modelling. Motivation: written Lingala is rarely digitised. Built by CDS in the MALOBA project (digitisation and acceleration of Lingala), with UNDP Republic of Congo and the Ministry of Posts, Telecommunications and the Digital Economy. Curation rationale: only transcriptions validated by a person are kept; duplicates and unrevised machine pre-annotations are removed; the 40-line OCR test set is excluded. Language / variety: written Lingala as found in the source documents, harmonised to the MALOBA spelling convention; the original text is kept in texte_brut. Data subject demographics: the documents are printed and handwritten texts held by SIL Congo; handwritten documents were anonymised before annotation. No personal data is included. Annotator demographics: a team of Lingala-speaking annotators, mostly graduates of the Faculté des Lettres et des Langues Vivantes, Université Marien Ngouabi (UMNG), trained in AI and IT and paid for their work by CDS, checked by a lead linguist. Identities and detailed sociodemographic data are not published. Speech situation: written text (not speech): mostly educational, agricultural and radio-script content. Composition: 1,282 unique lines, 104 rebuilt pages and 535 units for NLLB (22,498 words). Text only, no images. Collection process: see Collection below. Preprocessing / cleaning: cleaning of regions (see below), then deterministic spelling harmonisation (Unicode look-alikes fixed, tone marks removed, majority ɔ/ɛ spelling), every change listed in modifications. Uses: back-translation source and language modelling for Lingala. Must not be used to train OCR models evaluated on the MALOBA OCR test set. Distribution: Mozilla Data Collective, under NOODL-1.0. Maintenance: CDS ([email protected]). Changes are published as new versions. Legal and ethical compliance: SIL Congo, a MALOBA partner, provided the documents. Handwritten documents were anonymised; annotator identities are not published. Provenance: SIL Congo documents → Partkole and Label Studio annotations (1,747 regions) → 1,323 validated transcriptions → 1,282 harmonised lines (v1.1).
Collection. Source images come from Lingala documents held by SIL Congo: 26 handwritten documents, photographed and anonymised before annotation, and printed documents converted to images on the project's collection platform (Partkole). Annotators drew a rectangle or polygon around each text region in Label Studio and transcribed it exactly, labelling it as printed or handwritten. The project's 117 annotation tasks produced 1,747 text regions.
From annotations to the OCR image dataset (Hugging Face: Congo-digital-service/dataset-qwen-vl-lingala-qlora-vf). The regions were cleaned before use:
empty or incomplete boxes were removed (18); machine pre-annotations never corrected by a person were removed (263); duplicate texts were removed (143).
This leaves 1,323 unique transcriptions validated by a person; the 1,282 harmonised lines of this text dataset come from them. The OCR image dataset is a sample built from them to target the two Lingala open vowels:
164 lines containing ɔ or ɛ and 164 ordinary lines (328 in total), split 85/15 before any oversampling into 278 training and 50 test lines; the training lines containing ɔ or ɛ were then duplicated once, giving 417 training rows.
The test split is not oversampled. Duplicated training rows are intentional and should not be counted as independent examples.
Format. Each row is a multimodal chat example (ChatML): the user message holds the image of one text region (a crop, not a full page) and the instruction to transcribe the text exactly; the assistant message holds the reference transcription. Transcriptions keep the original spelling, including ɔ, ɛ, Ɔ and Ɛ.
Quality. Three independent Lingala-speaking validators reviewed all 50 test items and 41 training items. On the test split, 84 % of reference transcriptions were judged correct and 96 % at least partly correct. On the training sample, the figures are 80.5 % and 90.2 %. Inter-rater agreement (Krippendorff's α = 0.53) is below the project's 0.60 threshold, so these rates are indicative.
Reference result. Qwen2-VL-2B fine-tuned on that image dataset (Congo-digital-service/qwen-vl-lingala-qlora-vf) reaches 88.7 % character accuracy on the test split (CER 11.3 %, micro-averaged, normalised), against 60.0 % for the base model.
Known limitations.
The test split is small (50 items) and carries no source-document identifier, so results cannot be broken down by document. The collection is dominated by educational, agricultural and radio-script content from a single institutional source. Spelling follows each annotator; it is not harmonised in this dataset.
Related versions. qwen-vl-lingala-dataset-augmented (884 / 50 items) applies image degradations to 140 distinct training texts. It was not used to train the reference model. An earlier intermediate repository (1,480 / 82 rows) has been withdrawn.