Datasets | Mozilla Data Collective

Common Voice

Common Voice Scripted Speech 26.0 - Brahui

A collection of read speech recordings in Brahui.

License: CC0-1.0

Locale: brh

Task: ASR

Format: MP3

Size: 198.07 MB

Balochistan Educational and Cultural Organization

BECO Brahui Literature Corpus

A ~355k-token Brahui literary corpus of short stories, novels, and other creative works for linguistic research and NLP.

License: CC-BY-NC-SA-4.0

Locale: brh

Task: NLP

Format: TXT

Size: 1.19 MB

Balochistan Educational and Cultural Organization

Brahui Research Work Corpus

This corpus consists of research works, academic theses, and scholarly papers, comprising approximately 185,000 tokens. It covers a range of academic topics and formal registers, reflecting standardized writing practices and disciplinary conventions. The corpus is intended to support linguistic analysis, discourse studies, and computational research on academic and research-oriented text.

License: CC-BY-NC-SA-4.0

Locale: brh

Task: NLP

Format: TXT

Size: 1.13 MB

FAIR at Meta

BOUQuET

BOUQuET is a multidomain benchmark for machine translation quality. It contains 1358 sentences (318 paragraphs), fully parallel in 275 language varieties.

License: CC-BY-4.0

Locale: mul

Task: MT

Format: TSV

Size: 50.65 MB

Balochistan Educational and Cultural Organization

Talar (تلار) Barahui Magazine Corpus

A ~150,000-word Brahui corpus from the monthly magazine Talar, covering editorials, essays, fiction, poetry, and socio-cultural commentary.

License: CC-BY-NC-SA-4.0

Locale: brh

Task: NLP

Format: TXT

Size: 317.22 KB

Institute of African Digital Humanities

Bati-MultiDialectalASR-Dataset

This dataset contains paired audio and text resources for three Bati dialects (Kelleng, Mbougue, and Nyambat), which belong to the Yambasa group of Bantu languages found in Cameroon. It contains 13,344 audio clips totalling 6 hours, 8 minutes and 12.286 seconds and 44 audio/text mapping files totalling 13,346 lines. Due to its cross-dialectal nature, the dataset is suitable for multilingual automatic speech recognition tasks.

License: NOODL-1.0

Locale: btc

Task: ASR

Format: WAV, TSV

Size: 3.27 GB

Institute of African Digital Humanities

Bulu-ASR-Dataset

Bulu (bum) scripted speech dataset: 819 clips from 8 speakers (predominantly female) across 9 recording sessions.

License: NOODL-1.0

Locale: bum

Task: ASR

Format: MP3, TSV

Size: 42.11 MB

Maseno Centre for Applied Artificial Intelligence (MCAAI)

DhoNam: Dholuo Speech dataset

DhoNam: Dholuo Speech dataset is a speech corpus designed to supercharge Automatic Speech Recognition (ASR) and other speech technologies for Dholuo, one of Kenya’s major indigenous languages. This dataset contains native-speaker audio recordings collected through a platform where users read aloud a displayed sentence. The dataset includes the audio recordings and the corresponding prompt/sentence that was read.

License: NOODL-1.0

Locale: Luo

Task: ASR

Format: WEBM

Size: 2.49 GB

Institute of African Digital Humanities

Bamun-French Parallel Corpus 1.1

This dataset is an updated version of the "Bamun-French Parallel Corpus", a parallel corpus of texts in Bamun (Shupament) and French.

License: NOODL-1.0

Locale: bax

Task: MT

Format: TSV

Size: 99.78 KB

Institute of African Digital Humanities

Tupuri-ASR-Dataset

Tupuri (tui) scripted speech: 2,085 clips (≈2h46min) from different speakers across two dialect subsets — Bango and Banwere.

License: NOODL-1.0

Locale: tui

Task: ASR

Format: MP3,, TSV

Size: 97.82 MB

MirasAI

Sylheti Text Corpus by Haque Publishers

A 524K token Sylheti corpus featuring drama scripts and cultural texts in UTF-8 format for linguistic research and NLP development.

License: CC-BY-NC-4.0

Locale: syl

Task: NLP

Format: TXT

Size: 3.53 MB

Common Voice

Common Voice Scripted Speech 26.0 - Dhatki

A collection of read speech recordings in Dhatki.

License: CC0-1.0

Locale: mki

Task: ASR

Format: MP3

Size: 187.83 MB

DDD-Cambodia

Khmer ASR Cultural Dataset (Version 3 - Part 1)

Khmer ASR Cultural Dataset is licensed under CC-BY-SA-4.0. Please attribute Digital Divide Data when using it.

License: CC-BY-SA-4.0

Locale: khm

Task: ASR

Format: WAV

Size: 45.10 GB

Community

Sindhi Word Segmentation (SdSEG)

A sentence-level annotated dataset for training and evaluating sequence labeling models such as CRF, BiLSTM, and transformer-based architectures.

License: CC-BY-NC-4.0

Locale: snd

Task: NLP

Format: JSON

Size: 1.13 MB

Kaleem Art Press

Saraiki-English Parallel Corpus

English–Saraiki parallel corpus: 51,447 aligned sentence pairs (~0.89M words), translated by Kaleem Art Press for MT and Saraiki NLP research.

License: CC-BY-NC-4.0

Locale: mul

Task: MT

Format: CSV

Size: 1.92 MB

Institute of African Digital Humanities

Teke-Laali-TTS-Dataset

The dataset contains paired audio and text resources for Teke-Laali, a Bantu language spoken in the Congo. It consists of seven folders containing a total of 9,069 audio clips from raw audio recordings, with a total duration of 7:01:50.126 (HH:MM:SS.mmm). Additionally, there are seven audio/text mapping files containing a total of 9,069 lines. The dataset is suitable for TTS tasks.

License: NOODL-1.0

Locale: lli

Task: TTS

Format: WAV, TSV

Size: 635.61 MB

Institute of African Digital Humanities

Tupuri-Bango_TTS-Dataset (female voice)

Tupuri-Bango (tui) scripted TTS dataset: 2,034 clips (3h 53m 42s), single female speaker, AGLC orthography. Female-voice complement to the male-voice dataset.

License: NOODL-1.0

Locale: tui

Task: TTS

Format: MP3, TSV

Size: 169.22 MB

Institute of African Digital Humanities

Sample Tupuri-Bango_TTS-Dataset (male voice)

Tupuri-Bango (tui) scripted TTS dataset: 983 clips (≈1h 06m) from 10 sessions, AGLC orthography. Kaele & Mayo-Danay, Far-North, Cameroon.

License: NOODL-1.0

Locale: tui

Task: TTS

Format: MP3, TSV

Size: 46.93 MB

MDC DatasetsCompensated

🚧 Khowar-English Parallel Corpus 🚧

A parallel corpus of over 10,000 sentences of Khowar and English. The sentences are derived from a multimodal corpus consisting of 152 transcribed and translated Khowar language videos.

License: MDC DLA-1.0

Locale: khw, en

Task: MT

Format: TSV

Size: 407.57 KB

MirasAI

Rangpuri (অংপুরি Ôṅgpuri) Text Corpus

A 500K token corpus of Rangpuri literature and drama provided in UTF-8 format for linguistic research and NLP development.

License: CC-BY-NC-4.0

Locale: rkt

Task: NLP

Format: TXT

Size: 3.68 MB

MEDIAMEN

English–Punjabi (Shahmukhi) Parallel Sentences Corpus (Mediamen Archives)

This parallel sentences corpus containing 30,405 aligned sentence pairs with a total of approximately 0.62 million tokens, curated from the archival materials of Mediamen (Advertising Agency). The sentences were professionally translated from English into Punjabi (Shahmukhi) and are intended to support machine translation, linguistic research, and Punjabi language technology development, particularly for real-world and contemporary language use.

License: CC-BY-NC-4.0

Locale: en-PK, pnb

Task: MT

Format: CSV

Size: 1.08 MB

Institute of African Digital Humanities

Bulu-TTS-Dataset 1.0

The dataset consists of 3 hours and 16 minutes of denoised audio clips, each paired with text and read by a single Bulu speaker.

License: NOODL-1.0

Locale: bum

Task: TTS

Format: MP3, TSV

Size: 87.40 MB

Institute of African Digital Humanities

Bamun-French Parallel Corpus 2.0

This dataset is an extended and updated version of the "Bamun-French Parallel Corpus 1.1", a parallel corpus of 4,444 lines in Bamun and French.

License: NOODL-1.0

Locale: bax

Task: MT

Format: TSV

Size: 184.29 KB

Community

Mandar Balanipa Speech Corpus

This Mandar language dataset is text and audio data paired for languange documentation and linguistic research in technology.

License: CC-BY-NC-4.0

Locale: mdr

Task: TTS

Format: WEBM, TSV

Size: 281.70 MB

All Datasets

Not sure where to start?

Common Voice Scripted Speech 26.0 - Brahui

BECO Brahui Literature Corpus

Brahui Research Work Corpus

BOUQuET

Talar (تلار) Barahui Magazine Corpus

Bati-MultiDialectalASR-Dataset

Bulu-ASR-Dataset

DhoNam: Dholuo Speech dataset

Bamun-French Parallel Corpus 1.1

Tupuri-ASR-Dataset

Sylheti Text Corpus by Haque Publishers

Common Voice Scripted Speech 26.0 - Dhatki

Khmer ASR Cultural Dataset (Version 3 - Part 1)

Sindhi Word Segmentation (SdSEG)

Saraiki-English Parallel Corpus

Teke-Laali-TTS-Dataset

Tupuri-Bango_TTS-Dataset (female voice)

Sample Tupuri-Bango_TTS-Dataset (male voice)

🚧 Khowar-English Parallel Corpus 🚧

Rangpuri (অংপুরি Ôṅgpuri) Text Corpus

English–Punjabi (Shahmukhi) Parallel Sentences Corpus (Mediamen Archives)

Bulu-TTS-Dataset 1.0

Bamun-French Parallel Corpus 2.0

Mandar Balanipa Speech Corpus