Datasets | Mozilla Data Collective

Common Voice

Common Voice Scripted Speech 26.0 - Sakizaya

A collection of read speech recordings in Sakizaya.

License: CC0-1.0

Locale: szy

Task: ASR

Format: MP3

Size: 280.80 MB

Tamazight NLP

AmaWar

Amawal Warayni - ⴰⵎⴰⵡⴰⵍ ⴰⵙⵏⵎⴰⵍⴰⵢ ⵏ ⵉⵏⵓⵎⴰⴽ ⵏ ⵡⴰⵙⵙⴰⵖⵏ ⴷ ⵉⵎⵢⴰⴳⵏ ⵏ ⵜⵎⴰⵣⵉⵖⵜ ⵏ ⴰⵢⵜ ⵡⴰⵔⴰⵢⵏ

License: Apache-2.0

Locale: rif

Task: MT

Format: TSV

Size: 100.28 KB

Kaleem Art Press

Kaleem Art Press Saraiki Literature Corpus

This corpus contains approximately one million tokens of Saraiki text curated over the past ten years by Kaleem Art Press. It features a wide range of literary genres, including stories, short stories, novels, fiction, non-fiction, travelogues, poetry, biographies, and historical writings. All content is shared with full author approval. The dataset is intended to support linguistic research, Saraiki language technology development, and the preservation of cultural and literary heritage.

License: CC-BY-NC-4.0

Locale: skr

Task: OTH

Format: TXT

Size: 1.84 MB

Baloch Publishers Multan

Baloch Publishers Saraiki Literature Corpus

This corpus is a collection of one million tokens of Saraiki language. The data was produced under the Baloch Publishers over the last ten years. The corpus contains work of literature including dictionaries, short stories, novels, fiction, non-fiction, and travelogue. The data is being shared with the approval of the authors. It aims to support linguistic research, language technology development, and cultural preservation.

License: CC-BY-NC-4.0

Locale: skr

Task: NLP

Format: TXT

Size: 2.04 MB

CLEAR Global

TWB Parallel Sentence kits - Hausa (30k)

30,000 English–Hausa parallel sentences from CLEAR Global's Gamayun kits, translated by professional translators from Tatoeba source sentences.

License: CC-BY-4.0

Locale: hau

Task: MT

Format: TSV

Size: 1.68 MB

Taruen

Kazakh Proverbs Text Corpus

A corpus of traditional Kazakh proverbs from the Babalar sözi series (vols 65-69), designed for NLP and LLM cultural competence evaluations.

License: CC0-1.0

Locale: kk

Task: NLP

Format: TXT

Size: 592.14 KB

Common Voice

Common Voice Scripted Speech 26.0 - Wadiyara Koli

A collection of read speech recordings in Wadiyara Koli.

License: CC0-1.0

Locale: kxp

Task: ASR

Format: MP3

Size: 221.10 MB

Digital Divide Data

ddd-kenya-luhya-70hrs-asr

A collection of high-quality, crowdsourced audio recordings and verified transcriptions for the Luhya language, produced by Digital Divide Data.

License: CC-BY-4.0

Locale: luy

Task: ASR

Format: WAV, XLSX, TSV

Size: 13.90 GB

CLEAR Global

TWB Parallel Sentence kits - Swahili (5k)

5,000 English–Swahili parallel sentences from CLEAR Global's Gamayun mini-kit, translated by professional translators from Tatoeba source sentences.

License: CC-BY-4.0

Locale: swa

Task: MT

Format: TSV

Size: 347.61 KB

Common Voice

Common Voice Spontaneous Speech 4.0 - Sabah Bisaya

A collection of spontaneous responses to questions in Sabah Bisaya.

License: CC0-1.0

Locale: bsy

Task: ASR

Format: MP3

Size: 221.30 MB

Taruen

Uzbek Proverbs Text Corpus

A corpus of traditional Uzbek proverbs digitized from a 2005 volume, designed for NLP tasks and LLM cultural competence evaluations.

License: CC0-1.0

Locale: uz

Task: NLP

Format: TXT

Size: 142.68 KB

Kaleem Art Press

Saraiki-English Parallel Corpus

English–Saraiki parallel corpus: 51,447 aligned sentence pairs (~0.89M words), translated by Kaleem Art Press for MT and Saraiki NLP research.

License: CC-BY-NC-4.0

Locale: mul

Task: MT

Format: CSV

Size: 1.92 MB

CLEAR Global

TWB Parallel Sentence kits - Congo Swahili (25k)

25,305 French–Congo Swahili parallel sentences from CLEAR Global's Gamayun kits, translated by professional translators from Tatoeba source sentences.

License: CC-BY-4.0

Locale: swc

Task: MT

Format: TSV

Size: 2.18 MB

CLEAR Global

TWB Parallel Sentence kits - Lingala (5k)

5,000 French–Lingala parallel sentences from CLEAR Global's Gamayun mini-kit, translated by professional translators from Tatoeba source sentences.

License: CC-BY-4.0

Locale: lin

Task: MT

Format: TSV

Size: 494.43 KB

Institute of African Digital Humanities

Gbaya-Lay_ALCAM-MultimodalDataset

This dataset comprises example sentences in Gbaya-Lay with French equivalents, audio recordings by a native speaker, and a sentence–audio alignment file.

License: NOODL-1.0

Locale: gya

Task: NLP

Format: MP3, TSV

Size: 9.23 MB

Sujaak Adbi Sangat

Saraiki Quarterly Magazine Wasson Wehray Corpus

This corpus contains 11,79,200 tokens, from a Saraiki Quarterly Magazine "Wasson Wehray"

License: CC-BY-NC-4.0

Locale: skr

Task: OTH

Format: TXT

Size: 2.09 MB

CLEAR Global

TWB Parallel Sentence kits - Kanuri (5k)

5,000 English–Kanuri parallel sentences from CLEAR Global's Gamayun mini-kit, translated by professional translators from Tatoeba source sentences.

License: CC-BY-4.0

Locale: kau

Task: MT

Format: TSV

Size: 358.46 KB

Common Voice

Common Voice Scripted Speech 26.0 - Asheninka South Ucayali

A collection of read speech recordings in Asheninka South Ucayali.

License: CC0-1.0

Locale: cpy

Task: ASR

Format: MP3

Size: 200.91 MB

MEDIAMEN

English–Punjabi (Shahmukhi) Parallel Sentences Corpus (Mediamen Archives)

This parallel sentences corpus containing 30,405 aligned sentence pairs with a total of approximately 0.62 million tokens, curated from the archival materials of Mediamen (Advertising Agency). The sentences were professionally translated from English into Punjabi (Shahmukhi) and are intended to support machine translation, linguistic research, and Punjabi language technology development, particularly for real-world and contemporary language use.

License: CC-BY-NC-4.0

Locale: en-PK, pnb

Task: MT

Format: CSV

Size: 1.08 MB

Institute of African Digital Humanities

A Bakoko Sociocultural Dataset

Sociocultural dataset for the Bakoko community of Cameroon: identity, kinship, religion, economy, onomastics and a linguistic component.

License: NOODL-1.0

Locale: bkh

Task: OTH

Format: TSV, PDF

Size: 78.82 KB

Institute of African Digital Humanities

Akoose-ALCAM-MultimodalDataset

Curated Akoose (Bakosi) sentences with English translations, audio recordings, and sentence–audio alignments.

License: NOODL-1.0

Locale: bss

Task: NLP

Format: MP3, TSV

Size: 16.05 MB

Common Voice

Common Voice Scripted Speech 26.0 - Santali (Ol Chiki)

A collection of read speech recordings in Santali (Ol Chiki).

License: CC0-1.0

Locale: sat

Task: ASR

Format: MP3

Size: 20.51 MB

Common Voice

Common Voice Scripted Speech 26.0 - Sakha

A collection of read speech recordings in Sakha.

License: CC0-1.0

Locale: sah

Task: ASR

Format: MP3

Size: 510.25 MB

Common Voice

Common Voice Spontaneous Speech 4.0 - Kelabit

A collection of spontaneous responses to questions in Kelabit.

License: CC0-1.0

Locale: kzi

Task: ASR

Format: MP3

Size: 194.34 MB

All Datasets

Not sure where to start?

Common Voice Scripted Speech 26.0 - Sakizaya

AmaWar

Kaleem Art Press Saraiki Literature Corpus

Baloch Publishers Saraiki Literature Corpus

TWB Parallel Sentence kits - Hausa (30k)

Kazakh Proverbs Text Corpus

Common Voice Scripted Speech 26.0 - Wadiyara Koli

ddd-kenya-luhya-70hrs-asr

TWB Parallel Sentence kits - Swahili (5k)

Common Voice Spontaneous Speech 4.0 - Sabah Bisaya

Uzbek Proverbs Text Corpus

Saraiki-English Parallel Corpus

TWB Parallel Sentence kits - Congo Swahili (25k)

TWB Parallel Sentence kits - Lingala (5k)

Gbaya-Lay_ALCAM-MultimodalDataset

Saraiki Quarterly Magazine Wasson Wehray Corpus

TWB Parallel Sentence kits - Kanuri (5k)

Common Voice Scripted Speech 26.0 - Asheninka South Ucayali

English–Punjabi (Shahmukhi) Parallel Sentences Corpus (Mediamen Archives)

A Bakoko Sociocultural Dataset

Akoose-ALCAM-MultimodalDataset

Common Voice Scripted Speech 26.0 - Santali (Ol Chiki)

Common Voice Scripted Speech 26.0 - Sakha

Common Voice Spontaneous Speech 4.0 - Kelabit