Datasets | Mozilla Data Collective

Community

SKAD Ladino Speech Database (subset)

Spoken Ladino (Judeo-Spanish) from native speakers in Istanbul, recorded and transcribed by SKAD, re-packaged for ASR training and evaluation.

License: CC-BY-NC-SA-4.0

Locale: lad

Task: ASR

Format: WAV, TXT

Size: 4.99 GB

MDC DatasetsCompensated

🚧 Hazargi Multimodal Dataset 🚧

A Hazargi multimodal dataset with 10 hours of synchronized video, audio, transcriptions, speech segmentation, and ELAN annotations.

License: MDC DLA-1.0

Locale: haz

Task: NLP

Format: MP4, EAF

Size: 44.00 GB

Institute of African Digital Humanities

An Ewondo Sociocultural Dataset

Sociocultural dataset for the Ewondo community of Cameroon: identity, kinship, religion, economy, onomastics and a linguistic component.

License: NOODL-1.0

Locale: ewo

Task: OTH

Format: TSV, PDF

Size: 355.75 KB

Institute of African Digital Humanities

An Eton Sociocultural Dataset

Sociocultural dataset for the Eton community of Cameroon: identity, kinship, religion, economy, onomastics and a linguistic component.

License: NOODL-1.0

Locale: eto

Task: OTH

Format: TSV, PDF

Size: 367.19 KB

Institute of African Digital Humanities

A Ghomala Sociocultural Dataset

Sociocultural dataset for the Ghomala community of Cameroon: identity, kinship, religion, economy, onomastics and a linguistic component.

License: NOODL-1.0

Locale: bbj

Task: OTH

Format: TSV, PDF

Size: 346.33 KB

Institute of African Digital Humanities

A Chadian Arabic Sociocultural Dataset

Sociocultural dataset for the Chadian Arabic community of Cameroon: identity, kinship, religion, economy, onomastics and a linguistic component.

License: NOODL-1.0

Locale: shu

Task: OTH

Format: TSV, PDF

Size: 357.46 KB

PROXIMA

Sindhi Transliteration Dataset by Proxima AI

Sindhi transliteration dataset containing Sindhi sentences paired with Roman Sindhi transliterations for NLP research and model development.

License: CC-BY-NC-SA-4.0

Locale: snd

Task: NLP

Format: CSV

Size: 5.09 MB

Taruen

Chechen Proverbs and Sayings Text Corpus

A corpus of traditional Chechen proverbs and sayings digitized from a 1990 collection, designed for NLP tasks and LLM cultural competence evaluations.

License: CC0-1.0

Locale: ce

Task: NLP

Format: TXT

Size: 12.38 KB

Taruen

Bulgarian Proverbs and Sayings Text Corpus

A corpus of traditional Bulgarian proverbs and sayings digitized from a 1986 compilation, designed for NLP tasks and LLM cultural competence evaluations.

License: CC0-1.0

Locale: bg

Task: NLP

Format: TXT, TSV

Size: 132.39 KB

PROXIMA

Urdu Grammatical Error Correction Dataset

Paired Urdu dataset containing 600,000 incorrect correct sentence pairs for training grammatical and spelling error correction models.

License: CC-BY-NC-SA-4.0

Locale: urd

Task: NLP

Format: Parquet

Size: 107.99 MB

PROXIMA

Urdu AI Generated Quotes & Aphorisms

A collection of 3,317 AI-generated Urdu quotes and aphorisms for text-generation and language modeling research.

License: CC-BY-NC-SA-4.0

Locale: urd

Task: NLG

Format: JSON

Size: 174.90 KB

PROXIMA

Urdu OCR Multi Source Dataset (Proxima AI)

Urdu OCR dataset of 40,769 image text pairs compiled from scanned, printed, synthetic, and scene text sources.

License: CC-BY-NC-SA-4.0

Locale: urd

Task: CV

Format: PARQUET

Size: 436.48 MB

MDC Curators

CoVoST 2 Swedish - English

The dataset includes audio files in Swedish and their translations in English.

License: CC-BY-NC-4.0

Locale: sv-SE, en

Task: MT

Format: MP3, TSV

Size: 112.38 MB

AImpower.org

English Stuttered Speech

The dataset include 39 hours of transcribed and annotated conversational and voice command reading speech from 45 English speakers who stutter.

License: CC-BY-SA-4.0

Locale: en

Task: ASR

Format: WAV, TXT

Size: 9.75 GB

AImpower.org

Mandarin Stuttered Speech (StammerTalk)

The StammerTalk dataset contains 43 hours of spontenous conversations and reading of voice commands by 66 Mandarin Chinese speakers who stutter.

License: CC-BY-NC-SA-4.0

Locale: cn

Task: ASR

Format: WAV, TXT

Size: 4.61 GB

Sewá Morales

Ralámuli Ra’icháala TTS

Aproximadamente 2 horas con 20 minutos de habla leída de la lengua ralámuli, de una sola hablante.

License: CC-BY-NC-4.0

Locale: tar

Task: TTS

Format: WAV, TSV

Size: 422.68 MB

MDC Curators

CoVoST 2 English - Swedish

The dataset includes audio files in English and their translations in Swedish.

License: CC-BY-NC-4.0

Locale: en, sv-SE

Task: MT

Format: MP3, TSV

Size: 12.56 GB

PROXIMA

Urdu Alpaca

Filtered Urdu translation of the Stanford Alpaca instruction tuning dataset for Urdu LLM instruction fine-tuning.

License: CC-BY-NC-SA-4.0

Locale: urd

Task: NLG

Format: Parquet

Size: 20.26 MB

PROXIMA

Sindhi OCR by Proxima AI

Synthetically generated Sindhi OCR dataset of 278,000 text image pairs using dictionary words and multi word phrases.

License: CC-BY-NC-SA-4.0

Locale: snd

Task: CV

Format: PARQUET

Size: 1.34 GB

MDC Curators

Common Voice Scripted Speech 26.0 - Southern American English

A closed-form Southern-American-English-accent subset of Mozilla Common Voice Scripted Speech 26.0 (English, validated). MP3 audio + TSV metadata.

License: CC0-1.0

Locale: en-US

Task: ASR

Format: TSV, MP3

Size: 32.34 MB

Community

Priangan Dialect-Sundanese Parallel Corpus

This dataset consists of a parallel corpus of Sundanese translations of the Priangan dialect especially from Cianjur City in West Java, Indonesia.

License: CC-BY-NC-SA-4.0

Locale: sun

Task: MT

Format: tsv

Size: 366.64 KB

Institute of African Digital Humanities

A Bakoko Sociocultural Dataset

Sociocultural dataset for the Bakoko community of Cameroon: identity, kinship, religion, economy, onomastics and a linguistic component.

License: NOODL-1.0

Locale: bkh

Task: OTH

Format: TSV, PDF

Size: 78.82 KB

MDC DatasetsCompensated

🚧 African Actions Annotated Videos for AI (Health Domain) 🚧

A large multimodal dataset of actions performed in Nigeria: 10 egocentric videos and 1,666 time-stamped action-object-tool segments in the "health" domain.

License: MDC DLA 1.0

Locale: en, ig, ha, yo

Task: OTH

Format: MP4, JSON

Size: 4.81 GB

PROXIMA

MMLU ProX Sndhi

Sindhi translation of the MMLU-ProX benchmark for evaluating LLM reasoning and knowledge in a low resource language setting.

License: MIT

Locale: snd

Task: NLP

Format: CSV

Size: 2.04 MB

All Datasets

Compensated datasets

Conversational Speech in Gujarati in the Finance Domain

Conversational Speech in Manipuri in the Finance Domain

Conversational Speech in Manipuri in the Medical Domain

SKAD Ladino Speech Database (subset)

🚧 Hazargi Multimodal Dataset 🚧

An Ewondo Sociocultural Dataset

An Eton Sociocultural Dataset

A Ghomala Sociocultural Dataset

A Chadian Arabic Sociocultural Dataset

Sindhi Transliteration Dataset by Proxima AI

Chechen Proverbs and Sayings Text Corpus

Bulgarian Proverbs and Sayings Text Corpus

Urdu Grammatical Error Correction Dataset

Urdu AI Generated Quotes & Aphorisms

Urdu OCR Multi Source Dataset (Proxima AI)

CoVoST 2 Swedish - English

English Stuttered Speech

Mandarin Stuttered Speech (StammerTalk)

Ralámuli Ra’icháala TTS

CoVoST 2 English - Swedish

Urdu Alpaca

Sindhi OCR by Proxima AI

Common Voice Scripted Speech 26.0 - Southern American English

Priangan Dialect-Sundanese Parallel Corpus

A Bakoko Sociocultural Dataset

🚧 African Actions Annotated Videos for AI (Health Domain) 🚧

MMLU ProX Sndhi