Datasets | Mozilla Data Collective

Anjuman e Katib

Persian Literature Corpus by Najwai Sukhan

A curated Persian literary corpus of ~1.26M tokens spanning literature, poetry, educational writing, and culturally significant texts.

License: CC-BY-NC-4.0

Locale: fas

Task: NLP

Format: TXT

Size: 38.62 MB

Taraaz

Multilingual Humanitarian Response Eval (MHRE)

This multilingual humanitarian dataset contains 655 annotated datapoints evaluating AI chatbot safety and quality in migration and asylum scenarios across four language pairs (English–Farsi (Iranian Persian), Arabic, Kurdish (Sorani), Pashto). Built from 120 expert prompts, it includes outputs from GPT-4o, Gemini 2.5 Flash, and Mistral Small. The dataset provides both human evaluations from Respond Crisis Translation native-speaker evaluators and LLM-as-judge assessments (Gemini 2.5 Flash).

License: CC-BY-NC-SA-4.0

Locale: mul

Task: LLM

Format: csv

Size: 2.15 MB

Anjuman e Katib

Anjuman-e-Katib Farsi/Persian Literature Corpus

This corpus is a collection of more than one million tokens of Farsi/Persian language. The corpus contains work of literature including novels, fictional and non-fictional, poems and many more. the data is shared with the approval of the authors and aims to support linguistic research, language technology development and its preservation.

License: CC-BY-NC-4.0

Locale: fas

Task: NLP

Format: TXT

Size: 2.82 MB

MDC Community Concierge

VoxForge - Persian

16 minutes (157 utterances) of read speech of ‭فارسی‬ (Persian).

License: GPL-3.0

Locale: fa

Task: ASR

Format: WAV, TSV

Size: 58.20 MB

Aim Foundation

Dari Literature Corpus by Anjuman e Adabi Nayestan

A ~1 M-token Dari (Afghan Persian) literary corpus compiled by Anjuman e Adabi Nayestan, covering prose, poetry, and cultural texts in Perso-Arabic script.

License: CC-BY-NC-4.0

Locale: prs

Task: NLP

Format: TXT

Size: 12.67 MB

Common Voice

Common Voice Scripted Speech 26.0 - Persian

A collection of read speech recordings in Persian.

License: CC0-1.0

Locale: fa

Task: ASR

Format: MP3

Size: 10.45 GB

Keblagh e Azergi

Elkhani Hazargi Literature Corpus

Hazargi literary corpus (~0.5M tokens) of poetry, folklore, and prose texts representing Hazara linguistic and cultural heritage.

License: CC-BY-NC-4.0

Locale: haz

Task: NLP

Format: TXT

Size: 2.46 MB

RFE/RL

RFE/RL Persian News Text Corpus

This dataset is a longitudinal news corpus for the Persian language sourced from Radio Farda from 2001 to 2026. It contains over 350,000 articles (51M tokens).

License: CC-BY-NC-SA-4.0

Locale: fa

Task: NLP

Format: TXT

Size: 307.78 MB

MirasAI

Associated Press Pakistan 21.4 Million Pashto Corpus

Pashto news corpus from the Associated Press of Pakistan, containing 99,495 articles and 21.4 million words from 2015–2026.

License: CC-BY-NC-SA-4.0

Locale: pus

Task: NLP

Format: TXT

Size: 38.81 MB

Common Voice

Common Voice Scripted Speech 26.0 - Pashto

A collection of read speech recordings in Pashto.

License: CC0-1.0

Locale: ps

Task: ASR

Format: MP3

Size: 101.84 GB

MDC DatasetsCompensated

🚧 Northern Pashto Multimodal Dataset 🚧

A Northern Pashto multimodal dataset with 10 hours of synchronized video, audio, transcriptions, speech segmentation, and ELAN annotations.

License: MDC DLA-1.0

Locale: pbu

Task: NLP

Format: MP4, WAV, EAF

Size: 251.41 GB

MDC Curators

CoVoST 2 Persian - English

The dataset includes audio files in Persian and their translations in English.

License: CC-BY-NC-4.0

Locale: fa, en

Task: MT

Format: MP3, TSV

Size: 1.52 GB

The University of Melbourne

Hawrami Kurdish TTS dataset 1.0

This dataset contains high-quality single-speaker audio recordings in Hawrami Kurdish (Hewrami, ISO 639-3:hac), also known as the Gorani language, intended for building Text-to-Speech (TTS) and Automatic Speech recognition (ASR) systems. The dataset comprises 5 hours and 15 minutes of aligned audio and text data. Hawrami is classified as Definitely Endangered by UNESCO.

License: CC-BY-4.0

Locale: hac

Task: TTS

Format: WAV

Size: 706.11 MB

Common Voice

Common Voice Spontaneous Speech 4.0 - Pashto

A collection of spontaneous responses to questions in Pashto.

License: CC0-1.0

Locale: ps

Task: ASR

Format: MP3

Size: 223.81 MB

RFE/RL

RFE/RL Afghan Pashto News Text Corpus

This dataset is a longitudinal news corpus for the Pashto language, sourced from Radio Azadi, from 2000 to 2026, it contains over 204,000 articles (16M tokens).

License: CC-BY-NC-SA-4.0

Locale: ps

Task: NLP

Format: TXT

Size: 117.68 MB

Keblagh e Azergi

Keblagh-e-Azergi Hazargi literature corpus

This corpus is a collection of more than one hundred thousand tokens of Hazargi language. The corpus contains work of literature, poems, folk and short stories and dramas. The data is being shared with the approval of the authors. It claims to support linguistic research, language technology development and cultural preservation.

License: CC-BY-NC-4.0

Locale: haz

Task: NLP

Format: TXT

Size: 193.28 KB

MDC DatasetsCompensated

Single Speaker Urdu TTS dataset (Female Speaker, South Punjab)

A high-quality single-speaker Urdu text-to-speech (TTS) dataset with 25 hours of paired speech recordings and transcriptions for speech synthesis.

License: MDC DLA-1.0

Locale: ur

Task: TTS

Format: WAV, TSV

Size: 6.02 GB

MDC Curators

CoVoST 2 English - Persian

The dataset includes audio files in English and their translations in Persian.

License: CC-BY-NC-4.0

Locale: en, fa

Task: MT

Format: MP3, TSV

Size: 12.56 GB

Community

Urdu Multi-Speaker TTS Dataset

An Urdu multi-speaker TTS dataset distributed in 36 zip files, each containing audio files and a TSV mapping file, with approximately 10 hours of speech.

License: CC-BY-NC-4.0

Locale: urd

Task: TTS

Format: WEBM, TSV

Size: 514.54 MB

MirasAI

Associated Press Pakistan 74.4 Million Urdu Corpus

A 74.4M-word, 267,887-article Urdu news corpus from Associated Press Pakistan (2015-2026), released for non-commercial research with APP permission.

License: CC-BY-NC-SA-4.0

Locale: urd

Task: NLP

Format: TXT

Size: 143.39 MB

Common Voice

Common Voice Scripted Speech 26.0 - Central Kurdish

A collection of read speech recordings in Central Kurdish.

License: CC0-1.0

Locale: ckb

Task: ASR

Format: MP3

Size: 3.62 GB

MDC DatasetsCompensated

🚧 Hazargi Corpus for Speech Recognition 🚧

A Hazargi ASR dataset with approximately 10 hours of audio with transcriptions.

License: MDC DLA-1.0

Locale: haz

Task: ASR

Format: MP3, TSV

Size: 511.56 MB

The University of Melbourne

Central Kurdish TTS dataset 1.0

This dataset contains high-quality single-speaker audio recordings in Central Kurdish (ckb), intended for building Text-to-Speech (TTS) and Automatic Speech recognition (ASR) systems. The dataset comprises 2 hours and 18 minutes of aligned audio and text data.

License: CC-BY-4.0

Locale: ckb

Task: TTS

Format: wav

Size: 293.45 MB

MEDIAMEN

English–Punjabi (Shahmukhi) Parallel Sentences Corpus (Mediamen Archives)

This parallel sentences corpus containing 30,405 aligned sentence pairs with a total of approximately 0.62 million tokens, curated from the archival materials of Mediamen (Advertising Agency). The sentences were professionally translated from English into Punjabi (Shahmukhi) and are intended to support machine translation, linguistic research, and Punjabi language technology development, particularly for real-world and contemporary language use.

License: CC-BY-NC-4.0

Locale: en-PK, pnb

Task: MT

Format: CSV

Size: 1.08 MB

All Datasets

Not sure where to start?

Persian Literature Corpus by Najwai Sukhan

Multilingual Humanitarian Response Eval (MHRE)

Anjuman-e-Katib Farsi/Persian Literature Corpus

VoxForge - Persian

Dari Literature Corpus by Anjuman e Adabi Nayestan

Common Voice Scripted Speech 26.0 - Persian

Elkhani Hazargi Literature Corpus

RFE/RL Persian News Text Corpus

Associated Press Pakistan 21.4 Million Pashto Corpus

Common Voice Scripted Speech 26.0 - Pashto

🚧 Northern Pashto Multimodal Dataset 🚧

CoVoST 2 Persian - English

Hawrami Kurdish TTS dataset 1.0

Common Voice Spontaneous Speech 4.0 - Pashto

RFE/RL Afghan Pashto News Text Corpus

Keblagh-e-Azergi Hazargi literature corpus

Single Speaker Urdu TTS dataset (Female Speaker, South Punjab)

CoVoST 2 English - Persian

Urdu Multi-Speaker TTS Dataset

Associated Press Pakistan 74.4 Million Urdu Corpus

Common Voice Scripted Speech 26.0 - Central Kurdish

🚧 Hazargi Corpus for Speech Recognition 🚧

Central Kurdish TTS dataset 1.0

English–Punjabi (Shahmukhi) Parallel Sentences Corpus (Mediamen Archives)