Datasets | Mozilla Data Collective

All Datasets

Compensated Datasets

API

Contribute Datasets

Loading datasets...

Terms

FAQs

Participation Guidelines

Responsible Disclosure

News and Documentation

Data and Model Competitions

[email protected]

Terms

FAQs

Participation Guidelines

Responsible Disclosure

News and Documentation

Data and Model Competitions

[email protected]

Mozilla Data Collective is a mission-locked British company, incubated by Mozilla Foundation, and backed by the not-for-profit Mozilla.org

All Datasets

Not sure where to start?

Describe what you are building and the Data Assistant will find the right datasets for you.

Try the Data Assistant

Filtered by

Search results for “Persian 1 Million Text Corpus”

100 Datasets

Jazab Publishers

Jazab Sindhi Newspaper Corpus

The corpus contains 1.07 million tokens from the Jazab a Sindhi Newspaper published from the year 2023-2025. The text consists of the complete newspaper content including headlines, editorials, finance news and advertisements. The newspaper published in Karachi, Pakistan on daily basis.

License: CC-BY-NC-SA-4.0

Locale: snd

Task: NLP

Format: TXT

Size: 2.33 MB

MirasAI

Associated Press Pakistan 5.1 Million Arabic Corpus

A 5.1-million-word Arabic-language corpus of 41,963 Associated Press Pakistan (APP) news articles (2016–2026).

License: CC-BY-NC-SA-4.0

Locale: arb

Task: NLP

Format: TXT

Size: 9.60 MB

MirasAI

Associated Press Pakistan 16.4 Million Sindhi Corpus

A 16.4-million-word Sindhi-language corpus of 58,815 Associated Press Pakistan (APP) news articles (2015–2026).

License: CC-BY-NC-SA-4.0

Locale: snd

Task: NLP

Format: TXT

Size: 23.23 MB

Sindh Sujag Newspaper Agency

Sindh Sujag Newspaper Corpus

The corpus contains approximately 1.2 million tokens from the Sindh Sujag Newspaper Agency published between 2024 and 2025. It includes complete newspaper content such as headlines, editorials, finance news, and advertisements. The newspaper is published daily in Karachi, Pakistan, and represents a comprehensive collection of contemporary Sindhi journalistic writing.

License: CC-BY-4.0

Locale: snd

Task: NLP

Format: TXT

Size: 2.63 MB

CLEAR Global

Kanuri Books Corpus

Randomized sentences from Kanuri-language books by four authors, containing 10,281 sentences and 90,706 words.

License: CC-BY-4.0

Locale: kr

Task: LM

Format: TXT

Size: 545.68 KB

Taruen

Uzbek Proverbs Text Corpus

A corpus of traditional Uzbek proverbs digitized from a 2005 volume, designed for NLP tasks and LLM cultural competence evaluations.

License: CC0-1.0

Locale: uz

Task: NLP

Format: TXT

Size: 142.68 KB

MirasAI

Associated Press Pakistan 1.8 Million Chinese Corpus

A Chinese-language corpus of about 1.8 million Associated Press Pakistan news articles, useful for NLP and language model training.

License: CC-BY-NC-SA-4.0

Locale: zho

Task: NLP

Format: TXT

Size: 2.76 MB

Kaleem Art Press

Kaleem Art Press Saraiki Literature Corpus

This corpus contains approximately one million tokens of Saraiki text curated over the past ten years by Kaleem Art Press. It features a wide range of literary genres, including stories, short stories, novels, fiction, non-fiction, travelogues, poetry, biographies, and historical writings. All content is shared with full author approval. The dataset is intended to support linguistic research, Saraiki language technology development, and the preservation of cultural and literary heritage.

License: CC-BY-NC-4.0

Locale: skr

Task: OTH

Format: TXT

Size: 1.84 MB

RFE/RL

RFE/RL Kazakh News Text Corpus

Longitudinal Kazakh news corpus from Radio Azattyq (2003-2026) with over 139,000 articles and 35M tokens.

License: CC-BY-NC-SA-4.0

Locale: kk

Task: NLP

Format: TXT

Size: 126.81 MB

TAUSCompensated

🚧 Colloquial Parallel Text (English - Hausa) 🚧

100k native-reviewed colloquial sentences translated from US English into Hausa for fine-tuning conversational MT and AI models.

License: TAUS DUA

Locale: en-US, ha-NG

Task: MT

Format: TSV

Size: 4.49 MB

Sindh Line Publishers

The corpus contains 1.029 million tokens from the Sindh Line a Sindhi Newspaper published from the year 2024-2025. The text consists of the complete newspaper content including headlines, editorials, finance news and advertisements. The newspaper published in Karachi, Pakistan on daily basis

License: CC-BY-SA-4.0

Locale: snd

Task: NLP

Format: TXT

Size: 2.22 MB

Community

Sindhi to English Parallel Corpus

A Sindhi–English parallel corpus in JSONL format, designed for machine translation and cross-lingual NLP tasks across diverse real-world domains.

License: CC-BY-NC-4.0

Locale: snd, eng

Task: MT

Format: JSONL

Size: 2.50 MB

Institute of African Digital Humanities

Bamun-French Parallel Corpus 2.0

This dataset is an extended and updated version of the "Bamun-French Parallel Corpus 1.1", a parallel corpus of 4,444 lines in Bamun and French.

License: NOODL-1.0

Locale: bax

Task: MT

Format: TSV

Size: 184.29 KB

MDC DatasetsCompensated

🚧 Hazargi Multimodal Dataset 🚧

A Hazargi multimodal dataset with 10 hours of synchronized video, audio, transcriptions, speech segmentation, and ELAN annotations.

License: MDC DLA-1.0

Locale: haz

Task: NLP

Format: MP4, EAF

Size: 44.00 GB

Institute of African Digital Humanities

Bamun-French Parallel Corpus 1.1

This dataset is an updated version of the "Bamun-French Parallel Corpus", a parallel corpus of texts in Bamun (Shupament) and French.

License: NOODL-1.0

Locale: bax

Task: MT

Format: TSV

Size: 99.78 KB

Sujaak Adbi Sangat

Saraiki Quarterly Magazine Wasson Wehray Corpus

This corpus contains 11,79,200 tokens, from a Saraiki Quarterly Magazine "Wasson Wehray"

License: CC-BY-NC-4.0

Locale: skr

Task: OTH

Format: TXT

Size: 2.09 MB

Institute of Finno-Ugric/Uralic Studies, University of Hamburg

INEL Kamas Speech Corpus

A 14-hour Speech-to-Text dataset for Kamas, an extinct Samoyedic language waiting to be revitalized. Features audio aligned with text transcriptions.

License: CC-BY-NC-SA-4.0

Locale: xas

Task: ASR

Format: TSV, MP3

Size: 376.64 MB

RFE/RL

RFE/RL Crimean Tatar News Text Corpus

Longitudinal Crimean Tatar news corpus from Qırım.Aqiqat (2014-2026) with over 32,000 articles and 7.5M tokens.

License: CC-BY-NC-SA-4.0

Locale: crh

Task: NLP

Format: TXT

Size: 18.35 MB

Community

Awal Tamazight Dataset

A compilation of monolingual and parallel Tamazight language datasets created by CIEMEN as part of the Awal community language technology project.

License: CC-BY-4.0

Locale: zgh

Task: LM

Format: TSV, JSON, TXT

Size: 11.57 MB

Balochi Academy

Eastern Balochi Literature Corpus

A UTF-8 normalized Eastern Balochi literature corpus (~1.9M tokens) covering poetry, folklore, novels, and cultural texts for linguistic research and NLP.

License: CC-BY-NC-4.0

Locale: bgp

Task: NLP

Format: TXT

Size: 949.67 KB

RFE/RL

RFE/RL Azerbaijani News Text Corpus

Longitudinal Azerbaijani & Russian news corpus from Radio Azadlıq (2005-2026) with over 239,000 articles and 37M tokens.

License: CC-BY-NC-SA-4.0

Locale: az,ru

Task: NLP

Format: TXT

Size: 211.65 MB

MDC Curators

CoVoST 2 English - Arabic

The dataset includes audio files in English and their translations in Arabic.

License: CC-BY-NC-4.0

Locale: en, ar

Task: MT

Format: MP3, TSV

Size: 12.56 GB

PROXIMA

Helpful and Harmless RLHF Urdu

Urdu translation of Anthropic's HH-RLHF preference dataset for RLHF/reward-model research in Urdu.

License: CC-BY-NC-SA-4.0

Locale: urd

Task: NLP

Format: Parquet

Size: 10.37 MB

Community

Biomedical Urdu Named Entity Recognition (BioUNER)

Biomedical Urdu Named Entity Recognition dataset for advancing healthcare and clinical NLP research in Urdu.

License: CC-BY-NC-4.0

Locale: urd

Task: NLP

Format: CSV

Size: 336.69 KB