Datasets | Mozilla Data Collective

A literary corpus of Dameli texts preserving the language’s literary traditions, cultural heritage, narratives, and diverse forms of written expression.

Dameli Literature Corpus

License: CC-BY-NC-4.0

Locale: dml

Task: NLP

Format: TXT, DOCX

Size: 9.83 MB

Community

Sindhi Corpus

SdCorpus is a large scale monolingual Sindhi text corpus collected from publicly available web sources for NLP and language model development.

License: CC-BY-NC-4.0

Locale: snd

Task: LM

Format: TXT

Size: 321.00 MB

Longitudinal Kazakh news corpus from Radio Azattyq (2003-2026) with over 139,000 articles and 35M tokens.

RFE/RL Kazakh News Text Corpus

License: CC-BY-NC-SA-4.0

Locale: kk

Task: NLP

Format: TXT

Size: 126.81 MB

Common Voice

Common Voice Spontaneous Speech 4.0 - Afaan Oromo

A collection of spontaneous responses to questions in Afaan Oromo.

License: CC0-1.0

Locale: om

Task: ASR

Format: MP3

Size: 4.04 MB

Kaleem Art Press

Saraiki Literature Corpus

This contains multiple Saraiki Language books of Stories, Short Stories, Novel, Travelogue, Sentences and collection of articles.

License: CC-BY-NC-4.0

Locale: skr

Task: OTH

Format: TXT

Size: 1.84 MB

Common Voice

Common Voice Scripted Speech 26.0 - Afaan Oromo

A collection of read speech recordings in Afaan Oromo.

License: CC0-1.0

Locale: om

Task: ASR

Format: MP3

Size: 527.38 MB

Longitudinal Crimean Tatar news corpus from Qırım.Aqiqat (2014-2026) with over 32,000 articles and 7.5M tokens.

RFE/RL Crimean Tatar News Text Corpus

License: CC-BY-NC-SA-4.0

Locale: crh

Task: NLP

Format: TXT

Size: 18.35 MB

A 16.9M-word Balochi news corpus from Associated Press of Pakistan, covering 59,172 articles published between 2016 and 2026.

Associated Press Pakistan 16.9 Million Balochi Corpus

License: CC-BY-NC-SA-4.0

Locale: bal

Task: NLP

Format: TXT

Size: 22.85 MB

Balochi Academy

Eastern Balochi Literature Corpus

A UTF-8 normalized Eastern Balochi literature corpus (~1.9M tokens) covering poetry, folklore, novels, and cultural texts for linguistic research and NLP.

License: CC-BY-NC-4.0

Locale: bgp

Task: NLP

Format: TXT

Size: 949.67 KB

TAUSCompensated

🚧 Colloquial Parallel Text (English - Hausa) 🚧

100k native-reviewed colloquial sentences translated from US English into Hausa for fine-tuning conversational MT and AI models.

License: TAUS DUA

Locale: en-US, ha-NG

Task: MT

Format: TSV

Size: 4.49 MB

A 74.4M-word, 267,887-article Urdu news corpus from Associated Press Pakistan (2015-2026), released for non-commercial research with APP permission.

Associated Press Pakistan 74.4 Million Urdu Corpus

License: CC-BY-NC-SA-4.0

Locale: urd

Task: NLP

Format: TXT

Size: 143.39 MB

TAUSCompensated

🚧 Colloquial Parallel Text (English - Hausa) 🚧

49k native-reviewed colloquial sentences translated from US English into Hausa for fine-tuning conversational MT and AI models.

License: TAUS DUA

Locale: en-US, ha-NG

Task: MT

Format: TSV

Size: 2.93 MB

6.2 million tokens in the Vallader variety of Romansh from the daily newspaper ”La Quotidiana”.

Vallader Newspaper Corpus

License: CC0-1.0

Locale: rm-vallader

Task: OTH

Format: TSV

Size: 18.71 MB

Taruen

Uzbek Proverbs Text Corpus

A corpus of traditional Uzbek proverbs digitized from a 2005 volume, designed for NLP tasks and LLM cultural competence evaluations.

License: CC0-1.0

Locale: uz

Task: NLP

Format: TXT

Size: 142.68 KB

A 524K token Sylheti corpus featuring drama scripts and cultural texts in UTF-8 format for linguistic research and NLP development.

Sylheti Text Corpus by Haque Publishers

License: CC-BY-NC-4.0

Locale: syl

Task: NLP

Format: TXT

Size: 3.53 MB

Taruen

Kazakh Proverbs Text Corpus

A corpus of traditional Kazakh proverbs from the Babalar sözi series (vols 65-69), designed for NLP and LLM cultural competence evaluations.

License: CC0-1.0

Locale: kk

Task: NLP

Format: TXT

Size: 592.14 KB

Longitudinal Turkmen & Russian news corpus from Azatlyk Radiosy (2009-2026) with nearly 65,000 articles and 16.5M tokens.

RFE/RL Turkmen News Text Corpus

License: CC-BY-NC-SA-4.0

Locale: tk,ru

Task: NLP

Format: TXT

Size: 48.28 MB

1.3 million tokens in the Sutsilvan variety of Romansh from the daily newspaper “La Quotidiana”.

Sutsilvan Newspaper Corpus

License: CC0-1.0

Locale: rm-sutsilv

Task: OTH

Format: TSV

Size: 8.87 MB

6.1 million tokens in the Rumantsch Grischun variety of Romansh from the daily newspaper “La Quotidiana”.

Rumantsch Grischun Newspaper Corpus

License: CC0-1.0

Locale: rm-rumgr

Task: OTH

Format: TSV

Size: 19.03 MB

A diverse Hindi text corpus spanning literary works and news article blogs, curated for NLP research and language modeling tasks.

Hindi Literature & News Corpus

License: CC-BY-NC-SA-4.0

Locale: hin

Task: NLP

Format: TXT

Size: 19.52 MB

Anjuman e Katib

Anjuman-e-Katib Farsi/Persian Literature Corpus

This corpus is a collection of more than one million tokens of Farsi/Persian language. The corpus contains work of literature including novels, fictional and non-fictional, poems and many more. the data is shared with the approval of the authors and aims to support linguistic research, language technology development and its preservation.

License: CC-BY-NC-4.0

Locale: fas

Task: NLP

Format: TXT

Size: 2.82 MB

This dataset is longitudinal news corpus for the Tatar and Bashkir languages, sourced from Azatliq Radiosi, from 2001 to 2025, it contains over 105,000 articles (30M tokens).

RFE/RL Tatar-Bashkir News Text Corpus

License: CC-BY-NC-SA-4.0

Locale: tt,ba,ru

Task: NLP

Format: TXT

Size: 102.44 MB